Write File/Folder(Data)

Operation Name

Write File/Folder(Data)

Function overview

Write file/folder to Amazon S3.

Data Model

Data model of this type of component is table model type for input and XML type for output.
Refer to "Schema" for input/output schema.

Properties

For information about using variables, refer to "variables".
Basic settings
Item name Required / Optional Use of Variables Description Remarks
Name Required Not available Enter the name on the script canvas.  
Input data Required Not available Select the component on the script canvas.  
Required settings
Item name Required / Optional Use of Variables Description Remarks
Destination Required Not available Select Global Resources.
  • [Add]:
    Adds new global resource.
  • [Edit list]:
    Global resource settings can be edited with "Edit Resource list".
 
Create bucket when it doesn't exist Optional Not available Select whether to create when the specified Bucket does not exist.
  • [Checked]:
    If specified Bucket does not exist then create Bucket.
  • [Not Checked]: (default)
    If there is no specified Bucket, then do not create Bucket and set error.
 
Retrieve ACL information Optional Not available Select whether to include file access right information in the results.
  • [Checked]:
    Include file access right information in the results.
  • [Not Checked]: (default)
    Do not include file access right information in the results.
 
Encryption settings
Item name Required / Optional Use of Variables Description Remarks
Encryption Required Not available Select whether to encrypt file or not.
  • [No]:(default)
    Does not encrypt.
  • [Server side]
    Encrypts in Amazon S3 after uploading.
  • [Client side]
    Writes in Amazon S3 after encryption.
Public Key Required Available Enter file path of public key to encrypt.

Click [Browse] button to activate the file Selectr and Select the file.
  • Valid, if you select [Client side] in [Encryption].
  • [Public Key] needs to be the key created with RSA method of cryptographic algorithm.
  • Specify the absolute path of the DataSpider file system as the file path.
  • Constrained Characters of DataSpider File System except for path separator "/" cannot be used.
Data processing method
Item name Required / Optional Use of Variables Description Remarks
Mass data processing Required Not available Select a data processing method.
  • [Use script settings]: (default)
    Apply mass data processing settings of script property to adapter.
  • [Disable]:
    Mass data processing is not performed.
  • [Enable]:
    Mass data processing is performed.
 
Comment
Item name Required / Optional Use of Variables Description Remarks
Comment Optional Not available You can write a short description of this adapter.
The description will be reflected in the specifications.
 

Schema

Input Schema

<?xml version="1.0"?>
<table>
  <row>
    <column>bucketname</column>
    <column>public</column>
    <column>remotepath</column>
    <column>storageclass</column>
    <column>type</column>
    <column>data</column>
    <column>contenttype</column>
  </row>
  <row>
    :
  </row>
</table>
Column Name Description Remarks
bucketname Specify a name of a bucket to write.  
public Specify access controls of a file/folder to write.
  • true: Public
  • false: Private
  • If setting "public", "READ" right is granted to "AllUsers Group"
  • If you omit the "false" is set.
remotepath Specify a path on Amazon S3 to write.
  • The delimiter of a path is "/".
  • A path needs to start with "/".
  • A path of a folder do not need to end with "/".
  • The root folder ("/") cannot be written to.
storageclass Specify a Storage Class of a file/folder to write.
  • STANDARD: Standard
  • REDUCED_REDUNDANCY: RRS(Reduced Redundancy Storage)
  • If you omit the "Standard" is set.
type Specify a type of a file/folder to write.
  • File: File
  • Folder: Folder
 
data Specify the file contents to write in Base64 format or binary format.
  • If you omit an empty file is written.
  • Data is ignored when a folder is written.
contenttype Specify a Content-Type of a file/folder to write.
  • If you omit a Content-Type of the file to write, the Content-Type related to a name of the file is set.
  • If you omit a Content-Type of the folder to write, application/octet-stream is usually set.
One row is input per a file or a directory.

Output Schema

<?xml version="1.0"?>
<server>
  <bucket name="" status="" >
    <file etag="" name="" public="" remotepath="" size="" status="" storageclass="" type="" updated="" />
  </bucket>
</server>
Element Name Attribute Name Description Remarks
server -    
bucket - Appear in each Bucket which is written.  
name Output Bucket name of each written file/folder.
  • The value of bucketname specified in the input data.
status Output the status of the retrieved bucket.
  • Exist: Status that the Bucket exists.
  • ErrorBucketNotFound: The status that on Amazon S3, the specified Bucket does not exist in the specified bucketname.
  • Error: The status that because error has occurred, Bucket could not be created.
 
file - Appear in each written file/folder.  
etag Output ETag of each written file/folder.
  • If file/folder is not written correctly, the value is blank.
name Output name of written file/folder.
  • The value generated from remotepath specified in the input data.
public Output access right of retrieved files/folder.
  • true: Public
  • false: Private
  • If file/folder is not written correctly, the value is blank.
  • Should the "READ" authority be granted to "AllUsers Group", the access authority becomes "Public".
  • If [Retrieve ACL information] is [Not Checked] then the value is blank.
remotepath Output file path of Amazon S3 written file/folder.
  • The value of remotepath specified in the input data.
size Output size of written file / folder. The unit is byte.
  • Output when file/folder is written correctly.
  • In case of folder, "0" is output.
status Output status of written file/folder.
  • Exist: Status that file/folder exists.
  • ErrorClientEncryption:The status that failed to read, due to existence of error in the specified [Public Key].
  • Error: The status that because an error occurred file/ folder could not be written.
 
storageclass Output Storage Class of the gotten file/folder.
  • STANDARD: Standard
  • REDUCED_REDUNDANCY: RRS(Reduced Redundancy Storage)
  • If file/folder is not written correctly, the value is blank.
type Output classification of written file/folder.
  • File: File
  • Folder: Folder
 
updated Output the latest update date of written file/folder.
  • If file/folder is not written correctly, the value is blank.
  • Output by the format of [xmlfw.daterenderingformat] inside system property.If it has not been set, the default format will be "yyyy-MM-dd'T'HH: mm: ss.SSSZZ "is printed on.
    Example :2007-10-16T13: 15:22.738 +0900
  • Time zone is the time zone of DataSpiderServer.

Loading schema in Mapper

Schema is loaded automatically.
See "Edit Schema" for details with regards to defining a schema.

Mass data processing

Mass data processing is supported.

Transaction

Transaction is not supported.

Usage in a PSP script

Cannot be used in a PSP script.

Available component variables

Component variable name Description Remarks
count Return the number which is the total number of the written folder number and file number.
  • The value defaults to null.
  • The sum of "folder_count" and "file_count".
folder_count Return the number of written folder.
  • The value defaults to null.
file_count Return the number of written file.
  • The value defaults to null.
message_category Stroes the category to which corresponding message code belongs to, when an error occurs.
  • The value defaults to null.
message_code Stores its corresponding message code of occured error.
  • The value defaults to null.
message_level Stores the severity of the corresponding message code of the occured error.
  • The value defaults to null.
error_type Stores the type of the occured error.
  • The value defaults to null.
  • Error is represented in the format depicted below.
    Example:java.io.FileNotFoundException
  • The message may vary depending on the DataSpider Servista version.
error_message Stores the error message for the occured error.
  • The value defaults to null.
  • The message may vary depending on the DataSpider Servista version.
error_trace Stores stack trace of the occurred error.
  • The value defaults to null.
  • The message may vary depending on the DataSpider Servista version or the client application used.

Client encryption key length limitation

When using the first client encryption, error message of encryption may appear.
Client encryption requires maximum 256 Bit. However, Java Cryptography Extension(JCE) jurisdiction policy file restricts maximum key length used in encryption to 128 Bit.
You can dodge this restriction by installing Java Cryptography Extension(JCE) unlimited intense jurisdiction policy file.

Specification limitations

None.

Main exceptions

Exception name Causes Solution
ResourceNotFoundException
Resource definition could not be found.Name: []
[Destination] is not specified. Please specify [Destination].
ResourceNotFoundException
Resource definition could not be found.Name: [<name of Global Resources>]
Resource definition selected in [Destination] is not found. Please check the global resource specified in [Destination].
IllegalArgumentException
Invalid input data: <message>
The <message> is one of the following messages.
  • Bucket name is not specified.
  • Remote path is not specified.
  • File/folder type is not specified.
  • File/folder type<value> is invalid.
  • Storage Class<value> is invalid.
Invalid data is input. Check the input data. Please check Schema.
InvalidPropertyConfigurationException
Public key is not set.
[Public Key] is not specified. Specify [Public Key], or Specify [Client side] value to [Encryption].
InvalidPropertyConfigurationException
Specified public key does not exist.
File path specified in [Public Key] does not exist. Specify the existing file path to [Public Key].
InvalidPropertyConfigurationException
Specified public key is directory.
[Public Key] is set as directory path. Specify file path for [Public Key].
Status Code: 403, AWS Request ID: XXXXXXXXXXXXXXXX, AWS Error Code: InvalidAccessKeyId, AWS Error Message: The AWS Access Key Id you provided does not exist in our records., S3 Extended Request ID: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [Access Key ID] of global resources specified in [Destination] is not correct. Check the settings of the global resource specified in [Destination].
Status Code: 403, AWS Request ID: XXXXXXXXXXXXXXXX, AWS Error Code: SignatureDoesNotMatch, AWS Error Message: The request signature we calculated does not match the signature you provided. Check your key and signing method., S3 Extended Request ID: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [Secret Access Key] of the global resources specified in [Destination] is incorrect. Check the settings of the global resource specified in [Destination].
com.amazonaws.AmazonClientException
Unable to execute HTTP request: <endpoint>
[Endpoint] of the global resource specified in [Destination] is incorrect. Check the settings of the global resource specified in [Destination].

Notes