Read File/Folder (Data)

Operation name

Function overview

Reads the files/folder on Amazon S3.

Data model

The data model of this component is XML type for input data and table model type for output data.

= Remarks =

For more details on input/output schema, refer to Schemas.

Tutorials

For details about how to create a connection to Amazon S3, refer to the following tutorial:

Create a connection to Amazon S3

Properties

= Remarks =

For details on use of variables, refer to Variables.

Basic settings

Item name	Required/Optional	Use of variables	Description	Remarks
Name	Required	Not available	Enter a name that is used on the script canvas.
Input data	Required	Not available	Select a component on the script canvas.

Required settings

Item name	Required/Optional	Use of variables	Description	Remarks
Destination	Required	Not available	Select a connection resource. Add A new connection resource can be added. Edit list Connection resource settings can be edited in > HULFT INTEGRATE > Connections.
Include permissions in result	Optional	Not available	Select whether to include file access right information in the results. Selected Include file access right information in the results. Not selected (Default) Don't include file access right information in the results.

Item name

Required/Optional

Use of variables

Description

Remarks

Destination

Required

Not available

Select a connection resource.

Add: A new connection resource can be added.
Edit list: Connection resource settings can be edited in > HULFT INTEGRATE > Connections.

Include permissions in result

Optional

Not available

Select whether to include file access right information in the results.

Selected

Include file access right information in the results.

Not selected

(Default)

Don't include file access right information in the results.

Decryption settings

Item name	Required/Optional	Use of variables	Description	Remarks
Secret Key	Optional	Available	Enter the file path of the secret key to decrypt the client encrypted file. The Browse button launches a file chooser that allows you to select a file.	If the secret key isn't specified, the client encrypted file will be read without decryption. Note Specify an absolute path of HULFT Square Storage for the file path. With the exception of the path separator "/", the Restricted characters of HULFT Square Storage can't be used.

Item name

Required/Optional

Use of variables

Description

Remarks

Secret Key

Optional

Available

Enter the file path of the secret key to decrypt the client encrypted file.

The Browse button launches a file chooser that allows you to select a file.

If the secret key isn't specified, the client encrypted file will be read without decryption.

Note

Specify an absolute path of HULFT Square Storage for the file path.
With the exception of the path separator "/", the Restricted characters of HULFT Square Storage can't be used.

Data processing method

Item name	Required/Optional	Use of variables	Description	Remarks
Data processing method	Required	Not available	Select a data processing method. Apply script settings (Default) The setting for mass data processing in the script properties is applied to the connector. Do not perform mass data processing Mass data processing isn't performed. Perform mass data processing Mass data processing is performed.

Item name

Required/Optional

Use of variables

Description

Remarks

Data processing method

Required

Not available

Select a data processing method.

Apply script settings

(Default)

The setting for mass data processing in the script properties is applied to the connector.

Do not perform mass data processing

Mass data processing isn't performed.

Perform mass data processing

Mass data processing is performed.

Comment

Item name	Required/Optional	Use of variables	Description	Remarks
Comment	Optional	Not available	You can write a short description of this connector.

Schemas

Input schema

<?xml version="1.0"?>
<server>
  <bucket name="" >
    <file remotepath="" type="" />
  </bucket>
</server>

Element name	Attribute name	Description	Remarks
server	-
bucket	-	Specify for each bucket to read.
bucket	name	Specify a name of a bucket to read.
file	-	Specify for each file/folder to read.
	remotepath	Specify a path on Amazon S3 of the file/folder to read.	The delimiter of a path is "/". A path needs to start with "/". A path of a folder doesn't need to end with "/". The root folder ("/") can't be read.
	type	Specify a type of a file/folder to read. File: File Folder: Folder

= Remarks =

Output data of Get File/Folder List operation can directly be input to this operation.

Output schema

<?xml version="1.0"?>
<table>
  <row>
    <column>bucketname</column>
    <column>etag</column>
    <column>filename</column>
    <column>public</column>
    <column>remotepath</column>
    <column>size</column>
    <column>status</column>
    <column>storageclass</column>
    <column>type</column>
    <column>updated</column>
    <column>data</column>
    <column>contenttype</column>
  </row>
  <row>
    :
  </row>
</table>

Column name	Description	Remarks
bucketname	Outputs the bucket name of the retrieved file/folder.	The value of the name attribute of the bucket element specified in the input data.
etag	Outputs the ETag of the retrieved file/folder.	If file/folder couldn't be retrieved successfully, the value is blank.
filename	Outputs the name of the retrieved file/folder.	The value generated from the remotepath attribute of the file element specified in the input data.
public	Outputs the access right of the retrieved file/folder. true Public false Private	If Include permissions in result is selected, then this is output. If file/folder couldn't be retrieved successfully, the value is blank.
remotepath	Outputs file path on Amazon S3 of the retrieved file/folder.	The value of the remotepath attribute of the file element specified in the input data.
size	Outputs the size of the retrieved file/folder. The unit is byte.	If file/folder couldn't be retrieved successfully, the value is blank. In the case of a folder, "0" is always output.
status	Outputs the status of the retrieved file/folder. Exist The file/folder exists. Virtual The folder specified in remotepath doesn't exist on Amazon S3 but files and/or folders exist in the folder. ErrorBucketNotFound The bucket specified in bucketname doesn't exist on Amazon S3. ErrorRemoteFolderNotFound The folder specified in remotepath doesn't exist on Amazon S3. ErrorRemoteFileNotFound The file specified in remotepath doesn't exist on Amazon S3. ErrorClientEncryption Reading has failed because the specified Secret Key is incorrect. Error The file/folder couldn't be read due to an error.
storageclass	Outputs Storage Class of the retrieved file/folder. STANDARD Standard REDUCED_REDUNDANCY RRS(Reduced Redundancy Storage) STANDARD_IA Standard_IA	If file/folder couldn't be retrieved successfully, the value is blank.
type	Outputs the classification of the retrieved file/folder. File File Folder Folder	The value of the type attribute of the file element specified in the input data.
updated	Outputs the last updated date of the retrieved file/folder.	If file/folder couldn't be retrieved successfully, the value is blank.
data	Outputs the received file contents in binary format.	In the case of files, values are output. In the case of folders, values aren't output.
contenttype	Outputs the Content-Type that is set on Amazon S3 of the received file/folder.	If file/folder couldn't be retrieved successfully, the value is blank.

= Remarks =

One row is output per a file or a folder.

Loading schema in Mapper

The schema is loaded automatically.

= Remarks =

For details, refer to Edit Schema.

Mass data processing

Mass data processing is supported.

Parallel Stream Processing

PSP isn't supported.

Available component variables

Component variable name	Description	Remarks
count	The number of retrieved folders and files is stored.	The default value is null. The sum of "folder_count" and "file_count".
folder_count	The number of retrieved folders is stored.	The default value is null.
file_count	The number of retrieved files is stored.	The default value is null.
message_category	When an error occurs, the category of the message code corresponding to the error is stored.	The default value is null.
message_code	When an error occurs, the code of the message code corresponding to the error is stored.	The default value is null.
message_level	When an error occurs, the severity of the message code corresponding to the error is stored.	The default value is null.
error_type	When an error occurs, the error type is stored.	The default value is null. The format of the error type is as follows. Example: java.io.FileNotFoundException
error_message	When an error occurs, the error message is stored.	The default value is null.
error_trace	When an error occurs, the trace information for the error is stored.	The default value is null.

Message codes, exception messages, and limitations

Connector	Message code	Exception message	Limitations
Messages and limitations of the Amazon S3 connector