Read File/Folder

Operation name

Read File/Folder

Function overview

Reads data from a file/folder on Google Cloud Storage onto HULFT Square Storage.

Data model

The data model of this component is XML type.

Properties

= Remarks =

For details on use of variables, refer to Variables.

Basic settings

Item name	Required/Optional	Use of variables	Description	Remarks
Name	Required	Not available	Enter a name that is used on the script canvas.

Required settings

Item name	Required/Optional	Use of variables	Description	Remarks
Destination	Required	Not available	Select a connection resource. Add A new connection resource can be added. Edit list Connection resource settings can be edited in > HULFT INTEGRATE > Connections.
Bucket name of the read source	Required	Available	Select or enter the bucket to be read.
Folder path of the read source	Required	Available	Select or enter the folder path on Google Cloud Storage.	Specify a slash "/" at the beginning and the end. Consecutive slashes "//" can't be included. Note Specify an absolute path of Google Cloud Storage for the folder path.
File/Folder name of the read source	Required	Available	Select or enter the file/folder on Google Cloud Storage.	Wildcards ("*" and "?") can be used.
Save destination directory	Required	Available	Enter a directory path of HULFT Square Storage. The Browse button launches a directory chooser that allows you to select a directory.	Case-insensitive. Note Specify an absolute path of HULFT Square Storage for the directory path. With the exception of the path separator "/", the Restricted characters of HULFT Square Storage can't be used.
Overwrite when one exists	Optional	Not available	Select whether or not to overwrite a file when a file with the same name exists in the specified save destination directory. Selected Overwrite a file when a file with the same name exists in the specified save destination directory. Not selected (Default) When a file with the same name exists in the specified save destination directory, raise an error without overwriting.

Property actions

Item name	Description	Remarks
Update the list of bucket names of the read source	Get bucket names in the specified destination and set them in Bucket name of the read source.	If Destination is edited or a bucket has been edited on Google Cloud Storage, the changes can be reflected using this property action. Disabled when Destination isn't specified.
Update the list of folder paths of the read source	Get folder paths in the specified bucket and set them in Folder path of the read source.	If Bucket name of the read source is edited or folder path has been edited on Google Cloud Storage, the changes can be reflected using this property action. Disabled when Destination or Bucket name of the read source isn't specified.
Update the list of file/folder names of the read source	Get file/folder names in the specified folder path and set them in File/Folder name of the read source.	If Folder path of the read source is edited or file/folder name has been edited on Google Cloud Storage, the changes can be reflected using this property action. Disabled when Destination, Bucket name of the read source, or Folder path of the read source isn't specified.

Data processing method

Item name	Required/Optional	Use of variables	Description	Remarks
Data processing method	Required	Not available	Select a data processing method. Apply script settings (Default) The setting for mass data processing in the script properties is applied to the connector. Do not perform mass data processing Mass data processing isn't performed. Perform mass data processing Mass data processing is performed.

Item name

Required/Optional

Use of variables

Description

Remarks

Data processing method

Required

Not available

Select a data processing method.

Apply script settings

(Default)

The setting for mass data processing in the script properties is applied to the connector.

Do not perform mass data processing

Mass data processing isn't performed.

Perform mass data processing

Mass data processing is performed.

Comment

Item name	Required/Optional	Use of variables	Description	Remarks
Comment	Optional	Not available	You can write a short description of this connector.

Schemas

Input schema

None.

Output schema

<?xml version="1.0"?>
<bucket name="" status="" >
  <file errorcode="" localpath="" name="" remotepath="" size="" status="" statuscode="" type="" updated=""/>
</bucket>

Element name	Attribute name	Description	Remarks
bucket	-	Appears once for each retrieved bucket.
	name	Outputs the name of the retrieved bucket.
	status	Outputs the status of the retrieved bucket. Exist The bucket exists. ErrorBucketNotFound The specified Bucket name of the read source doesn't exist on Google Cloud Storage. The specified bucket name may be incorrect.
file	-	Appears once for each read file/folder.
	errorcode	When an error occurs on Google Cloud Storage, outputs the API error code defined on Google Cloud Storage.	If no error has occurred on Google Cloud Storage, the value is blank. For more details about API error code defined on Google Cloud Storage, refer to the following Web site: https://cloud.google.com/storage/docs/json_api/v1/status-codes
	localpath	Outputs the local path of the read file/folder.	If status is "RemoteFileNotFound", the value is blank. If status is "Error", the value may be blank.
	name	Outputs the name on Google Cloud Storage of the read file/folder.	If status is "RemoteFileNotFound", the value is blank. If status is "LocalFileNotFound", the value is blank. If status is "Error", the value is blank.
	remotepath	Outputs the file path on Google Cloud Storage of the read file/folder.	If status is "LocalFileNotFound", the value is blank. If status is "LocalFileExist" and type is "Folder", the value is blank. If status is "Downloaded" and type is "Folder", the value is a blank. If status is "Error", the value may be blank.
	size	When the read file/folder is a file, outputs the file size. The unit is byte.	When the file/folder wasn't read correctly, the value is blank. If it's a folder, the value is blank.
	status	Outputs the status of the read file/folder. Downloaded The file/folder has been read. RemoteFileNotFound The specified File/Folder name of the read source doesn't exist in the folder path specified in Folder path of the read source on Google Cloud Storage. LocalFileNotFound The specified Save destination directory doesn't exist. LocalFileExist A file/folder with the same name as the Folder path of the read source or File/Folder name of the read source exists in the Save destination directory. However, when Overwrite file when one exists is selected, the file is overwritten and then the status changes to "Downloaded". Error The file/folder hasn't been read, because an error has occurred.
	statuscode	When an error occurs on Google Cloud Storage, outputs the API status code defined on Google Cloud Storage.	If no error has occurred on Google Cloud Storage, the value is blank. For more details about the API status codes defined on Google Cloud Storage, refer to the following Web site: https://cloud.google.com/storage/docs/json_api/v1/status-codes
	type	Outputs the type of read file/folder. File File Folder Folder
	updated	Outputs the last updated date when the read file/folder is a file.	When the file/folder wasn't read correctly, the value is blank. If it's a folder, the value is blank.

Loading schema in Mapper

The schema is loaded automatically.

= Remarks =

For details, refer to Edit Schema.

Mass data processing

Mass data processing is supported.

Parallel Stream Processing

PSP isn't supported.

Available component variables

Component variable name	Description	Remarks
count	The number of target files/folders is stored.	The default value is null.
message_category	When an error occurs, the category of the message code corresponding to the error is stored.	The default value is null.
message_code	When an error occurs, the code of the message code corresponding to the error is stored.	The default value is null.
message_level	When an error occurs, the severity of the message code corresponding to the error is stored.	The default value is null.
error_type	When an error occurs, the error type is stored.	The default value is null. The format of the error type is as follows. Example: java.io.FileNotFoundException
error_message	When an error occurs, the error message is stored.	The default value is null.
error_trace	When an error occurs, the trace information for the error is stored.	The default value is null.

Message codes, exception messages, and limitations

Connector	Message code	Exception message	Limitations
Messages and limitations of the Google Cloud Storage connector