Read File

Operation Name

Read File

Function overview

Specifies files on Microsoft Azure Storage BLOB, and writes to the directory on DataSpider file system.

Data model

Data model of this component is XML type.

Properties

For information about using variables, refer to "variables".

Basic settings

Item name	Required / Optional	Use of Variables	Description	Remarks
Name	Required	Not available	Enter the name on the script canvas.

Required settings

Item name	Required / Optional	Use of Variables	Description	Remarks
Destination	Required	Not available	Select Global Resources. [Add]: Adds new global resource. [Edit list]: Global resource settings can be edited with "Edit Resource list".
Container name	Required	Available	Specify the container.	Refer to "Notes" for naming conventions of containers in Microsoft Azure.
File name	Required	Available	Specify the file.	Wildcards ("*" and "?") can be used in the file name. Distinguishes on upper/lower cases.
Get file information only	Optional	Not available	Select whether to get file information only without downloading the file to local system. [Checked]: Gets file information only without downloading the file to local system. [Not Checked]: (default) Download the file to local system.
Local directory	Required	Available	Specify a directory path of DataSpider filesystem. Select a directory in the directory Selectr that is launched by pressing [Browse] button.	The specified directory path needs to be absolute path relative to the DataSpider filesystem root. Except for the '/' character, characters not allowed to be used in DataSpider filesystem cannot be specified in the file path. Disabled if [Get file information only] is checked. Does not distinguish on upper/lower cases.
Overwrite	Optional	Not available	Select whether to overwrite when file with the same name exists on the specified local directory. [Checked]: (default) Overwrite when file with the same name exists on the specified local directory. [Not Checked]: Raises an error without overwriting when file with the same name exists on the specified local directory.	Disabled if [Get file information only] is checked.

Comment

Item name	Required / Optional	Use of Variables	Description	Remarks
Comment	Optional	Not available	Description for the usage of this adapter can be entered. When a description is provided, it will be reflected in the output of specification document.

Schema

Input Schema

None.

Output Schema

<?xml version="1.0"?>
<server>
  <container name="" status="" >
    <file etag="" localpath="" name="" size="" status="" type="" updated="" uri="" />
  </container>
</server>

Element name	Attribute name	Description	Remarks
server	-
container	-
	name	Outputs the name of the container of the retrieved file.
	status	Outputs the status of the container of the retrieved file. [Exist]: State that the container exists. [ErrorContainerNotFound]: State that the container does not exist.The specified [Container name] may be incorrect. [Error]: The existence of the container cannot be verified due to certain error.
file	-	Appears for each retrieved file.
	etag	Outputs the ETag of the retrieved file.	Output when the specified containers and files are successfully retrieved.
	localpath	Outputs the local path of the retrieved file.	Not output when [Get file information only] is checked.
	name	Outputs the name of the retrieved file.
	size	Outputs the size of the file retrieved.The unit is byte.	Output when the specified containers and files are successfully retrieved.
	status	Outputs the status of the retrieved file. [Exist]: State that the container exists. [ErrorRemoteFileNotFound]: State that the file on Microsoft Azure Storage BLOB does not exist. [ErrorLocalFileNotFound]: State that the specified [Local directory] does not exist, or the specified file does not exist. [ErrorLocalFileExist]: State that [Overwrite] is [Not Checked] and the file with the same name exists in the specified [Local directory]. [Error]: Could not read the file due to certain error.
	type	Outputs the type of the retrieved file.	Output when the specified containers and files are successfully retrieved.
	updated	Outputs the last updated date of the retrieved file.	The output format is yyyyMMddHHmmss.Example: "20071016131522" means 1:15 pm and 22 seconds on October 16, 2007. Time zone is the time zone of DataSpiderServer. Output when the specified file is successfully updated.
	uri	Outputs the URI of the retrieved file.	Output when the specified containers and files are successfully retrieved.

Loading schema in Mapper

Schema is loaded automatically.

For any details, please refer to the 'Editing Schema'

Mass data processing

Mass data processing is not supported.

PSP Usage

PSP is not supported.

Available component variables

Component variable name	Description	Remarks
count	Returns the number of retrieved files.	The value defaults to null.
message_category	Stores the category to which corresponding message code belongs to, when an error occurs.	The value defaults to null.
message_code	Stores its corresponding message code of occurred error.	The value defaults to null.
message_level	Stores the severity of the corresponding message code of the occurred error.	The value defaults to null.
error_type	Stores the type of the occurred error.	The value defaults to null. Error is represented in the format depicted below. Example:java.io.FileNotFoundException The message may vary depending on the DataSpider Servista version.
error_message	Stores the error message for the occurred error.	The value defaults to null. The message may vary depending on the DataSpider Servista version.
error_trace	Stores stack trace of the occurred error.	The value defaults to null. The message may vary depending on the DataSpider Servista version or the client application used.

Specification Limits

None.

Main exceptions

Exception name	Causes	Solution
ResourceNotFoundException Resource definition could not be found.Name: []	[Destination] is not specified.	Specify [Destination].
ResourceNotFoundException Resource definition could not be found.Name: [<name of Global Resources>]	Resource definition selected in [Destination] is not found.	Check the global resource specified in [Destination].
InvalidPropertyConfigurationException <Property name> is not specified.	[<Property name>] is not specified.	Specify the [<Property name>].
com.microsoft.windowsazure.services.core.storage.StorageException	Cannot be connected due to the following causes. There's a mistake in [Account] or [Access key] in the global resource specified in [Destination]. There are double-byte characters in [Account] in the global resource specified in [Destination].	Check the settings and connection status of the global resource specified in [Destination] .
java.security.InvalidKeyException Storage Key is not a valid base64 encoded string.	[Access key] of the global resource specified in [Destination] is incorrect.	Check the settings of the global resource specified in [Destination] .

Notes

Connects in HTTPS.

Naming conventions of containers in Microsoft Azure.
- Name of containers must conform to the following 4 rules.
  1. Container names begin with an alphabetic character or number,and can only include alphabetic characters, numbers or hyphens (-).
  2. A hyphen (-) must be used immediately before and after alphabetic characters or numbers.
  3. Alphabetic characters must be lower case letters.
  4. The length of container names must be 3 to 63 characters.
If non-connection-related errors (those not mentioned in section "main exception" error) occurs at run time, subsequent operations are performed having the status attribute of the file element set to "Error."

Retrieval of large volume files performed using multiple threads may cause operational error and the status attribute of the file element in the appropriate file will be set to "Error"
To remedy this situation, decrease the number of thread executes the operation concurrently.