Write File/Directory

Operation name

Function Overview

Write the specified file or a directory which exists in DataSpider File System to the specified container in Microsoft Azure BLOB Storage.

Data Model

XML type.

Properties

For details on use of variables, refer to “Variables”.

Basic settings

Item name	Required/Optional	Use of variables	Description	Remarks
Name	Required	Not available	Enter the name to show on the script canvas.

Required settings

Item name	Required/Optional	Use of variables	Description	Remarks
Destination	Required	Not available	Select a global resource. [Add]: Add a new global resource. [Edit list]: Global resource settings can be edited in “Edit resource list” screen.
Local directory	Required	Available	Enter a directory path of DataSpider File System. You can click [Browse] button to launch the directory chooser and find the directory.	Specify the directory path as an absolute path of DataSpider File System. Except for the path separator “/”, restricted characters of DataSpider File System cannot be used. Case-insensitive.
Local file/directory name	Required	Available	Select or enter a source (DataSpiderServer side) file/directory.	Wild cards ("*" and "?") can be used. Case-sensitive.
Set directory as target as well	Optional	Not available	Select whether or not to set a directory of [Local file/directory name] as target as well. [Checked]: Files and directories of [Local file/directory name] will be targets. [Not checked]: (default) Files of [Local file/directory name] will be targets.
Container name	Required	Available	Specify the container.	For details on naming conventions of container name in Microsoft Azure, refer to ”Notes”. When specifying root container, create a new container named $root and select it.
Create container when it doesn't exist	Optional	Not available	Select whether or not to create when the specified container does not exist. [Checked]: (default) Container will be created when the specified container does not exist. [Not checked]: When the specified container does not exist, it will be an error without creating it.	When checked, the visibility attribute of the created container will be “private”.
Virtual directory	Optional	Available	Select or enter a virtual directory.	When omitted, it will be root directory “/”. When the specified virtual directory does not exist, it will be created.
Overwrite when file exists	Optional	Not available	Select whether or not to overwrite when a file with the same name exists in the specified container. [Checked]: (default) Overwrite when a file with the same name exists in the specified container. [Not checked]: When a file with the same name exists in the specified container, it will be an error without overwriting.

Property actions

Item name	Description	Remarks
Update container name list	Get container names from the specified destination and set them in [Container name].	When [Destination] is changed or the container in Microsoft Azure BLOB Storage is modified, those changes can be reflected with this property action. Disabled when [Destination] is not specified.
Update virtual directory list	Get virtual directories in the specified container and set them in [Virtual directory].	When [Container name] is changed or the virtual directory in Microsoft Azure BLOB Storage is modified, those changes can be reflected by this property action. Disabled when [Destination] or [Container name] is not specified.

Data processing method

Item name	Required/Optional	Use of variables	Description	Remarks
Data processing method	Required	Not available	Select data processing method. [Apply script settings]: (default) Apply the mass data processing settings of the script property to the adapter. [Do not perform mass data processing]: Mass data processing will not be performed. [Perform mass data processing]: Mass data processing will be performed.

Comment

Item name	Required/Optional	Use of variables	Description	Remarks
Comment	Optional	Not available	You can write a short description of this adapter. The description will be reflected in the specifications.

Schema

Input schema

None.

Output schema

<?xml version="1.0"?>
<server>
  <container name="" status="" >
    <file etag="" localpath="" name="" size="" status="" type="" updated="" uri="" />
  </container>
</server>

Element name	Attribute name	Description	Remarks
server	-
container	-
	name	Outputs the container name of the written file.
	status	Outputs the container status of the written file. [Exist]: Status that container exists. [Created]: Status that the container was created newly. Status that [Create container when it doesn't exist] is [Checked] and the specified container was created successfully. [ErrorContainerNotCreated]: Status that failed to create the container. Status that [Create container when it doesn't exist] is [Checked] and failed to create the specified container failed. [ErrorContainerNotFound]: Status that the container does not exist. Status that [Create container when it doesn't exist] is [Not checked] and the specified container does not exist. Error: Status that the existence of container could not be confirmed due to an error.
file	-	Appears for each written file.
	etag	Outputs the ETag of the written file.	Output when the specified file is updated normally.
	localpath	Outputs the local path of the written file.
	name	Outputs the file name in Azure BLOB Storage.
	size	Outputs the size of the written file. Unit is byte.	Output when the specified file is updated normally.
	status	Outputs the status of the file. [Updated]: Status that the file is updated normally. [ErrorLocalFileNotFound]: Status that the specified [Local directory] or the specified [File name] does not exist. [ErrorRemoteFileExist]: Status that [Overwrite if file exists] is [Not checked] and a file of the same name exists in the specified container. [Error]: Status that the file could not be read due to an error.
	type	Outputs the type of the written file.	Output when the specified file is updated normally.
	updated	Outputs the date and time of the last updated file in which write has been performed.	Output in yyyyMMddHHmmss format. Example: “20071016131522” means October 16, 2007 13:15:22. Time zone will be in DataSpiderServer's time zone. Output when the specified file is updated normally.
	uri	Outputs the URI of the written file.	Output when the specified file is updated normally.

Loading schema in Mapper

Schema is loaded automatically.

Refer to “Edit Schema” for more details.

Mass Data Processing

Mass Data Processing is supported.

Transaction

Transaction is not supported.

PSP Usage

PSP is not supported.

Available Component Variables

Component variable name	Description	Remarks
count	The number of written files is stored.	Default value is null.
message_category	When an error occurs, the corresponding message code's category is stored.	Default value is null.
message_code	When an error occurs, the corresponding message code is stored.	Default value is null.
message_level	When an error occurs, the corresponding message code's severity is stored.	Default value is null.
error_type	When an error occurs, the error type is stored.	Default value is null. The error type will be in the following format. Example: “java.io.FileNotFoundException” The contents that will be stored may vary depending on DataSpider Servista versions.
error_message	When an error occurs, the error message is stored.	Default value is null. The contents that will be stored may vary depending on DataSpider Servista versions.
error_trace	When an error occurs, the error's trace information is stored.	Default value is null. The contents that will be stored may vary depending on DataSpider Servista versions and client applications.

Specification Limits

type attribute of file element of a written file will be “File”.

Major Exceptions

Exception name	Cause	Solution
ResourceNotFoundException Resource definition is not found. Name: []	[Destination] is not specified.	Specify [Destination].
ResourceNotFoundException Resource definition is not found. Name: [<global resource name>]	The resource definition selected in [Destination] is not found.	Check the global resource specified in [Destination].
InvalidPropertyConfigurationException <property name> is not specified.	[<property name>] is not specified.	Specify [<property name>].
InvalidPropertyConfigurationException The local directory does not exist.	The directory path specified in [Local directory] does not exist.	Specify a directory path that exists in [Local directory].
InvalidPropertyConfigurationException The path specified in local directory is not a directory.	The path specified in [Local directory] is not a directory.	Specify a directory path in [Local directory].
InvalidPropertyConfigurationException Could not read the local directory.	Could not read the directory specified in [Local directory].	Specify a readable directory path in [Local directory].
com.microsoft.azure.storage.StorageException	Status unavailable to connect due to the following causes. There is something wrong with [Account] or [Access key] of the global resource specified in [Destination]. 2 byte characters are included in [Account] of the global resource specified in [Destination].	Check the settings of the global resource specified in [Destination] and the connection.
java.security.InvalidKeyException Storage Key is not a valid base64 encoded string.	There is something wrong with [Access key] of the global resource specified in [Destination].	Check the settings of the global resource specified in [Destination].

Notes

HTTPS is used in communication between Microsoft Azure BLOB Storage and this adapter.

Naming conventions of container name in Microsoft Azure
- Containers should be named in compliance with the following 4 rules.
  1. A container name starts with alphabets or numerics. It should contain only alphabets, numerics, or hyphens (-).
  2. A hyphen (-) should be both just before and after alphabets or numerics.
  3. Alphabets should be in lower case letters.
  4. The length of a container name should be between 3 and 63 characters.
When an error other than connection related errors (an error that are not described in "Major Exceptions") occurs, sets "Error" in status attribute of file element of the file/container in which the error has occurred of the result data, outputs the result data, and continues to the subsequent process.

When writing files of huge volume in multi-threads, an error may occur (status attribute of file element of the relevant file ends with "Error"). In that case, reduce the multiplicity of threads and execute it.