Update

Operation Name

Update

Function Overview

Transfer the specified files in DataSpider File System to FTP server.

Data Model

XML type.

Properties

For details on use of variables, refer to “Variables”.

Basic settings

Item name	Required/Optional	Use of variables	Description	Remarks
Name	Required	Not available	Enter the name to show on the script canvas.

Required settings

Item name	Required/Optional	Use of variables	Description	Remarks
Destination	Required	Not available	Select a global resource. For details about how to configure global resource, refer to “Global Resource Properties”. [Add]: Add a new global resource. [Edit list]: Global resource settings can be edited in “Edit resource list” screen.
Path	Required	-	Specify paths. Each column can be operated with the following buttons. [Add]: Add a column. [Up]: Move the selected column upwards by one row. [Down]: Move the selected column downwards by one row. [Delete]: Delete the column.
Path/Remote path	Required	Available	Enter the file/directory in the FTP server with an absolute path starting with “/”.	When using wild card (“”/”?”) in [Local path]*, enter a directory name. Case-sensitive.
Path/Local path	Required	Available	Enter a directory path of DataSpider File System.	Specify the file path as an absolute path of DataSpider File System. Except for the path separator "/” and wild cards (“”/”?”), the restricted characters of DataSpider File System cannot be used. Wild cards ("" and "?") can be used in file name. Example: /upload/*.csv It corresponds to the file with csv extension which exists directly under /upload directory. Example: /upload/sales_data??.xls It corresponds to the file with xls extension, starting with “sales_data” followed by two characters, which exists directly under /upload directory. Case-insensitive.
Path/Data type	Required	Not available	Select data type. [ASCII]: [BINARY]: (default)

Update settings

Item name	Required/Optional	Use of variables	Description	Remarks
Directory option	Required	Not available	Select directory handling when updating data. [Ignore]: (default) Ignore directories and do not transfer them. Further, the directory information will not be included even in the result list. [Recursive]: Upload files/directories under the specified directory recursively.
Date option	Required	Not available	Select whether or not to include date information. [Exclude]: (default) Date information will not be included in the result. [Include]: Date information will be included in the result.
Append	Optional	Not available	Select whether or not to append when the file exists. [Checked]: When the file is in FTP server side path, the contents of the DataSpiderServer side file will be appended at the end of FTP server side file. The file will be newly created when the file does not exist in FTP server side path. [Not checked]: (default) When the file is in FTP server side path, the FTP server side file will be overwritten with the DataSpiderServer side file. The file will be newly created when the file does not exist in FTP server side path.	When checked, the file will be transferred using APPE command. When [Exist] is selected in [Criteria for updating], appending is not performed.
Check for update when transferring file	Optional	Not available	Select whether or not to check for updates of remote files when transferring files. [Checked]: (default) Transfer only when the criteria specified in [Criteria for updating] are satisfied. [Not checked]: Transfer forcibly.
Criteria for updating	Required	Not available	Select criteria for updating files. [Updated time]: (default) Compare the file updated time in FTP server side with that in DataSpiderServer side, and transfer when the one in DataSpiderServer side is newer. FTP server should support MDTM command. [Exist]: Transfer file/directory when it does not exist in FTP server side path.	Enabled when [Check for update when transferring file] is checked.

Comment

Item name	Required/Optional	Use of variables	Description	Remarks
Comment	Optional	Not available	You can write a short description of this adapter. The description will be reflected in the specifications.

Schema

Input schema

None.

Output schema

<?xml version="1.0"?>
<ftpserver name="" port="" user="">
  <pathnames>
    <pathname date="" filetype="" local="" remote="" size="" status="" type=""/>
  </pathnames>
</ftpserver>

Element name	Attribute name	Description	Remarks
ftpserver	-
	name	Outputs the host name of the connected FTP server.
	port	Outputs the port number of the connected FTP server.
	user	Outputs the user name used when connecting to FTP server.
pathnames	-
pathname	-	Appears for each file/directory to be updated.
	date	Outputs the last updated date and time of the file in yyyyMMddHHmmss format. Example: “20071016131522” shows October 16, 2007 13:15:22.	It is got only when [Date option] is [Include]. Empty, in the case of directory.
	filetype	Outputs the file type. File: File Directory: Directory
	local	Outputs the absolute path in the DataSpider File System of the file/directory.
	remote	Outputs the absolute path in the FTP server of the file/directory.
	size	Outputs the file size. Unit is byte.	Empty, in the case of directory.
	status	Outputs the upload status. Updated: Status where the file is uploaded and the file on FTP server is updated. Uptodated: Status where the file is not uploaded. When [Updated time] is selected in [Criteria for updating] and if the file in DataSpider is not newer than the file in the FTP server, it is output. Exists: Status where the file is not uploaded. When [Exist] is selected in [Criteria for updating] and if the file in DataSpider exists in the FTP server, it is output. NG: Status where the file was not uploaded due to some error.
	type	Outputs the data type. ASCII: Outputs when uploaded in ASCII format. BINARY: Outputs when uploaded in BINARY format.	Empty, in the case of directory.

Loading schema in Mapper

Schema is loaded automatically.

For more details, refer to “Edit Schema”.

Mass Data Processing

Mass data processing is not supported.

Transaction

Transaction is not supported.

PSP Usage

PSP is not supported.

Available Component Variables

Component variable name	Description	Remarks
count	The number of files/directories to be updated is stored.	Default value is null. The sum of “directory_count” and “file_count”.
directory_count	The number of directories to be updated is stored.	Default value is null.
file_count	The number of files to be updated is stored.	Default value is null.
message_category	When an error occurs, the corresponding message code's category is stored.	Default value is null.
message_code	When an error occurs, the corresponding message code is stored.	Default value is null.
message_level	When an error occurs, the corresponding message code's severity is stored.	Default value is null.
error_type	When an error occurs, the error type is stored.	Default value is null. The error type will be in the following format. Example: “java.io.FileNotFoundException” The contents that will be stored may vary depending on DataSpider Servista versions.
error_message	When an error occurs, the error message is stored.	Default value is null. The contents that will be stored may vary depending on DataSpider Servista versions.
error_trace	When an error occurs, the error's trace information is stored.	Default value is null. The contents that will be stored may vary depending on DataSpider Servista versions and client applications.

Specification Limits

ACCOUNT(ACCT) is not supported.

Confirmation whether updated or not is judged by getting the file updated time from FTP server.
Depending on FTP server, this command might not be able to used. In that case, do not put a check in [Check for update when transferring file], or put a check in [Check for update when transferring file] and select [Exist] in [Criteria for updating].
Also, when MDTM does not return GMT (Greenwich Mean Time), judging update might not be work well.

Only when MDTM command can be used in FTP server and the command returns GMT (Greenwich Mean Time), [Updated time] of [Check for update when transferring file] can be used.
Also, when there is a time difference between the FTP server and the host in which DataSpiderServer operates, this function cannot be used.

In this adapter, updated time cannot be set arbitrarily for files in DataSpider File System. Therefore, the time of OS in which DataSpiderServer operates is set to files got from FTP server.
Consequently, when there is a time difference between the FTP server and the OS in which DataSpiderServer operates, judging update might not be work well.

To upload files, you need to have permissions to read the files to be uploaded. Further, you need to have permissions to write to the FTP server path to be uploaded to.

Major Exceptions

Exception name	Cause	Solution
ResourceNotFoundException Resource definition is not found. Name: []	[Destination] is not specified.	Specify [Destination].
ResourceNotFoundException Resource definition is not found. Name: [<global resource name>]	The resource definition selected in [Destination] is not found.	Check the global resource specified in [Destination].
RemoteFileListParseException Failed to parse the remote file list. The encoding may be incorrect.	Failed to parse the remote file list.	Check the settings of [Encoding] selected in [Destination] and the settings of FTP server.
FTPException	This is an exception raised when FTP server returns some error.	Check the contents of the message.
com.enterprisedt.net.ftp.FTPException	This is an exception raised when FTP server returns some error.	Check the contents of the message.
java.io.FileNotFoundException	The path specified in [Local path] does not exist.	Check [Local path].
java.net.UnknownHostException	This is an exception raised when the specified FTP server is not found.	Check the FTP server settings.
java.net.ConnectException	This is an exception raised when unable to connect to FTP server.	Check whether the port number is correct and the FTP server is running.

Notes

All the file paths used in FTP Adapter are absolute path. All the items to input and output a path or a file path expect/output absolute path.

”/” is used for all the delimiters of file paths used in FTP Adapter.

Characters that cannot be used for file name
- Character code that is not supported by DataSpiderServer and external characters or extended character code cannot be used.
- Characters that cannot be used in the FTP server to be connected also cannot be used. Strings that cannot be used are as follows.
  The character code present within parenthesis are the character code in Shift_JIS.
  - ―(0x815c)
  - ～(0x8160)
  - ‖(0x8161)
  - －(0x817c)
  - ¢(0x8191)
  - £(0x8192)
  - ¬(0x81CA)
When using wild card in file path, it cannot be specified in the middle of file path.
- You can set file paths with wild card as follows.
  - /home/user/getdata/*
  - /home/user/getdata/*.txt
- File paths with wild card as the following cannot be set.
  - /home/user/*/sample.txt
  - /home/user/*data/*.txt
When an error other than network failure occurs during updating files, the error details is output in the log with the log level "WARN" and the operation continues.
In the output schema, "NG" is set in the status attribute of the file where the error occurred.