Update

Operation Name

Update

Function overview

Specify the file on DataSpider file system and transfer to FTP server.

Data model

Data model of this component is XML type.

Properties

For information about using variables, refer to "variables".

Basic settings

Item name	Required / Optional	Use of Variables	Description	Remarks
Name	Required	Not Available	Enter the name on the script canvas.

Required settings

Item name	Required / Optional	Use of Variables	Description	Remarks
Destination	Required	Not Available	Select Global Resources. Refer to "the global resource properties" to set up a global resource. [Add]: Adds new global resource. [Edit list]: Global resource settings can be edited with "Edit Resource list".
Path	Required	-	The path. Each column can be operated with the following buttons. [Add]: Adds the column. [Up]: Moves the order of the selected column upwards by one. [Down]: Moves the order of the selected column downwards by one. [Delete]: Deletes the column.
Path / Remote path	Required	Available	Enter file/directory on FTP server with absolute path starting from "/".	[Remote path] if use a wildcard ("*","?"), please enter the directory name. Case-sensitive.
Path / Local path	Required	Available	Enter the directory path of DataSpiderServcer.	Please specify an absolute path in the DataSpider file system. Constrained characters of DataSpider File System cannot be used except for path separator "/" and wildcard ("","?"). The file name can use wildcard ("","?"). Example: /upload/*.csv Files with csv extension is under /upload directory correspond. Example: /upload/sales data??.xls Xls files which begins with "sales data" followed by two characters under /upload directory correspond. Not case sensitive.
Path / Data type	Required	Not available	Select the data type. [ASCII]: [BINARY]: (default)

Update settings

Item name	Required / Optional	Use of Variables	Description	Remarks
Directory option	Required	Not available	Select the directory when updating data. [Ignore]: (Default) Ignore the directory and do not transfer. Directory information is not included in the list. [Recursive]: Uploads the file/directory in the specified hierarchy recursively.
Date option	Required	Not available	Select whether to include date information. [Exclude]: (default) Results do not include date information. [Include]: Results include date information.
Append	Optional	Not available	Select whether to append if the file exists. [Checked]: If there are files on the FTP server path, the end of the files of FTP server will be appended by the files from DataSpiderServer side. If there are no FTP files on the FTP server path, a new file will be created. [Not Checked]: (default) If there are files on the FTP server path, the end of the files of FTP server will be overwritten by the files from DataSpiderServer side. If there are no FTP files on the FTP server path, a new file will be created.	If checked, APPE the command to transfer files. [Criteria for updating] to the [Exist] is selected, append will not be done.
Check for updates when transferring files	Optional	Not available	Select whether to check to update check when transferring files to a remote file. [Checked]: (default) [Criteria for updating] transfer only if meet the condition specified. [Not Checked]: Forced to transfer.
Criteria for updating	Required	Not available	Select the condition when updating the files. [Date modified]: (default) Compare the update date between files on FTP server side and files on the DataSpiderServer, if files on DataSpiderServer side are newer, then transfer the files. The FTP server must support MDTM command. [Exist]: Transfers in case file/directory does not exist in the path of FTP server side.	[Check for updates when transferring files] check to enable.

Comment

Item name	Required / Optional	Use of Variables	Description	Remarks
Comment	Optional	Not Available	You can write a short description of this adapter. The description will be reflected in the specifications.

Schema

Input Schema

None.

Output Schema

<?xml version="1.0"?>
<ftpserver name="" port="" user="">
  <pathnames>
    <pathname date="" filetype="" local="" remote="" size="" status="" type=""/>
  </pathnames>
</ftpserver>

Element name	Attribute name	Description	Remarks
ftpserver	-
	name	Host name of connected FTP server
	port	Port number of connected FTP server
	user	User name to connect to the server
pathnames	-
pathname	-	Emerges accordingly to the update targeted file/directory.
	date	Outputs the update targeted file's last updated date in yyyyMMddHHmmss format. Example: "20071016131522" means 1:15 pm and 22 seconds on October 16, 2007.	[Date option] is gotten only when [Include] . The data of directory is empty.
	filetype	Outputs the file type of update targeted file/directory. File:File Directory:Directory
	local	Outputs the absolute path on DateSpider file system of update targeted file/directory.
	remote	Outputs the absolute path on FTP server of update targeted file/directory.
	size	Outputs the update targeted file size. The unit is byte.	The data of directory is empty.
	status	Upload Status Updated: File is uploaded, and file on FTP server is updated. Uptodated: File is not uploaded. This status is output when you select [Date modified] at [Criteria for updating] and file on DataSpider is not newer than file on FTP server. Exists: File is not uploaded. This status is output when you select [Exist] at [Criteria for updating], and file on DataSpider is existed on FTP server. NG: Not uploaded because of some reason.
	type	Data Types ASCII: ASCII output when format is ASCII. BINARY: output when format is BINARY.	The data of directory is empty.

Loading schema in Mapper

Schema is loaded automatically.

See "Edit Schema" for details with regards to defining a schema.

Mass Data Processing

Mass data processing is not supported.

Transaction

Transaction is not supported.

PSP Usage

PSP is not supported.

Available component variables

Component variable name	Description	Remarks
count	Stores the numbers of update targeted file/directory.	The value defaults to null. This value is the sum of "directory_count" and "file_count".
directory_count	Stores the numbers of update targeted directory.	The value defaults to null.
file_count	Stores the numbers of update targeted file.	The value defaults to null.
message_category	Stroes the category to which corresponding message code belongs to, when an error occurs.	The value defaults to null.
message_code	Stores its corresponding message code of occured error.	The value defaults to null.
message_level	Stores the severity of the corresponding message code of the occured error.	The value defaults to null.
error_type	Returns the error type when error occurred.	The value defaults to null. Error is represented in the format depicted below. Example:java.io.FileNotFoundException The message may vary depending on DataSpider Servista version.
error_message	Return the error message when error occurred.	The value defaults to null. The message may vary depending on DataSpider Servista version.
error_trace	Return trace information when error occurred.	The value defaults to null. The message may vary depending on DataSpider Servista version or the client application used.

Specification Limits

ACCOUNT (ACCT) is not supported.

Issue an MDTM command and receive the modification date of files from the FTP server to see whether to update.
Some FTP servers may not use this command.If so, do not check [Check for updates when transferring files], put a check [Check for updates when transferring files], select [Exist] in [Criteria for updating].
In addition, if MDTM does not return GMT (Greenwich Mean Time) you may have trouble in determining the update.

MDTM command can be used in FTP server, we can use [Date modified] in [Check for updates when transferring files] only when MDTM command returns GMT (Greenwich Mean Time).
In addition, this feature is not available if there is a difference between FTP server and DataSpiderServer host.

In DataSpider, the modification time of files on DataSpider can not be freely set.Therefore,time of files received from FTP server is set from DataSpider host.
Therefore, there may be an error if there is time difference between FTP server and DataSpiderServer.

To upload a file, you must have the permission to read the files uploaded. The path to upload files to FTP server is also a must.

Main exception

Exception name	Causes	Solution
ResourceNotFoundException Resource definition could not be found.Name: []	[Destination] is not specified.	Please specify [Destination] .
ResourceNotFoundException Resource definition could not be found.Name: [<name of Global Resources>]	Resource definition selected in [Destination] is not found.	Please check the global resource specified in [Destination]
RemoteFileListParseException Failed analyze the list of remote files. The encoding specified may be incorrect.	Failed analyze the list of remote files.	Ensure the [Encoding] specified in [Destination] and the FTP server settings are correct.
FTPException	Exception if any error returned by the FTP server.	Check the contents of the message.
com.enterprisedt.net.ftp.FTPException	Exception if any error returned by the FTP server.	Check the contents of the message.
java.io.FileNotFoundException	[Path / Local path] specified in Path does not exist.	Check [Path / Local path].
java.net.UnknownHostException	Exception if FTP server is not found.	Please check your FTP server configuration.
java.net.ConnectException	Exception if can not Connect to FTP Server.	Check if port number or FTP is running.

Notes

File path used by adapters is absolute path.Input and Output parts of the path and file path are all absolute path.

The separator used for all adapters is "/" .

Characters can not be used as file name.
- Not supported by dataSpiderServer and extended character encoding or external characters can not be used.
- Characters unable to connect to the FTP server cannot be used.String can not be used is as follows.
  Shift-JIS character code in parentheses .
  - ―(0x815c)
  - ～(0x8160)
  - ‖(0x8161)
  - －(0x817c)
  - ¢(0x8191)
  - £(0x8192)
  - ¬(0x81CA)
If use wildcards in file paths, can not specify wildcards in file paths.
- File path that contains wildcard as follows can be set.
  - /home/user/getdata/*
  - /home/user/getdata/*.txt
- File path that contains wildcard as follows cannot be set.
  - /home/user/*/sample.txt
  - /home/user/*data/*.txt
Start the upload process, except when an error occurs when uploading files network failure, the log level "WARN" log output to the error, and then continue later.
The output schema, sets the status attribute of the file that occurs an error "NG".