Get File/Directory List

Operation Name

Gets file/directory list from HDFS directory.

Data model of this component is XML type.

For information about using variables, refer to "variables".

Item name	Required/Optional	Use of variables	Description	Remarks
Name	Required	Not available	Enter the name to show on the script canvas.

Item name	Required/Optional	Use of variables	Description	Remarks
Destination	Required	Not available	Select a global resource. For more details on Global Resource settings, refer to "Global Resource Property". [Add]: Adds a new global resource. [Edit list]: Enables to edit the global resource settings in the "Edit resource list" screen.
HDFS Path	Required	Available	Enter the path of HDFS file/directory.	The following characters are not available. space < > " ^ [ ] { } % \| ` : ; Multibyte characters are not available.

Item name	Required/Optional	Use of variables	Description	Remarks
Get option	Required	Not available	Specify whether to include file/directory in the results. [Both file and directory]:(Default) Gets both file and directory. [File only]: Gets file only. [Directory only]: Gets directory only.

Item name	Required/Optional	Use of variables	Description	Remarks
Comment	Optional	Not available	You can write a short description of this adapter. The description will be reflected in the specifications.

None.

<?xml version="1.0" encoding="UTF-8"?>
<hdfs>
  <file name="" url="" owner="" size="" isDirectory="" updated="" />
</hdfs>

Element name	Attribute name	Description	Supplement
hdfs	-
file	-	Shows file/directory.
	name	Shows file/directory name.
	url	Shows the URL of WebHDFS
	owner	Shows the owner of file/directory.
	size	Shows the size of file.
	isDirectory	Shows file or directory.	In the case of directory, this attribute becomes "true".
	updated	Shows the final update time of file/directory.	Outputs with the format set in "xmlfw.daterenderingformat" of the system property. If this is not set, outputs with the default format "yyyy-MM-dd'T'HH:mm:ss.SSSZZ" Ex:2007-10-16T13:15:22.738+0900

Schema is loaded automatically.

For details, please refer to "Edit schema".

Mass data processing is not supported.

PSP is not supported.

Component variable name	Description	Remarks
count	Stores the count of the data.	The value defaults to null.
directoryPath	Stores the path of directory on HDFS which gets the lists.	The value defaults to null.
message_category	Stroes the category to which corresponding message code belongs to, when an error occurs.	The value defaults to null.
message_code	Stores its corresponding message code of occured error.	The value defaults to null.
message_level	Stores the severity of the corresponding message code of the occured error.	The value defaults to null.
error_type	Returns the error type when error occurred.	The value defaults to null. Error is represented in the format depicted below. Example:java.io.FileNotFoundException The message may vary depending on the DataSpider Servista version.
error_message	Return the error message when error occurred.	The value defaults to null. The message may vary depending on the DataSpider Servista version.
error_trace	Return trace information when error occurred.	The value defaults to null. The message may vary depending on the DataSpider Servista version or the client application used.

If multibyte or "%" is included in the name of getting file/directory, you cannot get properly or becomes error upon getting.

Exception name	Causes	Solution
ResourceNotFoundException Resource definition is not found. Name:[]	[Destination] is not specified.	Specify [Destination].
ResourceNotFoundException Resource definition is not found. Name:[<Global resource name>]	Resource definition selected in [Destination] is not found.	Check the global resource specified in [Destination].
InvalidPropertyConfigurationException <Property name> is not specified.	[<Property name>]is not specified.	Specify [<Property name>].
java.io.FileNotFoundException	File specified in [HDFS file path] does not exist.	Check [HDFS file path].