Read Variable-Length File

Operation Name

Function overview

Read variable-length file.

Data model

Data model of this type of component is table model type.

Properties

For information about using variables, refer to "variables".

Basic settings

Item name	Required / Optional	Use of Variables	Description	Remarks
Name	Required	Not available	Enter the name on the script canvas.

Required settings

Item name	Required / Optional	Use of Variables	Description	Remarks
File	Required	Available	Enter the file path to read variable-length. Click [Browse] button to activate the file Selectr and Select the file.	Specify the absolute path of the DataSpider file system as the file path. Constrained Characters of DataSpider File System except for path separator "/" cannot be used.
Delimiter mode	Required	Not available	Select how to specify the delimiter. [Select from list]: (Default) Select delimiter character from the list. [Enter directly]: Input the delimiter character directly into the field. [Enter character code]: Input delimiter character into field with character code.	Specified by the method selected, the corresponding input fields ([Delimiter list] / [Delimiter]) switches.
Delimiter list	Optional	Not available	Select a delimiter character. [Space]: (default) [TAB] (Tab): [Comma]: [Semicolon]: [Slash]: [New line]:	Enables when you select [Delimiter mode] from [Select from list].
Delimiter	Optional	Available	Enter a delimiter character. [Enter directly] case Directly input from the keyboard. [Enter character code] case [Encoding] type of encoding specified in bytes. Octal (prefix "0" <zero digit>), decimal (no prefix), hexadecimal (prefix "0x" <X of the alphabet and the number zero> or "#") input is available . If padding is composed of multiple bytes, input with being separated by commas.Ex.: 0x10, 0x13	Enables when [Enter character code] or [Enter directly] are selected in [Delimiter mode].
Double quote escape	Required	Not available	Select whether to make the double quote escape. [Disable]: (default) No double-quote escape. Delimiter can not be held as field value.Delimiter can not be held as field value. [Enable]: Perform double-quote escape. The field delimiter, if contains quotes or double quotes around field values. Double quotes if it contains a field value represented by two letter quotes a letter quotation marks.	Across the field in double quotes, double quotes, or if there is no delimiter in the field you want to escape is an error.
Format	Required	Not available	Select a file format definition that is variable with global resources. How to set the "Variable Wizard" Please for more information. [Add]: Adds new global resource. [Edit]: Global resource settings can be edited with "Edit Resource list".

Property Action

Item name	Description	Remarks
Preview	Start the previewer, display the contents of the file specified in [File].	Disables when specified file dose not exist, or variables are set in [File]. Encode specified in [Encoding] is displayed. You can not edit the file contents. Files with file size smaller than 1MB can be displayed in previewer.If the file size exceeds 1MB, 1MB size contents from the beginning of the file are displayed.

Read settings

Item name	Required / Optional	Use of Variables	Description	Remarks
Encoding	Required	Available	Select or input read file encode. Use only the canonical name supported by Java SE Runtime Environment 8 when specifying the encoding directly in the field. Refer to "Supported Encodings"(http://docs.oracle.com/javase/8/docs/technotes/guides/intl/encoding.doc.html) for details.	For information regarding the default values used, refer to "Default encoding".
New line code	Required	Not available	Select a new line code. [AUTO]: (default) Implementation-dependent [CR]: Primarily Mac OS (Version 9) [LF]: Primarily UNIX/Linux [CR+LF]: Primarily Microsoft Windows
Retrieve empty fields as empty string	Optional	Not available	Select whether to retrieve the value of the field as an empty string or as null if the field is empty. [Checked]: Retrieved as empty string [Not Checked]:(default) Retrieved as null

Mass data processing

Item name	Required / Optional	Use of Variables	Description	Remarks
Mass data processing	Required	Not available	Select a data processing method. [Use script settings]: (default) Use adapter to set script properties of the data process. [Disable]: Mass data processing is not performed. [Enable]: Mass data processing is performed.

Comment

Item name	Required / Optional	Use of Variables	Description	Remarks
Comment	Optional	Not available	You can write a short description of this adapter. The description will be reflected in the specifications.

Schema

Input Schema

None.

Output Schema

The number of columns varies depending on the [Format]

See "Schema of Table Model" for details regarding schema structure.

Loading schema in Mapper

Schema is loaded automatically.

For any details, please refer to the 'Editing Schema'

Mass data processing

Mass data processing is supported.

PSP Usage

PSP is supported.

For details on PSP, refer to "Parallel Stream Processing".

Available component variables

Component variable name	Description	Remarks
count	Return number of read columns.	The value defaults to null. Null when using Parallel Stream Processing.
message_category	Stroes the category to which corresponding message code belongs to, when an error occurs.	The value defaults to null.
message_code	Stores its corresponding message code of occured error.	The value defaults to null.
message_level	Stores the severity of the corresponding message code of the occured error.	The value defaults to null.
error_type	Stores the type of the occured error.	The value defaults to null. Error is represented in the format depicted below. Example:java.io.FileNotFoundException The message may vary depending on the DataSpider Servista version.
error_message	Stores the error message for the occured error.	The value defaults to null. The message may vary depending on the DataSpider Servista version.
error_trace	Stores stack trace of the occurred error.	The value defaults to null. The message may vary depending on the DataSpider Servista version or the client application used.

Null and empty string

If the field is empty, it is retrieved either as null or empty character, according to the selection for the [Retrieve empty fields as empty string].

If the record defined in the input data structure does not exists, the data of the fields in that record becomes null.

Specification Limits

About naming conventions
- Exists between those elements with the same name in same parent element, can not be read properly.
  Specifically, it applies for the following elements.
  - Groups
  - Records directly beneath the structure definition
  - Records in the same group
  - Fields in the same record
- Record group name field is required.
About the number of occurrences for group/record
- If the [Occurrence] of the group is [Unlimited], when the corresponding input data appears, checks whether it can be applied to the data analysis of the next group.If applicable, apply to the next group.If not applicable, use the group to analysis and move on.
- If the [Occurrence] of the record is [Unlimited], when the corresponding input data appears, checks whether it can be applied to the data analysis of the next record.If applicable, apply to the next record.If not applicable, use the record to analysis and move on.
- If the [Occurrence] of the record or the group is [Unlimited], the maximum value is Integer.MAX_VALUE of Java which is 2147483647.
- If the [Occurrence] of the group is [Omittable] error does not occur even when the time of occurrence of the group is less than the specified occurrence.
- If the [Occurrence] of the record is [Omittable] error does not occur even when the time of occurrence of the record is less than the specified occurrence.
About the date type field
- Time and date appear in a string is dependent on what the default locale.If date data which can not be parsed by the default locale is read, read the data as a string.
About mass data processing
- For large data records in the group so, OutOfMemoryError might occur.Others
Others
- External characters are not supported.
- When field in [Column list] is deleted by [Structure definition], field remains in [Column list].
- If select group by [Structure definition] and add field to [Column list], field order cannot be ensured
- Tag, an unspecified number of occurrences do not have data carriage returns, the analysis can not determine the exact criteria for the structural analysis can not be switched.
- Running log is a "transaction" and is intended implicitly defined to summarize the structure of the group is no limit occurrences.

Main exceptions

Exception name	Cause	Solution
ResourceNotFoundException Resource definition could not be found.Name:[]	The [Format] is not specified.	Specify the [Format].
ResourceNotFoundException Resource definition could not be found.Name:[<Global Resource Name>]	The resource definition specified in the [Format] is not found.	Make sure the global resource specified in the [Format].
InvalidPropertyConfigurationException <Property name> is not specified.	[<Property name>] is not specified.	Specify the [<Property name>].
FileIsDirectoryException	Path input in [File] is directory.	Input file path in [File].
StructuredTextParseException Could not read the file in the specified format.	The file specified in the [File] can not be parsed by the format specified in the [Format].	Make sure the [File] or [Format].
java.io.FileNotFoundException	The file specified in [File] does not exist.	Check the [File].
java.io.UnsupportedEncodingException	The encoding that is not supported is specified in [Encoding].	Specify an encoding that is supported in Java SE Runtime Environment 8. Refer to "Supported Encodings"(http://docs.oracle.com/javase/8/docs/technotes/guides/intl/encoding.doc.html) for details.