Read CSV File

Operation Name

Read CSV File

Function overview

Read file with CSV (Comma Separated Values) format, or TSV (Tab Separated Values) file format, etc.

Data model

Data model of this type of component is table model type.

Properties

For information about using variables, refer to "variables".

Basic setting

Item name	Required / Optional	Use of Variables	Description	Remarks
Name	Required	Not available	Enter the name on the script canvas.

Required settings

Item name	Required / Optional	Use of Variables	Description	Remarks
File	Required	Available	Input file path of CSV format or TSV format. Click [Browse] button to activate the file Selectr and Select the file.	Specify the absolute path of the DataSpider file system as the file path. Constrained characters of DataSpider File System except for path separator "/" cannot be used.
Delimiter mode	Required	Not available	Select how to specify the delimiter. [Select from list]: (default) Select delimiter from the list. [Enter directly]: Input delimiter directly into the field. [Enter character code]: Input delimiter into field with character code.	Corresponding input field is changed according to selected specifying method.
Select from list / Delimiter	Optional	Not available	Select delimiter. [Comma]: (default) Specify a half comma(,). [Tab]: Specify the tab. [Space]: Specify space ( ) . [Semicolon]: Specify a semicolon (;). [Slash]: Specify a slash (/). [Vertical bar]: Specify a Vertical bar (\|).	Valid when you select [Select from list] from [Delimiter mode].
Enter directly / Delimiter	Optional	Available	Input one delimiter.	Valid when you select [Enter directly] from [Delimiter mode]. You can not specify the double quotation as delimiter.
Enter character code / Delimiter	Optional	Available	Enter character code. Character code input is converted to encode specified in [Encoding]. Input with Octal (prefix "0" <the number zero>), decimal (no prefix), hexadecimal (prefix "0x" <the alphabet X and the number zero> or "#") is available. If padding is composed of multiple bytes, input with being separated by commas.Ex.: 0x10, 0x13	Valid when you select [Enter character code] from [Delimiter mode]. You can input character code converted to one character. You cannot specify character code converted to line code or double quotation.
Column list	Optional	-	Specify the column. Each column can be operated with the following buttons. [Add]: Add a column. [Up]: Move the order of the selected column upwards by one. [Down]: Move the order of the selected column downwards by one. [Delete]: Delete the column.	Read data of all the columns set in [Column list]. Column name specified in [Column list] is displayed in schema of Mapper.
Column list/Column name	Required	Available	Input the name of the column. You can set the first row of the file specified in [File] of property action of [Update column list] as the column name.

Property Action

Item name	Description	Remarks
Preview	Start the previewer, display the contents of the file specified in [File].	Invalid when specified file dose not exist, or variables are set in [File]. Encoding specified in [Encoding] is displayed. You can not edit the file contents. Files with file size smaller than 1MB can be displayed in previewer.If the file size exceeds 1MB, 1MB size contents from the beginning of the file are displayed.
Update column list	Set the first line of the file specified in [File] as the column name.	Invalid when specified file dose not exist, or variables are set in [File].
Get column name from the first row	Select the file in the file Selectr and set the first line of the file as the column name.
Get column count	Select the file in the file Selectr and set the number of columns of the file as the number of columns of the column name.

Read Setting

Item name	Required / Optional	Use of Variables	Description	Remarks
Encoding	Required	Available	Select or input read file encode. Use only the canonical name supported by Java SE Runtime Environment 8 when specifying the encoding directly in the field. Refer to "Supported Encodings"(http://docs.oracle.com/javase/8/docs/technotes/guides/intl/encoding.doc.html) for details.	For information regarding the default values used, refer to "Default encoding".
Do not read the first line as a value	Optional	Not available	Select whether to treat the first line of the specified file as the data. [Checked]: Not as data. [Not Checked]: (default) As data.

Multi-thread processing settings

Item name	Required/Optional	Use of variables	Description	Remarks
Enable multi-thread processing setting	Optional	Not available	Specify whether or not to perform multi-thread processing if the component receiving the result data supports it. [Checked]: Performs multi-thread processing if the component receiving the result data supports it. [Not Checked]: (default) Does not perform multi-thread processing.	For details on multi-thread processing, refer to "Multi-thread Processing". When this property is checked, multi-thread processing is not performed if the component that receives the result data does not support it. When this property is checked with mass data processing being enabled, multi-thread processing will be prioritized over mass data processing and mass data processing will not be performed if the component receiving the result data supports multi-thread processing.

Data processing method

Item name	Required / Optional	Use of Variables	Description	Remarks
Mass data processing	Required	Not available	Select a data processing method. [Use script settings]: (default) Apply mass data processing settings of script property to adapter. [Disable]: Mass data processing is not performed. [Enable]: Mass data processing is performed.

Comment

Item name	Required / Optional	Use of Variables	Description	Remarks
Comment	Optional	Not available	You can write a short description of this adapter. The description will be reflected in the specifications.

Schema

Input Schema

None.

Output Schema

Depending on the [Column list] setting, the number of columns may be different.

See "Schema of Table Model" for details regarding schema structure.

Loading schema in Mapper

Schema is loaded automatically.

See "Edit Schema for any details.

Mass data processing

Mass data processing is supported.

PSP Usage

PSP is supported.

For details on PSP, refer to "Parallel Stream Processing".

Available component variables

Component variable name	Description	Remarks
count	Return the number of lines of read data.	The value defaults to null. Null when using Parallel Stream Processing.
filePath	Return file path of read file.	The value defaults to null. Null when using Parallel Stream Processing.
message_category	Stroes the category to which corresponding message code belongs to, when an error occurs.	The value defaults to null.
message_code	Stores its corresponding message code of occured error.	The value defaults to null.
message_level	Stores the severity of the corresponding message code of the occured error.	The value defaults to null.
error_type	Returns the error type when error occurred.	The value defaults to null. Error is represented in the format depicted below. Example:java.io.FileNotFoundException The message may vary depending on DataSpider Servista version.
error_message	Return the error message when error occurred.	The value defaults to null. The message may vary depending on DataSpider Servista version.
error_trace	Return trace information when error occurred.	The value defaults to null. The message may vary depending on DataSpider Servista version or the client application used.

Null and empty string

Any empty strings retrieved will become null.

Specification Limits

CSV adapter is processed according to the following rules.
1. Data are separated by a specified delimiter.
2. If delimiter,"""(double quotes), or a line break is contained, it is necessary to describe "''" in both the beginning and the end of the data.
  Describe as "'' ''" if "''" is contained.
Example: In case of the delimiter is "," (half comma)

Data value Notation

1,000 yen "1,000 yen"

AB (newline) CD "AB (newline) CD"

He says "I,m fine" "He says" "I,m fine" ""

If the size of column data to be handled is very large, OutOfMemoryError may occur.
If this happens, you need to make the size of the column data small enough to be handled.

Data value	Notation
1,000 yen	"1,000 yen"
AB (newline) CD	"AB (newline) CD"
He says "I,m fine"	"He says" "I,m fine" ""

Main exceptions

Exception name	Causes	Solution
InvalidPropertyConfigurationException Property name is not specified.	[<Property name>] is not specified.	Specify the [<Property name>]
InvalidPropertyConfigurationException Invalid delimiter is specified.	Invalid character is input in [Delimiter]	Input valid character in [Delimiter].
FileIsDirectoryException	Path input in [File] is directory.	Input file path in [File] .
java.io.FileNotFoundException	File specified in [File] does not exist.	Check [File].
java.io.UnsupportedEncodingException	Encodings that are not supported in [Encoding] are specified.	Specify an encoding that is supported in Java SE Runtime Environment 8. Refer to "Supported Encodings"(http://docs.oracle.com/javase/8/docs/technotes/guides/intl/encoding.doc.html) for details.