Read CSV File

Operation Name

Read CSV File

Function Overview

Read a file of format such as CSV (Comma Separated Values), TSV (Tab Separated Values), etc.

Data Model

Table model type.

Properties

For details on use of variables, refer to “Variables”.

Basic settings

Item name	Required/Optional	Use of variables	Description	Remarks
Name	Required	Not available	Enter the name to show on the script canvas.

Required settings

Item name	Required/Optional	Use of variables	Description	Remarks
File	Required	Available	Enter the file path of CSV or TSV. You can click [Browse] button to launch file chooser and find the file.	Specify the file path as an absolute path of DataSpider File System. Except for the path separator "/”, restricted characters of DataSpider File System cannot be used.
Delimiter mode	Required	Not available	Select a method to specify delimiter. [Select from list]: (default) Select a delimiter character from list. [Enter directly]: Enter the delimiter character directly in the field. [Enter character code]: Enter the delimiter character using character code in the field.	Depending on selected method, the corresponding input field is displayed.
Select from list/Delimiter	Optional	Not available	Select a delimiter character. [Comma]: (default) Specify comma (,). [Tab]: Specify tab. [Space]: Specify space ( ). [Semicolon]: Specify semicolon (;). [Slash]: Specify slash (/). [Vertical bar]: Specify vertical bar (\|).	Enabled when [Select from list] is selected in [Delimiter mode].
Enter directly/Delimiter	Optional	Available	Enter one character for delimiter.	Enabled when [Enter directly] is selected in [Delimiter mode]. Double quotation cannot be specified in delimiter character.
Enter character code/Delimiter	Optional	Available	Enter the character code. The character code is converted with the encoding specified in [Encoding]. You can enter the code as octal (with prefix "0" <number 0>), decimal (without prefix), and hexadecimal (with prefix "0x" <alphabet x with numeric 0> or "#"). When padding is in multi-byte string, enter it by separating with comma. Example: 0x10,0x13	Enabled when [Enter character code] is selected in [Delimiter mode]. Character code that converts to one character can be entered. Character code that converts to newline code or double quotation cannot be specified.
Column list	Optional	-	Specify columns. Each column can be operated with the following buttons. [Add]: Add a column. [Up]: Move the selected column upwards by one row. [Down]: Move the selected column downwards by one row. [Delete]: Delete the column.	Data will be read only for the number of columns set in [Column list]. Column name set in [Column list] will be displayed in Mapper schema.
Column list/Column name	Required	Available	Enter the column name. The first row of the file specified in [File] can be set as column names using [Update column list] property action.

Property actions

Item name	Description	Remarks
Preview	When preview is clicked, the contents of the file specified in [File] is displayed.	Disabled when the specified file does not exist or variables are set in [File]. Displayed with the encoding specified in [Encoding]. File contents cannot be edited. The maximum file size that can be displayed in the previewer is 1MB. When the file size exceeds 1MB, the file is displayed up to 1MB from the beginning of the file.
Update column list	The row specified in [Starting row] of the file specified in [File] is set as column names.	Disabled when the specified file does not exist or variables are set in the following properties. [File] [Starting row]
Read column names from first row of file	Select the file in file chooser, and the first row of that file is set as column names.
Read the number of columns from file	Select the file in file chooser, and the number of columns of that file is set as the number of column names.

Read settings

Item name	Required/Optional	Use of variables	Description	Remarks
Encoding	Required	Available	Select or enter the encoding of the file to be read. When you enter encoding, specify one of the encodings supported in Java SE Runtime Environment 8. For more details, refer to “Supported Encodings” (http://docs.oracle.com/javase/8/docs/technotes/guides/intl/encoding.doc.html).	For more details on default value, refer to “Encoding Selected by Default”.
Starting row	Optional	Available	Enter the starting row.	Default value is 1 and the value will be set to 1 when you omitted. When [Do not get first row as value] is checked, data will be read from the row subsequent to [Starting row].
Do not get first row as value	Optional	Not available	Select whether or not to handle the starting row of the specified file as data. [Checked]: Do not handle as data. [Not checked]: (default) Handle as data.

Multi-thread processing settings

Item name	Required/Optional	Use of variables	Description	Remarks
Enable multi-thread processing settings	Optional	Not available	Specify whether or not to perform multi-thread processing when the component to receive the result data supports it. [Checked]: Multi-thread processing will be performed when the component to receive the result data supports it. [Not checked]: (default) Multi-thread processing will not be performed.	For more details on multi-thread processing, refer to Multi-thread Processing. When it is checked and the component to receive the result data does not support multi-thread processing, multi-thread processing will not be performed. When it is checked and mass data processing is enabled, and the component receive the result data supports multi-thread processing, multi-thread processing will be prioritized and mass data processing will not be performed.

Data processing method

Item name	Required/Optional	Use of variables	Description	Remarks
Data processing method	Required	Not available	Select data processing method. [Apply script settings]: (default) Apply the mass data processing settings of the script property to the adapter. [Do not perform mass data processing]: Mass data processing will not be performed. [Perform mass data processing]: Mass data processing will be performed.

Comment

Item name	Required/Optional	Use of variables	Description	Remarks
Comment	Optional	Not available	You can write a short description of this adapter. The description will be reflected in the specifications.

Schema

Input schema

None.

Output schema

The number of columns varies depending on the [Column list] settings.

Refer to “Schema of table model type” for more details.

Loading schema in Mapper

Schema is loaded automatically.

Refer to “Edit Schema” for more details.

Mass Data Processing

Mass Data Processing is supported.

PSP Usage

PSP is supported.

Refer to “Parallel Stream Processing” for more details.

Available Component Variables

Component variable name	Description	Remarks
count	The number of read data is stored.	Default value is null. Values are not stored in PSP.
filePath	The read file path is stored.	Default value is null. Values are not stored in PSP.
message_category	When an error occurs, the corresponding message code's category is stored.	Default value is null.
message_code	When an error occurs, the corresponding message code is stored.	Default value is null.
message_level	When an error occurs, the corresponding message code's severity is stored.	Default value is null.
error_type	When an error occurs, the error type is stored.	Default value is null. The error type will be in the following format. Example: “java.io.FileNotFoundException” The contents that will be stored may vary depending on DataSpider Servista versions.
error_message	When an error occurs, the error message is stored.	Default value is null. The contents that will be stored may vary depending on DataSpider Servista versions.
error_trace	When an error occurs, the error's trace information is stored.	Default value is null. The contents that will be stored may vary depending on DataSpider Servista versions and client applications.

Null and Empty Strings

When the target data is empty, the read data will be null.

Specification Limits

CSV adapter operates based on the following rules.
1. Data are separated by the specified delimiter character.
2. When the data contains delimiter character, ["] (double quotation), or new line, make sure to put ["] at the beginning and the end of data.
  When ["] is included in data, put that as 2 consecutive quotations like [""].
Example: When delimiter character is “,” (comma)

Data value Notation

1,000 yen “1,000 yen"

AB(newline)CD “AB(newline)CD”

He says "I'm fine" “He says ""I'm fine"""

When the data size of a single column in the file to be read is large, OutOfMemoryError may occur.
In that case, make the maximum heap size of DataSpiderServer larger or decrease the data size of a single column.

Data value	Notation
1,000 yen	“1,000 yen"
AB(newline)CD	“AB(newline)CD”
He says "I'm fine"	“He says ""I'm fine"""

Major Exceptions

Exception name	Cause	Solution
InvalidPropertyConfigurationException <property name> is not specified.	[<property name>] is not specified.	Specify [<property name>].
InvalidPropertyConfigurationException Invalid delimiter is specified.	Invalid character is entered in [Delimiter].	Enter the valid character in [Delimiter].
FileIsDirectoryException	The path entered in [File] is directory.	Enter file path in [File].
java.io.FileNotFoundException	The file specified in [File] does not exist.	Check [File].
java.io.UnsupportedEncodingException	Unsupported encoding is specified in [Encoding].	Specify an encoding supported in Java SE Runtime Environment 8. For more details, refer to “Supported Encodings” (http://docs.oracle.com/javase/8/docs/technotes/guides/intl/encoding.doc.html).