Write CSV Data

Operation Name

Write CSV Data

Function Overview

Writes CVS format file to HDFS.

Data Model

Data model of this component is Table Model type.

Properties

For information about using variables, refer to "variables".
Basic settings
Item name Required / Optional Use of variables Description Remarks
Name Required Not available Enter the name to show on the script canvas.  
Input data Required Not available Select the component on the script canvas.  
Required settings
Item name Required / Optional Use of variables Description Remarks
Destination Required Not available Select a global resource. For more details on Global Resource settings, refer to "Global Resource Property".
  • [Add]:
    Adds a new global resource.
  • [Edit list]:
    Enables to edit the global resource settings in the "Edit resource list" screen.
  
HDFS file Path Required Available Enters HDFS file path.
  • The following characters are not available.
    • space < > " ^ [ ] { } % | ` : ;
  • Multibyte characters are not available.
Column list Optional Not available Specify the column.

Each column can be operated with the following buttons.
  • [Add]:
    Adds a column.
  • [Up]:
    Moves the order of the selected column upwards by one.
  • [Down]:
    Moves the order of the selected column downwards by one.
  • [Delete]:
    Deletes a column.
  • Read data of all the columns set in [Column list].
  • Column name specified in [Column list] is displayed in schema of Mapper.
Column list/Column name Required Available Input the name of the column.

You can set the first row of the file specified in [File] of property action of [Update column list] as the column name. 		 
Column list/Quotation Optional Not available Select whether to enclose each column with double quotation.
  • [Checked]:
    Encloses the column with double quotation.
  • [Not checked]:(default)
    Does not enclose the column with double quotation.
  
Property action
Item name Description Remarks
Update column list Set the first line of the file specified in [HDFS file Path] as the column name.
  • Invalid when specified file does not exist, or variables are set in [HDFS file Path].
Get column name from the first row Select the file in the file Selectr and set the first line of the file as the column name.  
Get column count Select the file in the file Selectr and set the number of columns of the file as the number of columns of the column name.  
Write settings
Item name Required / Optional Use of variables Description Remarks
Encoding Required Available Select or input read file encode.

Use only the canonical name supported by Java SE Runtime Environment 8 when specifying the encoding directly in the field.
Refer to "Supported Encodings"(http://docs.oracle.com/javase/8/docs/technotes/guides/intl/encoding.doc.html) for details.
  • Default value is "UTF-8".
Line Feed code Optional Not available Select the Line Feed code.
  • [AUTO]:(default)
    Corresponds to the Line Feed code of the operating OS of DataSpiderServer.
    • In case of Windows:
      CR+LF
    • In case of UNIX:
      LF
  • [CR]:
  • [LF]:
  • [CR+LF]:
  
Overwrite Optional Not available Select whether to overwrite, if the specified file exists.
  • [Checked]:(default)
    Overwrites
  • [Not checked]:
    Does not overwrite. If a file already exists, error occurs.
  
Additional writing Optional Not available Select whether to add in the end of the file or to overwrite, if the specified file exists.
  • [Checked]:
    Adds in the end of the file.
  • [Not checked]:(default)
    Overwrites.
  • This action is valid only when [Overwrite] is selected.
Insert column name in the first row Optional Not available Select whether to insert the column name in the first row of the specified file.
  • [Checked]:
    Inserts [Column name] of [Column list] in the first row.
  • [Not checked]:(default)
    Does not insert the column name.
  • Column name will be inserted in the first row of the result data, and not to the first row of the writing file.
Does not insert column name if a file exists Optional Not available Select whether to insert column name to the first row of the result data, if the specified data exists.
  • [Checked]:
    Does not insert column name to the first row of the result data, if the specified file exists.
  • [Not checked]:(default)
    Inserts column name to the first row of the result data.
  • This action is valid only when [Insert column name to the first row] is selected.
Comment
Item name Required / Optional Use of variables Description Remarks
Comment Optional Not available You can write a short description of this adapter.
The description will be reflected in the specifications.		  

Schema

Input Schema

Numbers of column differs depending on the settings of [Column list].
For schema structure, please refer to "Table Model Type Schema".

Output Schema

None.

Loading Schema in Mapper

Schema is loaded automatically.
See "Edit Schema for any details.

Transaction

Transaction is not supported.

PSP Usage

PSP is supported.
For details on PSP, refer to "Parallel Stream Processing".

Available Component Variables

Component variable name Description Remarks
count Return number of read columns.
  • The value defaults to null.
  • Null when using Parallel Stream Processing.
filePath Return file path of read file.
  • The value defaults to null.
message_category Stroes the category to which corresponding message code belongs to, when an error occurs.
  • The value defaults to null.
message_code Stores its corresponding message code of occured error.
  • The value defaults to null.
message_level Stores the severity of the corresponding message code of the occured error.
  • The value defaults to null.
  • Does not store values in PSP.
error_type Returns the error type when error occurred.
  • The value defaults to null.
  • Error is represented in the format depicted below.
    Example:java.io.FileNotFoundException
  • The message may vary depending on DataSpider Servista version.
error_message Return the error message when error occurred.
  • The value defaults to null.
  • The message may vary depending on DataSpider Servista version.
error_trace Return trace information when error occurred.
  • The value defaults to null.
  • The message may vary depending on DataSpider Servista version or the client application used.

Main Exceptions

Exception name Causes Solution
ResourceNotFoundException
Resource definition is not found. Name:[]		 [Destination] is not specified. Specify [Destination].
ResourceNotFoundException
Resource definition is not found. Name:[<Global resource name>]		 Resource definition selected in [Destination] is not found. Check the global resource specified in [Destination].
InputDataNotFoundException [Input data] is not specified. Specify [Input data], or draw a data flow.
InvalidPropertyConfigurationException
<Property name> is not specified.		 [<Property name>] is not specified. Specify [<Property name>].
FileExistsException A file specified in [HDFS file path] already exists. Edit the file name of [HDFS file path], or select [Overwrite] in [Write settings].
FileIsDirectoryException The path input in [HDFS file path] is the directory. Input the file path in [HDFS file path].
java.io.UnsupportedEncodingException Encodes that are not supported in [Encode] is specified. Specify the encodes that are supported in Java SE Runtime Environment 8.
For more details, please refer to "Supported Encoding"(http://docs.oracle.com/javase/8/docs/technotes/guides/intl/encoding.doc.html).
org.apache.hadoop.ipc.RemoteException
io.bytes.per.checksum(512) and blockSize(<Numerical value of block size>) do not match. blockSize should be a multiple of io.bytes.per.checksum		 The value of [Block size] is not in multiples of io.bytes.per.checksum value of HDFS. Check the value of [Block size] of the global resource specified in [Destination].