Input settings field

This section explains the fields of the input settings.

(1) Main fields

Format

The Input File Format screen will change depending on the selection.

CSV

: CSV data

Format

: Format data

Multi Format

: Multi format data

Hierarchy Format

: Hierarchy format data

XML

: XML data

Excel

: Excel format

TEXT

: Text data

You can select only text as the output file format.

Database Table

: Database table data

Note

If settings are made on matching files, conditions, and mapping information, the settings will be removed when you change the setting on this field.

Comments

Comment about the input file to be set

(2) Input file fields

ID

CSV

FMT

MFMT

HFMT

XML

Excel

DB

ID for identifying the input file.

Specify a format ID, a multi format ID, a CSV ID, an XML ID, an Excel ID, a hierarchy format ID, or a database table ID, depending on the format. If Format is neither CSV nor Text, make sure that you specify this field.

Note

If settings are made on matching files, conditions, and mapping information, the settings will be removed when you change the setting on this field.

File name

CSV

FMT

MFMT

HFMT

XML

Excel

TEXT

Input file name used with the data processing process.

Specify with absolute path or relative path. Select the Standard input check box if the standard input is to be used.

Handling of 0 records

All

The behavior when the input data size is 0 bytes.

Treat as normal

: Treated as normal data.

Treat as abnormal

: Treated as an error.

Standard input

CSV

FMT

MFMT

TEXT

Select to make the file name standard input or not.

When Standard input is set and data processing is executed, the system will wait for input. At this time, input the data.

Omit ID

CSV

You can omit ID. If you select this, Delimiter and Enclosure character are active.

Delimiter

CSV

The delimiter for each field of the CSV format file.

Specify from the pull down list.

You can specify it when you select Omit ID.

Delimiter (hexadecimal)

CSV

If Enter hexadecimal value is selected for Delimiter, specify a delimiter. Enter the value by using hexadecimal characters.

Enclosure character

CSV

The enclosure character for character data in the CSV format file.

Specify from the pull down list.

You can specify it when you select Omit ID.

Note

If Enclosure character is not specified, all input fields are treated as character data.

Enclosure character (hexadecimal)

CSV

If Enter hexadecimal value is selected for Enclosure character, specify an enclosure character. Enter the value by using hexadecimal characters.

Check number of fields

CSV

If ID is specified, select whether to check the number of fields in the actual file and the layout information.

When this check box is selected, upon the data processing execution, if the number of fields in the actual file with the number of fields in the layout information is different, an error occurs. Also, upon the preview, an error occurs.

Use header record

CSV

Set the number of rows for the header record of the input file.

The value excluding the specified number of rows is treated as the input data.

Record number

CSV

Specify the number of records to skip when Use header record is set to ON. If the title row is 1, you can specify a maximum of 99 records.

Handling of fields not valid for CSV sort key

CSV

How the situation is handled when the field specified in the sort setting of the input settings does not exist.

If you omit CSV ID in the input settings, this setting is enabled.

Allow

: Processes the value of field that does not exist as Null.

Do not allow

: Make it an error without conversion.

Numeric value byte order

FMT

MFMT

HFMT

DB

The numeric value byte order for the input file.

This field is enabled only if Field type is B or F.

XML record unit

XML

How 1 record is handled when the both the input file and output file are in XML format.

1

: Handles the route node as one record.

2

: Handles the setting node 1 level under the route as 1 record.

3

: Handles the setting node 2 level under the route as 1 record.

Element

: Handles the setting node under the specified element as 1 record.

= Remarks =

For details about the XML record unit, see DataMagic Operation Manual.

Handling of null values in numeric fields

DB

How a null value in the input data is handled.

Select whether to handle a null in the input data as a null or to replace the null with a 0.

Zero (0)

: Null will be handled as 0 of type Z.

Null

: Null will be handled as Null of type M.

Enable mapping

TEXT

Select whether to allow the mapping setting.

Note

Unless you want to use functions when setting the mapping, set the check to OFF. If the check is set to ON, note the following points:

  • Processing performance will decrease compared with when the check is set to OFF.

  • The portion from a null character up to a line break code is cut off regardless of other settings.

(3) Code conversion fields

Kanji code type

All

The kanji code type of the input file. Select from Shift JIS, JEF, EUC, IBM kanji, KEIS, NEC kanji, UTF-8, UTF-16, or JIS. If Format in the input settings is CSV, XML, or Database Table, select from Shift JIS, EUC, UTF-8, or UTF-16.

For Excel, the kanji encoding type is fixed to UTF-8.

= Remarks =

In the case of DataMagic Desktop grade, this field is displayed but cannot be changed.

Note

In the data processing information settings to connect to Oracle, specify the same character set as NLS_LANG for Kanji code type in the input settings and the output settings. If the settings are different, an error occurs in data processing.

Unicode byte order

All

UTF-16 byte order.

If UTF-16 is selected for Kanji code type for the input file, specify the byte order.

Unicode byte order is not displayed if Format in the input settings is Excel.

Same as host

: Depends on the byte order of the connected hardware (Windows or UNIX).

Big-endian

: Utilizes Big-endian.

Little-endian

: Utilizes Little-endian.

= Remarks =

In the case of DataMagic Desktop grade, this field is not displayed.

Line break code

CSV

TEXT

Specify the line break code.

Up to this line break code will be determined as a record.

If Kanji code type is UTF-16, enter this value as either 4 bytes or 8 bytes. If Kanji code type is anything other than UTF-16, enter this value as either 2 bytes or 4 bytes.

EBCDIC set

FMT

MFMT

HFMT

TEXT

The 1-byte code set type.

If Kanji code type for input file is an EBCDIC type or JIS encoding, specify this field.

The code set types that you can specify are as follows.

Table 2.36 1-byte Code Set Type for Input File (EBCDIC Type, JIS)

EBCDIC type

JIS

EBCDIC kana

EBCDIC lowercase

EBCDIC ASCII

EBCDIC ASPEN

Japan (Latin) for IBM

Japan (Latin) Extended for IBM

NEC Katakana

Japan (Katakana) Extended for IBM

 

EBCDIC kana

EBCDIC lowercase

EBCDIC ASCII

EBCDIC ASPEN

Japan (Latin) for IBM

Japan (Latin) Extended for IBM

NEC Katakana

Japan (Katakana) Extended for IBM

ASCII

= Remarks =

In the case of DataMagic Desktop grade, this field is not displayed.

Note

This field is enabled if the Input kanji code type is an EBCDIC type (JEF, IBM kanji, KEIS, NEC kanji) and if EBCDIC set for the output settings is anything other than User table 1 through User table 3.

Convert characters other than either JIS Level 1 or JIS Level 2 characters as well

All

This enables the conversion of characters other than JIS Level 1 and JIS Level 2 characters. For example, this enables the conversion of widely used JIS Level 3 and JIS Level 4 characters such as circled characters and ㈱, without requiring you to register such characters in the external character table.

This takes effect if Shift JIS, IBM kanji, UTF-8, and UTF-16 are specified for Kanji code.

= Remarks =

In the case of DataMagic Desktop grade, this field is not displayed.

(4) Sorting fields

If the input merge file is set in the Merge tab, after the merge, the value set in this tab will be applied to the sort processing.

Sort input data

CSV

FMT

Excel

Specify whether to sort the input data.

Other fields in this tab will be enabled by specifying this field.

No.

CSV

FMT

Excel

Automatically display the key number.

Field No.

CSV

FMT

Excel

Field No. used for sort condition.

If Format is CSV and no ID is specified, specify the field number as a numeric value from 1 to 1000. Specify by using Reference button for other cases.

= Remarks =

If Format in the input settings is CSV and you directly enter a value into Field No., if the specified field does not exist, an error occurs at execution. However, if Handling of fields not valid for CSV sort key is set to Allow in the Layout tab for the input settings, a conversion is performed.

Field Name

CSV

FMT

Excel

Specify the field name to be used in a sort condition on the Select Field screen.

= Remarks =

Identical fields cannot be registered at the same time.

Type

CSV

FMT

Excel

Field type used for sort condition. Specify Character, Numeric value, or Date.

For details about the formats that you can specify in Date, see DataMagic Operation Manual.

Note

If this value is specified as a numeric value type, it is internally converted to floating point for comparison. The accuracy will be a maximum of 15 digits for Windows and 16 digits for UNIX/Linux.

Sort order

CSV

FMT

Excel

Specifies the sort order.

Ascending

: Sorts fields in ascending order. Character strings are sorted in order of character code from low to high.

Descending

: Sorts fields in descending order. Character strings are sorted in order of character code from high to low. When a sort is performed,

Note

DataMagic creates an intermediate format temporary file. For details, see DataMagic Operation Manual.

(5) Merging fields

File name

CSV

FMT

Specify the input merge file name.

The maximum number that you can set is 20.

Delete duplicate lines

CSV

FMT

Select whether to delete the duplicated data out of data in the input file and the input merge file.

Merge keys

CSV

FMT

Specify the field to perform the duplication check.

You can specify this field if Delete duplicate lines is selected.

No.

CSV

FMT

The merge key number is automatically displayed.

Field No.

CSV

FMT

Specify the field number for the merge key.

Field Name

CSV

FMT

Specify the field name corresponding to Field No. on the Select Field screen.

Priority keys

CSV

FMT

Specify which record is prioritized for extraction if duplicate records matching the condition exist when extracting a record by specifying Delete duplicate lines.

No.

CSV

FMT

The key number for Priority keys is automatically displayed.

Field No.

CSV

FMT

Specify the field number for prioritized extract condition for duplicated records.

Field Name

CSV

FMT

Specify the field for prioritized extract condition for duplicated records.

Type

CSV

FMT

The field type used for Priority keys. Specify Character, Numeric value, or Date.

Sort order

CSV

FMT

The sort order used for Priority keys. Specify either Ascending or Descending.