Data Check

Operation Name

Data Check

Function Overview

Checks the value or format of the input data, and outputs the result with the selected format.

Properties

For information about using variables, refer to "Variables".

Basic settings

Item name	Required/Optional	Use of variables	Description	Remarks
Name	Required	Not available	Enter the name on the script canvas.
Input data	Required	Not Available	Select a component on the script canvas.

Required settings

Item name	Required/Optional	Use of variables	Description	Remarks
Use global resource as data check condition	Required	Not Available	Select whether to use global resource as data check condition. [Checked]: Uses global resource. [Not checked]:(default) Does not use global resource.
Data check condition definition	Required	Not Available	Select a data check condition definition registered in global resource. Configurable items in global resource are the same as [Data check condition list]. [Add]: Adds a new global resource. [Edit list]: Enables to edit the global resource settings in the "Edit resource list".
Data check condition list	Optional	-	Specify the check conditions corresponding to input data column. Each column can be operated by the following buttons. [Add]: Adds a column. [Move up]: Moves the order of selected column upwards by one. [Move down]: Moves the order of selected column downwards by one. [Delete]: Deletes the selected column.	Displayed if[Use global resource as data check condition] is [Not checked].
Data check condition list/Column number	Required	Available	Input the column number of the check targeted input data.
Data check condition list/Required	Optional	Not Available	Select whether to treat the data corresponding to [Column number] as required. [Checked]: Treat the column as required. [Not checked]:(default) Treat the column as optional.	If the data corresponding to a column number whose [Required] is checked is null or an empty string, the following checks, [Type][Minimum][Maximum][Option][Regex], will not be performed.
Data check condition list/Type	Required	Not Available	Select the type against which the data corresponding to [Column number] will be checked. [String(string size)]:(default) [String(byte size)]: [Whole number]: [Decimal number]: [Date/Time]:	If [Whole number] or [Decimal number] is specified in [Type] and target input data to be checked is a non-numeric value such as "abc", the following checks, [Minimum][Maximum][Option][Regex], will not be performed.
Data check condition list/Minimum	Optional	Available	Enter the permitted minimum string size, minimum byte size, or minimum value of the data corresponding to [Column number].	If [String(string size)] is specified in [Type], [Minimum] will be treated as digit number. If [String(byte size)] is specified in [Type], [Minimum] will be treated as byte number. If [Whole number] or [Decimal number] is specified in [Type], [Minimum] will be treated as value. If [Date/time] is specified in [Type], value set to [Minimum] will be ignored.
Data check condition list/Maximum	Optional	Available	Enter the permitted maximum string size, maximum byte size, or maximum value of the data corresponding to [Column number].	If [String(string size)] is specified in [Type], [Maximum] will be treated as digit number. If [String(byte size)] is specified in [Type], [Maximum] will be treated as byte number. If [Whole number] or [Decimal number] is specified in [Type], [Maximum] will be treated as value. If [Date/time] is specified in [Type], value set to [Maximum] will be ignored.
Data check condition list/Option	Optional	Available	Specify the option to check the data of [Column number].	For details on how to specify option, refer to "Option".
Data check condition list/Regex	Optional	Available	Specify the regular expression to check the data of [Column number].	The available regular expression pattern is in accordance with java.util.regex.Pattern class of Java. For details on Pattern, refer to "Java(TM) Platform, Standard Edition 8 API Specification(http://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html)".
Output record	Required	Not Available	Select the types of output record. [Output only records that match condition]:(default) Outputs only the records that match condition. [Output only records that don't match condition]: Outputs only the records that don't match condition. [Output all records and check results]: Outputs all records with check results appended.	If [Output all records and check results] is selected, outputs all columns of the input data with an extra column of check result.

Condition evaluation settings

Item name	Required/Optional	Use of variables	Description	Remarks
Evaluation method for multiple condition	Required	Not Available	Specify how to evaluate conditions if multiple data check conditions exist. [AND evaluation]:(default) Check result will be matched only if all conditions match. [OR evaluation]: Check result will be matched if any of the conditions matches. If multiple data check conditions exist for the same column number in [Data check condition list], evaluating method for the conditions can be set in [Specify the evaluation method of multiple conditions for the same column number] and [Evaluation method for the same column number].
Specify the evaluation method of multiple conditions for the same column number	Optional	Not Available	Select whether to specify evaluating method for the conditions if multiple data check conditions exist for the same column number. [Checked]: Performs check using the evaluation method specified in [Evaluation method for the same column number]. [Not checked]:(default) If data check condition multiply exists to the same column number, performs check using evaluation method specified in [Evaluation method for multiple condition].
Evaluation method for the same column number	Optional	Not Available	Specify how to evaluate conditions if multiple data check conditions exist for the same column number. [AND evaluation]:(default) Check result will be matched only if all conditions specified in [Data check condition list] match. [OR evaluation]: Check result will be matched if any of the conditions specified in [Data check condition list] matches.	Enabled if [Specify the evaluation method of multiple conditions for the same column number] is [Checked].

Data processing method

Item name	Required/Optional	Use of variables	Description	Remarks
Data processing method	Required	Not Available	Select a data processing method. [Use script settings]:(default) Apply mass data processing settings of script property to adapter. [Does not perform mass data processing]: Mass data processing is not performed. [Performs mass data processing]: Mass data processing is performed.

Comment

Item name	Required/Optional	Use of variables	Description	Remarks
Comment	Optional	Not available	You can write a short description of this adapter. The description will be reflected in the specifications.

Option

In [Option], it is possible to assign type-specific settings with "Key=Value" format. To assign multiple keys, separate them with "," (comma).
Options available for each type are as follows.

Corresponding type	Key	Meaning	Value	Remarks
String(byte size)	CS	Set the encoding to be used in the calculation of the number of bytes.	Specify an encoding that is supported in Java SE Runtime Environment 8. Refer to "Supported Encodings"(http://docs.oracle.com/javase/8/docs/technotes/guides/intl/encoding.doc.html) for details.	For information regarding the default values used if not specified in [Option], refer to "Default encoding".
Whole number	AF	Set whether to allow full-width number.	true: Allows full-width number. false:(default) Does not allow full-width number.
Decimal number	I	Set maximum number of digits of the integer part.	Available input value is "1-2147483647".
Decimal number	D	Set maximum number of digits of the fractional part.	Available input value is "1-2147483647".
Date/time	DF	Set the date format.	Date format supported in java.text.SimpleDateFormat can be set. For further details, refer to "Java(TM) Platform, Standard Edition 8 API Specification (http://docs.oracle.com/javase/8/docs/api/java/text/SimpleDateFormat.html)".

Schema

Input Schema

The same as the schema of the input source component.

Output Schema

The same as the schema of the input source component.
If [Output all records and check results] is selected in [Output record], an extra column of string type will be added to the output schema, and the check result will be output to the column as a string of "OK" or "NG".

For more details on schema structure, refer to "Table model format schema".

Loading Schema on Mapper

Schema needs to be loaded manually.
Specify the schema of reading data.

For details, refer to "Edit Schema".

Mass data processing

Mass data processing is supported.

PSP Usage

PSP is not supported.

Available Component Variables

Component variable name	Description	Remarks
count	Stores all the received record numbers.	The value defaults to 0.
matched_count	Stores the number of records that matched data check conditions.	The value defaults to 0.
unmatched_count	Stores the number of records that does not matched data check conditions.	The value defaults to 0. Detailed information on data that didn't match can be received by Data check error detail receiving operation.
message_category	Stores the category of message code corresponding to the occurred error.	The value defaults to null.
message_code	Stores the code of message code corresponding to the occurred error.	The value defaults to null.
message_level	Stores the severity of message code corresponding to the occurred error.	The value defaults to null.
error_type	Returns the error type when error occurred.	The value defaults to null. Error is represented in the format depicted below. Example:java.io.FileNotFoundException The value may vary depending on the DataSpider Servista version.
error_message	Returns the error message when error occurred.	The value defaults to null. The value may vary depending on the DataSpider Servista version.
error_trace	Returns the trace information when error occurred.	The value defaults to null. The value may vary depending on the DataSpider Servista version or the client application used.

Specification Limits

Input data can only be specified with table model type components.

Main Exceptions

Exception name	Cause	Solution
ResourceNotFoundException Resource definition could not be found.Name:[]	[Data check condition definition] is not specified.	Specify [Data check condition definition].
ResourceNotFoundException Resource definition could not found. Name:[<global resource name>]	Resource definition specified in [Data check condition definition] is not specified.	Check the global resource specified in [Data check condition definition].
InvalidPropertyConfigurationException Value of [<item name>] at line <row number> of the check condition list is incorrect.	Check condition is not specified.	Check the check condition.
InvalidPropertyConfigurationException Value is not specified in [Column number] at line <row number> of the check condition list.	[Column number] is not specified in the check condition list.	Specify [Column number].
InvalidPropertyConfigurationException [Column number] specified at line <row number> of the check condition list exceeds [<column number of input data>], the number of columns at input data.	[Column number] of check condition list is not correctly specified.	In [Column number], specify numeric values below column number of input data.