Data Check

Operation Name

Data Check

Function Overview

Check the values and formats of input data and output the result in selected format.

Properties

For details on use of variables, refer to “Variables”.
Basic settings
Item name Required/Optional Use of variables Description Remarks
Name Required Not available Enter the name to show on the script canvas.  
Input data Required Not available Select a component on the script canvas.  
Required settings
Item name Required/Optional Use of variables Description Remarks
Use global resource for data check condition Required Not available Select whether or not to use global resource for data check condition.
  • [Checked]:
    Use global resource.
  • [Not checked]: (default)
    Do not use global resource.
  
Data check condition definition Required Not available Select a data check condition definition registered in global resources.

Items that can be set in global resource are same as that of [Data check condition list].
  • [Add]:
    Add a new global resource.
  • [Edit list]:
    Global resource settings can be edited in “Edit resource list” screen.
  
Data check condition list Optional - Specify conditions with which input data columns will be checked.

Each column can be operated with the following buttons.
  • [Add]:
    Add a column.
  • [Up]:
    Move the selected column upwards by one row.
  • [Down]:
    Move the selected column downwards by one row.
  • [Delete]:
    Delete the column.
  • Displayed when [Use global resource for data check condition] is [Not checked].
Data check condition list/Column number Required Available Enter a column number of input data to be a target.  
Data check condition list/Required Optional Not available Select whether or not to set [Column number] data as a required field.
  • [Checked]:
    It is set as required field.
  • [Not checked]:
    It is not set as required field.
  • When the column where [Required] is checked of the input data is null or empty string, [Type][Minimum][Maximum][Option][Regular expression] will not be checked after that.
Data check condition list/Type Required Not available Select in what type to check the data of [Column number].
  • [String (character count)]: (default)
  • [String (byte count)]:
  • [Integer]:
  • [Decimal]:
  • [Date/Time]:
  • When specifying [Integer] or [Decimal] in [Type] and the input data to be checked is a non-numeric value, such as "abc", [Minimum][Maximum][Option][Regular expression] will not be checked after that.
Data check condition list/Minimum Optional Available Enter the minimum number of digits, minimum bytes, or value, to allow for [Column number] data.
  • When [String (character count)] is specified in [Type], [Minimum] will be handled as number of digits.
  • When [String (byte count)] is specified in [Type], [Minimum] will be handled as byte count.
  • When [Integer] or [Decimal] is specified in [Type], [Minimum] will be handled as value.
  • When [Date/Time] is specified in [Type], value set in [Minimum] will be ignored.
Data check condition list/Maximum Optional Available Enter the maximum number of digits, maximum bytes, or value, to allow for [Column number] data.
  • When [String (character count)] is specified in [Type], [Maximum] will be handled as number of digits.
  • When [String (byte count)] is specified in [Type], [Maximum] will be handled as byte count.
  • When [Integer] or [Decimal] is specified in [Type], [Maximum] will be handled as value.
  • When [Data/Time] is specified in [Type], the value set in [Maximum] will be ignored.
Data check condition list/Option Optional Available Specify the option to check [Column number] data with.
  • For more details on how to specify option, refer to “Option”.
Data check condition list/Regular expression Optional Available Specify the regular expression to check [Column number] data with.
  • The available regular expression patterns conform to java.util.regex.Pattern.
    For more details on Pattern, refer to “Java(TM) Platform, Standard Edition 8 API Specification” (http://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html).
Output record Required Not available Select a record type to be output.
  • [Output only records matching conditions]: (default)
    Output only records that match the conditions.
  • [Output only records not matching conditions]:
    Output only the records that does not match the conditions.
  • [Output all records and check results]:
    Assign check results to all the read records and output that.
  • When [Output all records and check results] is selected, a check result column will be added to columns of input data and output.
Condition evaluation settings
Item name Required/Optional Use of variables Description Remarks
Evaluation method of multiple conditions Required Not available Set how to evaluate multiple data check conditions.
  • [AND evaluation]: (default)
    Check will be passed through when all the conditions are satisfied.
  • [OR evaluation]:
    Check will be passed through when any of the conditions is satisfied.
When there are multiple data check conditions for the same column number in [Data check condition list], you can set the evaluation method for that using [Specify evaluation method of multiple conditions for same column number] and [Evaluation method for same column number]. 		 
Specify evaluation method of multiple conditions for same column number Optional Not available Select whether or not to specify how to evaluate multiple data check conditions for the same column number.
  • [Checked]:
    Use the evaluation method specified in [Evaluation method for same column number] to check.
  • [Not checked]: (default)
    Use the evaluation method specified in [Evaluation method of multiple conditions] to check even when there are multiple data check conditions for the same column number.
 
Evaluation method for same column number Optional Not available Set how to evaluate multiple data check conditions for same column number.
  • [AND evaluation]: (default)
    Check will be passed through when all the conditions specified for the column in [Data check condition list] are satisfied.
  • [OR evaluation]:
    Check will be passed through when any of the conditions specified for the column in [Data check condition list] is satisfied.
  • Enabled when [Specify evaluation method of multiple conditions for same column number] is [Checked].
Data processing method
Item name Required/Optional Use of variables Description Remarks
Data processing method Required Not available Select data processing method.
  • [Apply script settings]: (default)
    Apply the mass data processing settings of the script property to the adapter.
  • [Do not perform mass data processing]:
    Mass data processing will not be performed.
  • [Perform mass data processing]:
    Mass data processing will be performed.
  
Comment
Item name Required/Optional Use of variables Description Remarks
Comment Optional Not available You can write a short description of this adapter.
The description will be reflected in the specifications.		  

Option

In option items, type specific settings can be assigned in “key=value” format. When setting multiple keys, delimit them using “,” (comma).
Options that can be set for each type are as follows.

Corresponding type Key Meaning Value Remarks
String (byte count) CS Set the encoding to be used in calculation of byte count. Specify the encoding supported in Java SE Runtime Environment 8.
For more details, refer to “Supported Encodings” (http://docs.oracle.com/javase/8/docs/technotes/guides/intl/encoding.doc.html).
Integer AF Set whether to allow full-width digits.
  • true:
    Full-width digits are allowed.
  • false: (default)
    Full-width digits are not allowed.
  
Decimal I Set the maximum number of digits for the integer part. Allowed values are 1-2147483647.  
Decimal D Set the maximum number of digits for the fractional part. Allowed values are 1-2147483647.  
Date/Time DF Set the date format. Date format supported in java.text.SimpleDateFormat can be set.
For more details on SimpleDateFormat, refer to “Java(TM) Platform, Standard Edition 8 API Specification”
(http://docs.oracle.com/javase/8/docs/api/java/text/SimpleDateFormat.html).
  

Schema

Input schema

Same as the schema of input component.

Output schema

Same as the schema of input component.
When [Output all records and check results] is selected in [Output record], one string type column is added to the schema of result data and the check result for record will be output as "OK" / "NG".
Refer to “Schema of table model type” for more details.

Loading schema in Mapper

Schema should be loaded manually.
Specify the schema of the data to be read.
Refer to “Edit Schema” for more details.

Mass Data Processing

Mass Data Processing is supported.

PSP Usage

PSP is not supported.

Available Component Variables

Component variable name Description Remarks
count The number of all read records is stored.
  • Default value is 0.
matched_count The number of records matched the data check conditions is stored.
  • Default value is 0.
unmatched_count The number of records that did not match the data check conditions is stored.
message_category When an error occurs, the corresponding message code's category is stored.
  • Default value is null.
message_code When an error occurs, the corresponding message code is stored.
  • Default value is null.
message_level When an error occurs, the corresponding message code's severity is stored.
  • Default value is null.
error_type When an error occurs, the error type is stored.
  • Default value is null.
  • The error type will be in the following format.
    Example: “java.io.FileNotFoundException”
  • The contents that will be stored may vary depending on DataSpider Servista versions.
error_message When an error occurs, the error message is stored.
  • Default value is null.
  • The contents that will be stored may vary depending on DataSpider Servista versions.
error_trace When an error occurs, the error's trace information is stored.
  • Default value is null.
  • The contents that will be stored may vary depending on DataSpider Servista versions and client applications.

Specification Limits

Major Exceptions

Exception name Cause Solution
ResourceNotFoundException
Resource definition is not found. Name: []		 [Data check condition definition] is not specified. Specify [Data check condition definition].
ResourceNotFoundException
Resource definition is not found. Name: [<global resource name>]		 The resource definition selected in [Data check condition definition] is not found. Check the global resource specified in [Data check condition definition].
InvalidPropertyConfigurationException
The value of [<item name>] in row [<row number>] of the check condition list is incorrect.		 Check conditions are not specified properly. Confirm the check conditions.
InvalidPropertyConfigurationException
Value is not entered in [Column number] specified in row [<row number>] of the check condition list.		 [Column number] of check condition list is not entered. Enter [Column number].
InvalidPropertyConfigurationException
The value of [Column number] specified in row [<row number>] of the check condition list exceeds the number of columns of input data [<the number>].		 [Column number] of check condition list is not specified properly. For [Column number], specify a number not larger than the number of columns in input data.