Other auxiliary components

HULFT Square offers various auxiliary components to enable you to describe operations with greater flexibility.

This section describes the following most frequently used components in order:

1. Monitor exception

2. Conditional branch

3. Mapper

4. Multi-Stream Converter (Join, Aggregate, Sort)

 

The following script is used for explanation.

For the CSV file, use the same file as Create a script that processes a CSV file.

Prepare the CSV file beforehand at the following path in the storage.

/Personal/data/inputdata.csv

Monitor exception

Monitor exception is a component for monitoring exceptions for target processing and performing exception processing.

This example explains how to send an email with the message "An error has occurred." when an exception occurs during the reading and writing processing of the CSV file.

For details about the settings and notes for each field for monitor exceptions, refer to Monitor Exception.

1. From the Tool palette in Designer, drag Basic > Flow > Monitor Exception onto the script canvas.



2. When you drag the Monitor Exception operation, the following component icon is placed on the script canvas.

   • Start Monitor Exception

     

   • Try

     

   • Catch

     

   • End Monitor Exception

     

3. Place the monitor target processing (reading and writing processing of the CSV file) between Try and End Monitor Exception.

   

4. Create processing when an exception occurs (processing to send the error by email).

   From the Tool palette in Designer, drag Network > Mail > Send One E-mail onto the script canvas.



The New Send One E-mail operation dialog opens.

5. In the Required settings tab, set the To/Cc/Bcc mail address.

6. In Subject on the Message tab, enter "An error has occurred.".

   

7. Click Finish.

8. Place the Send One E-mail operation between Catch and End Monitor Exception.



9. Temporarily rename the /Personal/data/inputdata.csv CSV file with another name to cause an error in the reading processing of the CSV file.

10. Run the script.



11. Confirm that the mail has arrived at the specified To/Cc/Bcc mail address after the script has ended.

12. After verifying the log, return the name of the CSV file that you renamed in step 8 back to /Personal/data/inputdata.csv.

Conditional branch

Setting up a conditional branch enables you to change the content of processing when it meets certain conditions.

For details about the settings and notes for each field for conditional branches, refer to Conditional Branch.

1. From the Tool palette in Designer, drag Basic > Flow > Conditional Branch onto the script canvas.



The New Condition dialog opens.



2. Click Add.

   The condition Condition is not set is added to the Condition list.

3. Double-click Condition is not set.

   The condition changes to (Click to edit).



4. Click the condition and specify the following:

   For Type of condition, select Comparison between variable and fixed value.

   Specify "not less than" and "1" for Variable "csv_write:count".



5. Select Finish.

6. Place the component in such a way that the flow branches at the condition before the End icon.



7. From the Tool palette in Designer, drag Basic > Flow > end onto the script canvas.

8. Draw a flow from condition to the End icon.



9. Select the End icon connected to condition and open the property inspector.

10. For Return value, enter 1.



When data isn't written to /Personal/data/outputdata.csv, the return code of the script becomes 0.

When data is written, the return code of the script becomes 1.

Mapper

Mapper is a component that converts or modifies data read by one component and writes that data to another component, or assigns data to variables by using a dedicated GUI tool (Mapper editor).

Mapper is the general term for the following three component types:

• Document Mapper

  (Operation name: Mapping)

• Variable Mapper

  (Operation name: Assign Variables)

• Merge Mapper

  (Operation name: Merge)

The main functions of Mapper are as follows:

• Arithmetic/Logical calculation

• String manipulation

• Conditional data extraction

• Sorting

• Encoding conversion

• Merge

Connect the items displayed at the right and left of the Mapper editor screen with a line, and specify the mapping for input and output. You can also perform various calculations and operations according to logic functions by placing Mapper logic within the mapping.

 

In this example, you'll use Document Mapper to create a script for converting and writing data read from a CSV file.

1. From the Tool palette in Designer, drag Conversion > Basic > Mapping onto the script canvas.



2. Draw a process flow and a data flow from csv_read to mapping.

3. In the same way, draw a process flow and a data flow from mapping to csv_write.



4. Double-click csv_write.

   The Write CSV File operation properties dialog opens.



5. Click Add, double-click the Column name input field, and enter productName.

6. Click Add once more, double-click the Column name input field, and enter salesAmount.



7. Click Finish.

   The Write CSV File operation properties dialog closes.

8. Double-click the mapping icon.

   The Mapper editor opens.



9. From the input source component schema in the left pane, select the element productName and drag it onto the element productName of the output target component schema in the right pane.



The data of the element productName is transferred to the output target without being converted.

The data of the element salesAmount is converted after it's read from the input source and transferred to the output target.

10. From the Tool palette in Designer, drag Number > Basic > Numeric Constant onto the mapping canvas.



11. Double-click the Numeric Constant logic that you just placed.

The Numeric Constant Logic dialog opens.

In Comment, enter add 50, and in Number, enter 50.



12. Click Finish.

13. From the Tool palette in Designer, drag Number > Calculation > Add onto the mapping canvas in the same way.

14. Click Finish.

15. From the input source component schema on the left, select the element salesAmount and drag it onto Add.

16. Drag Numeric Constant onto Add.

17. Select Add and drag it onto the element salesAmount in the output target component schema on the right.



18. Save the script.

19. Run the script.



20. Open Personal/data/outputdata.csv from storage, and verify the content of the written data to see whether the value for the salesAmount is properly converted from Personal/data/inputdata.csv.

Multi-Stream Converter (Join, Aggregate, Sort)

Besides Mapper, specialized components are also provided for Join, Aggregate, and Sort operations.

 

These components specialized for table model type are run by Multi-Stream Converter, a high-speed engine optimized for conversions, and are good at handling mass data. Especially the use of a multi-core CPU allows for the effective use of resources, parallel processing, and high-speed execution with low memory use (compared to conventional Mappers).

Multi-Stream Converter provides maximum results when combined with components that support Multi-Thread Processing.

It also features a specialized UI for each operation and can be intuitively operated and configured.

In addition, Multi-Stream Converter has the following features:

• Join

• Conditions for joining are like SQL and are easy to understand.

• Two data sets are joined in one operation.

• Aggregate

• It can be used by table model type components.

• Input data can be grouped and various aggregate functions can be specified.

• Sort

• The number of sort keys is not limited.

• You can specify the following for keys:

• String

• Number

• Date/Time

• Strings that represent numbers or date/time

By understanding and appropriately using these features, you can develop scripts that can handle mass data such as for IoT or big data.

 

Depending on the situation, the use of Mapper may also be a suitable option. The section below describes for which actions for you can use Mapper features instead of Multi-Stream Converter.

 

The Mapper features that correspond to the Join feature are as follows:

• And-Integrate Merge

• Left-Right-Join Merge

Use Mapper for the following actions:

• Joining three or more data sets in one operation

• Joining by XML type components

• Graphically defining small data input and output mapping in a dedicated GUI

 

The Mapper features that correspond to the Aggregate feature are as follows:

• Grouping

• Sum

• Number of Node

• Maximum Node

• Minimum Node

Use Mapper for the following actions:

• Aggregating by XML type components

• Combining with Mapper logic to perform aggregation

 

The Mapper features that correspond to the Sort feature are as follows:

• Sort by Key

• Sort by Two Keys

Use Mapper for the following actions:

• Sorting by XML type components

• Combining with Mapper logic to perform sorting

• Sorting by specifying the order of priority of upper case and lower case characters in character string order