Smart Compiler

Overview of Smart Compiler

Smart Compiler is a feature to analyze script content to detect portions where Parallel Stream Processing (hereafter PSP) is applicable and implement it automatically.

Use of PSP maximizes the performance of script.

= Reference =

For details about PSP, refer to the section Parallel Stream Processing.

How to use

The Smart Compiler function is enabled by default.

You can switch between enable and disable in Option > Edit under the Tools menu of Designer.

Creating a data flow

When the Smart Compiler function is enabled and a data flow is created on the script, it checks if execution as PSP is possible. If possible, PSP is applied automatically.

A data flow with PSP is called PSP data flow.

Running a script

Run a script. The PSP data flow portion is executed as PSP.

The status of processing for PSP is output to an XML log.

See an output example of the XML log when the script shown below is executed.

Output example

Starting executing scripts.
 
****** start csv_read processing ******
[csv_read(script@35b45d3f)]PSP started.
 
****** start csv_write processing ******
Extracted data within [100]row.
Inserted data within [100]row.
Finished [csv_read(script@35b45d3f)]PSP.
 
Finished executing scripts.

= Hint =

You can filter execution logs by selecting the icons displayed in the execution history of Designer, even if these are executed as PSP.

In this way, you can check one by one the content of operations executed in parallel, such as Mapper and component icons.

= Reference =

For details, refer to Filtering execution log.

How to check PSP data flow

To check if the script data flow is PSP data flow, select Show PSP data flow in the View menu of Designer.

= Reference =

For details about the menu, refer to View menu .

When the view setting is enabled, PSP data flow is displayed with a thick yellow border.

Also, when selecting components, the set of operations incorporated into PSP from read through write blinks.

Bulk changes

To change data flows to PSP data flows all at once, in Project explorer in Designer, select PSP data flow operation > Enable at once from the right-click menu of a project or script.

Use this function to apply PSP to projects that are already created.

Note

If multiple data flows are drawn from the same component, you can't change to PSP data flow.

= Reference =

For details about the menu of Designer, refer to Project explorer .

Manual setting

To manually set a PSP data flow, select Enable/Disable PSP data flow from the right-click menu of the data flow.

Use this method in cases when multiple data flows are drawn from the same component, but data isn't reused due to certain settings such as conditional branches.

Specification limitations

When you draw multiple data flows from the same component, PSP data flow becomes temporarily invalid.

This behavior is caused by specification limitations for PSP. However, you can manually set PSP depending on the situation, such as when you use conditional branches.
If you draw multiple data flows from the same component and a component that isn't supported for PSP is included as a target, manual settings can't be applied.