Execution in multithreaded process

Data processing can be executed in a multithreaded process. If you specify to execute transformation in a multithreaded process, because it is possible to use system resources more efficiently than in a single-thread process, data processing finishes faster.

To execute data processing in a multithreaded process, specify such execution in the Options tab of the System Environment Settings screen. In addition, you can also specify whether to execute in a multithreaded process in the Options tab of the Data Processing Settings screen.

= Remarks =

  • In step execution of data processing, data processing is always executed in a single-thread process.

  • In a test execution of data processing, data processing is always executed in a single-thread process.

Note

  • Depending on settings of methods to input/output data, methods to communicate with a database, and details of data processing, processing might not necessarily become faster.

  • If you execute data processing in a multithreaded process, each of the following increases; -

    • Use of memory

    • Use of CPU resources (especially multi-CPUs)

    If other programs (such as a database system) are running concurrently with DataMagic on the same server, the performance of the programs might be degraded. When you specify an operation in a multithreaded process, we recommend that first you confirm the overall performance of the server beforehand.

  • The multithreaded settings are enabled according to a priority order. If more highly prioritized settings are applied, any lower prioritized settings are ignored.

    1. The utled command (-thread)

    2. The Options tab on the Data Processing Settings screen

    3. The fields on the System Environment Settings screen, or the Use of multiple threads field (ed_use_thread) in the system environment settings file (huledenv.conf)

  • With a single-core processor, on the contrary, the throughput might decrease when an instance of data processing over data in Text format is performed in multi-threaded processes.