[Glossary] Data Integration
Data integration is the process of extracting and combining data from various sources, changing its format, and storing the data in destinations.
In HULFT Square, you can handle various input and output data from different resources via connectors even without the deep knowledge of the specification of each resource. The following diagram depicts how data integration is realized in HULFT Square.
Figure 4.1 Data integration overview
Designer
Designer enables you to develop, run, and debug ETL scripts graphically. No coding is required to create the scripts. You can set up data connections and data integration processes by simply dragging icons and setting configurations.
To launch Designer, select the Open project icon for the target project on the Projects page.
Figure 4.2 Designer
Operations
Operations are units that perform processing such as reading, converting, and writing data. Each operation on Designer is displayed as an icon (component icon). For example, an operation to read data from a CSV file is called the Read CSV File operation and is represented by the component icon csv_read.
Components
A component represents an operation.
Components are categorized into the following types:
-
Connector
-
Converter
-
Script component
In the Tool palette in Designer, components are displayed as icons.
Figure 4.3 Components
A connector is a component to read and write data mainly from external systems such as databases. The following categories of connectors are available:
-
Basic
-
Database
-
File
-
Application
-
Network
-
Cloud
-
Cryptography
-
HULFT
Connectors are displayed in the lower part of the Tool palette in Designer.
Figure 4.4 Connectors
Some of the connectors may require additional licenses in order to use them.
Connectors are known as adapters in DataSpider Servista.
Converters
A converter is a component for converting variables and result data read by a connector.
The following types of converters are available:
-
Mapper
-
Character
-
XSLT
In the Tool palette in Designer, converters are displayed in the Basic and Conversion categories.
Figure 4.5 Converters
Mappers
A Mapper is a component that converts or modifies data read by specific components and writes the data to other components or assigns the data to variables.
The following types of Mappers are available:
-
Document Mapper
-
Variable Mapper
-
Merge Mapper
In the Tool palette in Designer, the three Mappers are displayed in the Basic and Conversion categories.
Figure 4.6 Mappers
The Mapper editor is used to set up Mappers.
Script components
Script components represent the functions required to create scripts, such as controlling flows, calling other scripts, and creating memos.
In the Tool palette of Designer, they belong to the Basic category and are classified into the Processing, Flow, or Others subcategories.
Figure 4.7 Script components