Basic tasks
Let's go through the basic required tasks to automate the data integration operations.
For information about the items created in the tasks, refer to Glossary.
Outline of basic tasks
Before creating the data integration process, you need to configure the following settings:
User management
Next, you can start to create a data integration process and automate it:
ETL process creation
Details of tasks
As preparation for creating the data integration process, you need to complete the initial settings described below.
User management
An administrator needs to set up who can access HULFT Square and what users can do through user management. Configure users, groups, and policies by performing the following operations.
-
- Navigation menu:
-
> USER MANAGEMENT > Groups
Create groups to manage users.
-
- Navigation menu:
-
> USER MANAGEMENT > Users
Invite users and add them to groups.
-
- Navigation menu:
-
> ACCESS MANAGEMENT > Policies
Create policies and add required permissions to the policies. The created policies can be applied to groups on the Groups page.
For more details, refer to Set up users and groups.
Asset management
To perform data integration processes, you need to prepare assets such as services, resources, and profiles.
Set up those assets accordingly. Administrators and users with the appropriate permissions can set up assets.
-
- Navigation menu:
-
> SERVICES > Storage
Add a file or directory if you need to.
-
Create a HULFT Integrate service
- Navigation menu:
-
> SERVICES > HULFT Integrate
Create an execution environment for data integration.
-
Create a HULFT Transfer service (optional)
- Navigation menu:
-
> SERVICES > HULFT Transfer
Create an execution environment for file transfer.
-
Create a VPC endpoint (optional)
- Navigation menu:
-
> NETWORK > VPC Endpoint
If the transfer direction is outbound, create a VPC endpoint.
-
Create a connection resource (optional)
- Navigation menu:
-
> HULFT INTEGRATE > Connections
Create a connection to a data source. You can create other required resources such as schemas or variables.
= Remarks =Connection resources are known as global resources in DataSpider Servista.
Schema resources are known as global schemas in DataSpider Servista.
-
- Navigation menu:
-
> SYSTEM > Profiles
Create profiles when you manage specific resource configurations or switch configurations. For information how to switch configurations, refer to Switch profiles between the development environment and production environment.
Now, you can create a data integration process.
Create ETL scripts for data integration, associate them with created assets, and automate the ETL scripts.
- Navigation menu:
-
> HULFT INTEGRATE > Projects
Create a project to manage ETL scripts.
- Navigation menu:
-
> HULFT INTEGRATE > Projects
Select the created project and select the Open project icon to open Designer. On Designer, you can create ETL scripts.
- Navigation menu:
-
> HULFT INTEGRATE > Projects
To execute the data integration process, you need to deploy the project as a job. Select the project and select the Create new job icon. Then specify the following items required to automate the process:
-
Type of job
-
Script to execute
-
Profile (resources)
-
Services
Refer to Tutorial. You can find useful practical user scenarios.