Basic tasks

Let's go through the basic required tasks to automate the data integration operations.

For information about the items created in the tasks, refer to Glossary.

Outline of basic tasks

Before creating the data integration process, you need to configure the following settings:

Next, you can start to create a data integration process and automate it:

Details of tasks

As preparation for creating the data integration process, you need to complete the initial settings described below.

 

User management

An administrator needs to set up who can access HULFT Square and what users can do through user management. Configure users, groups, and policies by performing the following operations.

  • Create groups

    Navigation menu:

    > USER MANAGEMENT > Groups

    Create groups to manage users.

     

  • Invite users

    Navigation menu:

    > USER MANAGEMENT > Users

    Invite users and add them to groups.

     

  • Create policies

    Navigation menu:

    > ACCESS MANAGEMENT > Policies

    Create policies and add required permissions to the policies. The created policies can be applied to groups on the Groups page.

For more details, refer to Set up users and groups.

 

Asset management

To perform data integration processes, you need to prepare assets such as services, resources, and profiles.

Set up those assets accordingly. Administrators and users with the appropriate permissions can set up assets.

 

  • Set storage

    Navigation menu:

    > SERVICES > Storage

    Add a file or directory if you need to.

     

  • Create a HULFT Integrate service

    Navigation menu:

    > SERVICES > HULFT Integrate

    Create an execution environment for data integration.

     

  • Create a HULFT Transfer service (optional)

    Navigation menu:

    > SERVICES > HULFT Transfer

    Create an execution environment for file transfer.

     

  • Create a VPC endpoint (optional)

    Navigation menu:

    > NETWORK > VPC Endpoint

    If the transfer direction is outbound, create a VPC endpoint.

     

  • Create a connection resource (optional)

    Navigation menu:

    > HULFT INTEGRATE > Connections

    Create a connection to a data source. You can create other required resources such as schemas or variables.

    = Remarks =

    Connection resources are known as global resources in DataSpider Servista.

    Schema resources are known as global schemas in DataSpider Servista.

     

  • Create a profile (optional)

    Navigation menu:

    > SYSTEM > Profiles

    Create profiles when you manage specific resource configurations or switch configurations. For information how to switch configurations, refer to Switch profiles between the development environment and production environment.

     

Now, you can create a data integration process.

ETL process creation

Create ETL scripts for data integration, associate them with created assets, and automate the ETL scripts.

 

(1) Create a project

Navigation menu:

> HULFT INTEGRATE > Projects

Create a project to manage ETL scripts.

 

(2) Create scripts

Navigation menu:

> HULFT INTEGRATE > Projects

Select the created project and select the Open project icon to open Designer. On Designer, you can create ETL scripts.

 

(3) Create a job

Navigation menu:

> HULFT INTEGRATE > Projects

To execute the data integration process, you need to deploy the project as a job. Select the project and select the Create new job icon. Then specify the following items required to automate the process:

  • Type of job

  • Script to execute

  • Profile (resources)

  • Services

 

(4) Check event logs

Navigation menu:

> SERVICE EVENTS

After executing the job, you can view the details of events related to script executions and file transfers.

Navigation menu:

> JOB EVENTS

You can check the execution status of jobs.

You can display details about the script executed by the job.

 

Refer to Tutorial. You can find useful practical user scenarios.