Filter:
Submit Search

Extract table data

Operation name

Extract table data

Function overview

Obtains data from a table on Google BigQuery, and outputs it to a file on Google Cloud Storage.

Data model

The data model of this component is XML type.

Properties

= Remarks =

For details on use of variables, refer to Variables.

Basic settings

Item name

Required/Optional

Use of variables

Description

Remarks

Name

Required

Not available

Enter a name that is used on the script canvas.

 

Required settings

Item name

Required/Optional

Use of variables

Description

Remarks

Destination

Required

Not available

Select a connection resource.

Add

A new connection resource can be added.

Edit list

Connection resource settings can be edited in > HULFT INTEGRATE > Connections.

 

Project ID

Required

Available

Select or enter the project ID.

 

Dataset name

Required

Available

Select or enter the dataset name.

 

Table name

Required

Available

Select or enter the table name.

 

Destination bucket name

Required

Available

Select or enter the destination bucket name.

 

Destination folder path

Required

Available

Select or enter the destination folder path.

  • Specify a slash "/" at the beginning and the end.

  • Consecutive slashes "//" can't be included.

  • If the specified folder doesn't exist, the folder is created.

Note

Specify an absolute path of Google Cloud Storage for the folder path.

Destination file name

Required

Available

Select or enter the destination file name.

  • "/" can't be included.

  • If the output file size exceeds 1 GB, you must include an asterisk "*".

  • For details on the operation when an asterisk "*" is included in the destination file name, refer to the following Web site that describes exporting table data on Google BigQuery (https://cloud.google.com/bigquery/docs/exporting-data#exporting_data_into_one_or_more_files).

Overwrite file when one exists

Optional

Not available

Select whether to overwrite the file when a file with the same name as the destination file for writing exists.

Selected

Overwrite.

Not selected

(Default)

Don't overwrite. Raise an error without overwriting the file when the specified write destination file already exists.

  • When an asterisk "*" is included in Destination file name, this item will be selected and disabled.

  • When an asterisk "*" is included in Destination file name and the script is executed, it performs the same as when this item is selected.

Property actions

Item name

Description

Remarks

Update the list of project IDs

Get projects in the specified destination and set them to Project ID.

  • If Destination is specified/edited or projects have been added/edited on Google BigQuery, the changes can be reflected using this property action.

Update the list of dataset names

Get datasets in the specified project and set them in Dataset name.

  • If Project ID is specified/edited or a dataset has been added/edited on Google BigQuery, the changes can be reflected using this property action.

  • If a variable is set to Project ID, this action will be invalid.

Update the list of table names

Get tables in the specified dataset and set them in Table name.

  • If Dataset name is specified/edited or a table has been added/edited on Google BigQuery, the changes can be reflected using this property action.

  • If a variable is set to Project ID or Dataset name, this action will be invalid.

Update the list of destination bucket names

Get buckets in the specified destination and set them in Destination bucket name.

  • If Destination is specified/edited or a bucket has been added/edited on Google Cloud Storage, the changes can be reflected using this property action.

Update the list of destination folder paths

Get folder paths in the specified bucket and set them in Destination folder path.

  • If Destination bucket name is specified/edited or a folder has been added/edited on Google Cloud Storage, the changes can be reflected using this property action.

  • If a variable is set to Destination bucket name, this action will be invalid.

Write settings

Item name

Required/Optional

Use of variables

Description

Remarks

Output file format

Optional

Not available

Select an output file format.

CSV

(Default)

Write data to Google Cloud Storage in CSV format.

TSV

Write data to Google Cloud Storage in TSV format.

 

Insert To The First Row

Optional

Not available

Select whether to obtain the column name in the table definition, and output it to the first row of the output file.

Selected

Output the column name.

Not selected

(Default)

Don't output the column name.

 

Comment

Item name

Required/Optional

Use of variables

Description

Remarks

Comment

Optional

Not available

You can write a short description of this connector.

 

Schemas

Input schema

None.

Output schema

<?xml version="1.0" encoding="UTF-8"?>
<bucket name=" >
 <files folderPath=" >
  <file name=" />
 </files>
</bucket>

Element name

Attribute name

Description

Remarks

bucket

-

Appears once for each specified bucket.

 

name

Outputs the input value of the Destination bucket name property as is.

 

files

-

Appears once for each specified folder.

 

folderPath

Outputs the input value of the Destination folder path property as is.

 

file

-

Appears once for each file to which data was output.

 

name

Output the file name to which data was output.

 

Loading schema in Mapper

The schema is loaded automatically.

= Remarks =

For details, refer to Edit Schema.

Mass data processing

Mass data processing isn't supported.

Transaction

Transaction isn't supported.

Parallel Stream Processing

PSP isn't supported.

Available component variables

Component variable name

Description

Remarks

message_category

When an error occurs, the category of the message code corresponding to the error is stored.

  • The default value is null.

message_code

When an error occurs, the code of the message code corresponding to the error is stored.

  • The default value is null.

message_level

When an error occurs, the severity of the message code corresponding to the error is stored.

  • The default value is null.

error_type

When an error occurs, the error type is stored.

  • The default value is null.

  • The format of the error type is as follows.

    Example: java.io.FileNotFoundException

error_message

When an error occurs, the error message is stored.

  • The default value is null.

error_trace

When an error occurs, the trace information for the error is stored.

  • The default value is null.

Message codes, exception messages, and limitations

Connector

Message code

Exception message

Limitations

Messages and limitations of the Google BigQuery connector

check

check

check

Policies     (c) Saison Technology Co.,Ltd. 2021

TOP