Extract Table Data

Operation Name

Extract table data

Function Overview

Obtain data from a table on Google BigQuery, and extract it to a file on Google Cloud Storage.

Data Model

The data model of this component is XML type.

Properties

For details on using variables, refer to "Variables".

Basic Settings

Item name	Required/Optional	Use of variables	Description	Remarks
Name	Required	Not Available	Enter the name on the script canvas.

Required Settings

Item name	Required/Optional	Use of variables	Description	Remarks
Destination	Required	Not Available	Select Global Resources. [Add]: Adds a new global resource. [Edit list]: Allows you to edit the global resource settings on the "Edit resource list" screen.
Project ID	Required	Available	Select or enter the project ID.
Dataset name	Required	Available	Select or enter the dataset name.
Table name	Required	Available	Select or enter the table name.
Destination bucket name	Required	Available	Select or enter the destination bucket name.
Destination folder path	Required	Available	Select or enter the destination folder path.	Specify a folder path as an absolute path on Google Cloud Storage. Specify a slash (/) at the beginning and the end. Consecutive slashes (//) cannot be included. Creates the folder when specified folder does not exist.
Destination file name	Required	Available	Select or enter the destination file name.	A slash (/) cannot be included. If the output file size exceeds 1 GB, you must include an asterisk (). For details on the operation when an asterisk () is included in the destination file name, refer to the following Web site that describes extracting table data on Google BigQuery (https://cloud.google.com/bigquery/docs/exporting-data#exporting_data_into_one_or_more_files).
Overwrite file when one exists	Optional	Not Available	Select whether to overwrite a file when a file with the same name as the write file exists. [Checked]: Overwrites file. [Not checked]: (default) File is not overwritten. Raises an error without overwriting a file when the specified write destination file exists.	When an asterisk () is included in [Destination file name]* this is [Checked] and the function is disabled. When an asterisk () is included in [Destination file name]* and a script is executed, it performs as [Checked].

Property Action

Item name	Description	Remarks
Update the list of project IDs	Get projects in the specified destination, and set them to [Project ID].	If [Destination] is specified/edited, or projects have been added/edited on Google BigQuery, you can apply the changes by using this property action.
Update the list of dataset names	Get datasets in the specified project, and set them in [Dataset name].	If [Project ID] is specified/edited, or a dataset has been added/edited on Google BigQuery, you can apply the changes by using this property action. Disabled if variable is set to [Project ID].
Update the list of table names	Get tables in the specified dataset, and set them in [Table name].	If [Dataset name] is specified/edited, or a table has been added/edited on Google BigQuery, you can apply the changes by using this property action. Disabled if variable is set to [Project ID] or [Dataset name].
Update the list of destination bucket names	Get buckets in the specified destination, and set them in [Destination bucket name].	If [Destination] is specified/edited, or a bucket has been added/edited on Google Cloud Storage, you can apply the changes by using this property action.
Update the list of destination folder paths	Get folder paths in the specified bucket, and set them in [Destination folder path].	If [Destination bucket name] is specified/edited, or a folder has been added/edited on Google Cloud Storage, you can apply the changes by using this property action. If a variable is set to [Destination bucket name], this action will be invalid.

Write Settings

Item name	Required/Optional	Use of variables	Description	Remarks
Output file format	Optional	Not Available	Select an output file format. [CSV]: (default) Write data to Google Cloud Storage in CSV format. [TSV]: Write data to Google Cloud Storage in TSV format.
Insert To The First Row	Optional	Not Available	Select whether to obtain the column name in the table definition, and output it to the first row of the output file. [Checked]: Output the column name. [Not Checked]: (default) Do not output the column name.

Comment

Item name	Required/Optional	Use of variables	Description	Remarks
Comment	Optional	Not Available	You can write a short description of this adapter. The description will be applied in the specifications.

Schema

Input Schema

None

Output Schema

<?xml version="1.0" encoding="UTF-8"?>
<bucket name="" >
 <files folderPath="" >
  <file name="" />
 </files>
</bucket>

Element name	Attribute name	Description
bucket	-	Appears in each specified bucket.
bucket	name	Output the input value of property in the [Destination bucket name] as is.
files	-	Appears in each specified folder.
files	folderPath	Output the input value of property in the [Destination folder path] unchanged.
file	-	Appears for each file for which data was output.
file	name	Output the file name for which data was output.

Loading Schema in Mapper

Schema is loaded automatically.

For details, refer to "Edit Schema".

Mass Data Processing

Mass Data Processing is not supported.

Transaction

Transaction is not supported.

PSP Usage

PSP cannot be used.

Available Component Variables

Component variable name	Description	Remarks
message_category	When an error occurs, the category of the message code corresponding to the error that occurred is stored.	The default value is null.
message_code	When an error occurs, the code of the message code corresponding to the error that occurred is stored.	The default value is null.
message_level	When an error occurs, the severity of the message code corresponding to the error that occurred is stored.	The default value is null.
error_type	When an error occurs, the type of error that occurred is stored.	The default value is null. Error type is in the following format: Example: "java.io.FileNotFoundException" The contents to be stored may vary depending on the version of DataSpider Servista.
error_message	When an error occurs, the message of the error that occurred is stored.	The default value is null. The contents to be stored may vary depending on the version of DataSpider Servista.
error_trace	When an error occurs, the trace information of the error that occurred is stored.	The default value is null. The contents to be stored may vary depending on the version and client applications of DataSpider Servista.

Specification Limits

When you interrupt an operation for a file that includes an asterisk (*) in the file name, files that have already been output remain on Google Cloud Storage.

For more details on specification limits common to each operation, refer to "Google BigQuery Adapter Limitations".

Major Exceptions

Exception Name	Cause	Solution
ResourceNotFoundException Resource definition is not found. Name:[]	[Destination] is not specified.	Specify [Destination].
ResourceNotFoundException Resource definition is not found. Name:[<Name of global resources>]	Resource definition selected in [Destination] is not found.	Check the global resource specified in [Destination].
InvalidPropertyConfigurationException <property name> is not specified.	[<Property name>] is not specified.	Specify [<Property name>].
FileAlreadyExistsException	The file already exists.	Check the setting value for the operation or the status of Google Cloud Storage.
BigQueryServiceException Failed to execute the job.	Failed to execute the job.	Identify the cause of the error by referring to the error logs, and resolve the problem.
BigQueryServiceException Failed in the operation for Google BigQuery.	Failed in the operation for Google BigQuery	Identify the cause of the error from the status code and error code by referring to the error message list page of Google BigQuery (https://cloud.google.com/bigquery/docs/error-messages), and then resolve the problem.
CloudStorageServiceException	Failed in the operation for Google Cloud Storage.	Identify the cause of the error from the status code and error code by referring to the status code and error code list page of Google Cloud Storage (https://cloud.google.com/storage/docs/json_api/v1/status-codes), and then resolve the problem.
ResourceManagerServiceException	Failed in the operation for Google Resource Manager.	Identify the cause of the error from the status code and error code by referring to the error message list page of Google Resource Manager (https://cloud.google.com/resource-manager/docs/core_errors), and then resolve the problem.

Notes

HTTPS is used for communication between Google BigQuery/Google Cloud Storage and this adapter.