Read Data (Replication)

Operation Name

Read Data (Replication)

Function Overview

This operation is to read data, using the getUpdated() or getDeleted() call of the SOAP API.

The getUpdated() retrieves a set of IDs that have been created or updated within the specified timespan.
But with this component, you can read the values of other fields using retrieve() call.

The getDeleted() retrieves a list of records (ID and deleted date) that have been deleted within the specified timespan.


With regarding to the API specification utilized by this operation, please refer to the version of the API document, selected for the Global Resource, in the below-mentioned link.

Data Model

The data model of this component is the Table Model type.

Properties

For information about using variables, refer to "variables".
Basic Settings
Item name Required/Optional Use of Variables Description Supplement
Name Required Not Available Enter the name that appears on the script canvas.  
Required Settings
Item name Required/Optional Use of Variables Description Supplement
Destination Required Not Available Select Global Resources.
  • [Add...]:
    Add new global resource.
  • [Edit...]:
    Global resource settings can be edited by [Edit Resource list].
  
Table Name Required Not Available Select the name of a table in Salesforce.  
Replication Target Required Not Available Select the replication target.
  • [Added/Updated data]:(default)
    Read data that have been added/updated during the period specified by the last updat date tab.
  • [Deleted data]:
    Read data that have been deleted during the period specified by the last update date tab.
  
Schema definition Required - Specify items to be read from Salesforce.
  • If [Deleted data] is selected in [Replication Target], the schema definition will be the fixed value.
  • Limiting the settings to only items needed in the subsequent processing (deleting unnecessary items from the Schema Definition) helps improve processing performance.
Schema definition/Label Required Not Available Display the lable name of the column of the table specified in [Table Name].  
Schema definition/API Required Not Available Display the API name of the column of the table specified in [Table Name].  
Schema definition/Type Required Not Available Display the data type of the column of the table specified in [Table Name].  
Property Action
Item name Description Supplement
Create schema... The items to be read from Salesforce can be added/deleted.
For setting methods, please refer to "Create Schema."		  
Date of Last Update
Item name Required/Optional Use of Variables Description Supplement
Period of last update :From Required Available Enter the range of dates which were added, updated or deleted in Salesforce.
  • Due to API specifications, with the date of the execution of the operation marked as the base date, any date later than 30 days after the base date cannot be specified.
  • An error will occur if any date, later than 30 days after the base date, is specified.
Period of last update :To Required Available Enter the range of dates which were added, updated or deleted in Salesforce.  
Option Settings
Item name Required/Optional Use of Variables Description Supplement
Column name type Required Not Available Choose the display type of the column name, for displaying the schema in mapping.
  • [Label]:
    The label name (item name) defined in Salesforce is displayed.
  • [API] :(default)
    The API reference name defined in Salesforce is displayed.
  
Batch size Required Available Specify a batch size for obtaining the data reading results from Salesforce.
  • This is used for retrieve() call when [Added/Updated data] is selected in [Replication Target].
  • The default value is "2000".
  • A number of 1 or greater, up to 2000, can be set. An error occurs if any number outside of this range is set.
  • Depending on the operating environment, increasing the batch size generally helps improve the performance of the read data operation.
  • According to the Salesforce API specifications, if you set a batch size to 2000 when reading 10,000 records, the result data set will be obtained in 5 parts.
Property Action
Item name Description Supplement
Display Table Information... You can check the table structure of the object you are operating.
For more information on how to view the table structure, please refer to Display Table Information		 Click [Load All Table Information....] if you would like to check the table structure of other objects.
Load All Table Information... Retrieve all possible table information
After running, you can verify the retrieved information from [Table Information].		  
Read schema definition from file... Select a file from the file chooser and read the name of field API on the first line of the file specified in the comma-separated values. Set this as the schema definition.
  • Please specify "UTF-8" encoding for the selected file.
Large Data Processing Settings
Item name Required/Optional Use of Variables Description Supplement
Large Data Processing Required Not Available Select a Large Data Processing Settings.
  • [Use the setting in the script]:(default)
    Apply Large Data Processing Settings settings of script property to adapter.
  • [Disable Large Data Processing]:
    Large Data Processing is not performed.
  • [Enable Large Data Processing]:
    Large Data Processing is performed.
  
Comment
Item name Required/Optional Use of Variables Description Supplement
Comment Optional Not Available You can write a short description of this adapter.
The description will be reflected in the specifications.		  

Schema

Input Schema

None.

Output Schema

The number of columns depends on the [Schema definition] settings.

Reading Schema by Mapper

The schema will be read automatically.

Large Data Processing

Large Data Processing is supported.

Usage on PSP Script

Cannot be used on PSP Script.

Available Component variables

Component Variable Name Description Supplement
count The number of data read is stored.
  • The default value is null.
latest_date_covered The candidate of timestamp for [Period of last update :From] setting is stored for the upcoming call.
  • The default value is null.
  • The value will not be stored when using API 26.0 and before.
server_url The end point URL after Login is stored.
  • The default value is null.
session_id The session Id is stored.
  • The default value is null.
message_category In the case that an error occurs, the category of the message code corresponding to the error is stored.
  • The default value is null.
message_code In the case that an error occurs, the code of the message code corresponding to the error is stored.
  • The default value is null.
message_level In the case that an error occurs, the importance of the message code corresponding to the error is stored.
  • The default value is null.
operation_api_exception_code The ExceptionCode of the occured error, in a case of API error, is stored.
  • The default value is null.
  • For any error other than an API Error, the value is not stored.
  • The content to be stored may change according to the version of DataSpider Servista.
operation_error_message If an error occurs, the error message of the occured error is stored.
  • The default value is null.
  • The content to be stored may change according to the version of DataSpider Servista.
operation_error_trace When an error occurs, the trace information of the occured error is stored.
  • The default value is null.
  • The content to be stored may change according to the version of DataSpider Servista.

Create Schema

The data items to be read from and/or to be written into Salesforce can be specified in the creation of a schema.
By specifying only the items indispensable for processing, the data traffic to Salesforce can be reduced, generally improving the processing performance.



Number in the Image Name Description Supplement
(1) Selected Fields Select items for reading from and/or writing to Salesforce.  
(2) UnSelected Fields Select items not for reading from and/or writing to Salesforce.
  • You can filter fields by inputting or selecting values from the input above the column name.
    Text fields will be filtered as Starting With the input value.

Display Table Information

This display shows the available operations on Salesforce tables as well as field properties of the tables.
Also, the field information can be printed out.



Number in the Image Name Description Supplement
(1) Table Name Select the table whose structure to be shown.  
(2) Table Information Display the available operations on the selected table.  
(3) Length Display the Number of Digit of item  
(4) External ID Display whether or not the object item is set as an external ID.  
(5) Createble Display whether or not it can be set a value when adding data.  
(6) Updatable Display whether or not it can be set a value when updating data.  
(7) Nullable Display whether or not it can be set NULL when adding or updating data.  
(8) Default value Display whether or not Salesforce automatically set a default value when adding data.  
(9) Reference To Display the referring object name if the item is in reference relationship or master-servant relationship.  

Specification Limits

Main exceptions

Exception Name Reason Resolution
ResourceNotFoundException
Resource Definition is Not Found. Name:[]		 [Destination] is not specified. Specify [Destination].
ResourceNotFoundException
Resource Definition is Not Found. Name:[<Global Resource Name>]		 The resource definition selected in [Destination] cannot be found. Verify the global resource specified in [Destination]
java.net.UnknownHostException This exception occurs when the PROXY server specified in the global resource cannot be found. Verify the condition of the PROXY server. Or verify [Proxy Host] of the global resource specified in the [Destination].
  • API 23.0 or earlier
    org.apache.commons.httpclient.HttpConnection$ConnectionTimeoutException
  • API 26.0 or later
    java.net.SocketTimeoutException
    connect timed out
 A time-out has occurred while connecting to Salesforce. Verify the network condition and Salesforce server condition. Or check [Connection timeout(sec)] of the global resource specified in the [Destination].
  • API 23.0 or earlier
    org.apache.commons.httpclient.HttpRecoverableException
    java.net.SocketTimeoutException: Read timed out
  • API 26.0 or later
    java.net.SocketTimeoutException
    Read timed out
 A time-out has occurred while waiting for a responce from the server after connecting to Salesforce. Verify the network condition and Salesforce server condition. Or check [Timeout(sec)] of the global resource specified in the [Destination].
jp.co.headsol.salesforce.adapter.exception.SalesforceAdapterIllegalArgumentException Invalid value is set for the property of Salesforce adapter. Check the error message, and verify the settings.
com.sforce.soap.partner.fault.LoginFault Login to Salesforce has failed. Check the ExceptionCode or error message, and refer to the information about this type of error in Salesforce-related documents etc.
com.sforce.soap.partner.fault.UnexpectedErrorFault An unexpected error has occured while processing to Salesforce. Check the ExceptionCode or error message, and refer to the information about this type of error in Salesforce-related documents etc.

Notes