Delete File/Folder

Operation Name

Delete File/Folder

Function overview

Delete the folder/files on Amazon S3.

Data Model

Data model of this component is XML type.

Properties

For information about using variables, refer to "variables".

Basic settings

Item name	Required / Optional	Use of Variables	Description	Remarks
Name	Required	Not available	Enter the name on the script canvas.

Required settings

Item name	Required / Optional	Use of Variables	Description	Remarks
Destination	Required	Not available	Select Global Resources. [Add]: Adds new global resource. [Edit list]: Global resource settings can be edited with "Edit Resource list".
Bucket name	Required	Available	Specify Bucket.	If specifying Bucket which large number of files exist in [Bucket name], it may takes time to update list of [Folder path]. For avoidance method, refer to "Notes" section.
Delete bucket	Optional	Not available	Select whether to delete the Bucket. [Checked]: Delete bucket. [Not Checked]: (default) Delete file/ folder.
Folder path	Required	Available	Enter Amazon S3 folder path.	The Amazon S3 folder path specified needs to be absolute path. Constrained characters of DataSpider File System cannot be used except for path separator "/". If you check [Delete bucket], then it is disabled.
Delete folder	Optional	Not available	Select whether to delete the folder specified in [Folder path]. [Checked]: Delete the folder specified in [Folder path] and all files/folders in the folder. [Not Checked]: (default) Do not delete the folder specified in [Folder path]. Behavior against a file/folder in the folder depends on [File/folder name] and [Recursive processing] settings.	If you check [Delete bucket], then it will be disabled.
File/folder name	Required	Available	Specify a file/folder in the folder specified in [Folder path] Delete the specified file or files whose name matches the specified wildcard. Behavior depends on [Recursive processing] setting when a folder is specified or a name of a folder matches the specified wildcard.	If you check [Delete bucket] or [Delete folder], then it will be disabled. Wildcards("*" and "?") are available. "/" Can not be included.
Recursive processing	Optional	Not available	For a specified folder, select whether to delete the file/folder recursively or not. [Checked]: If the specified folder in [File/folder name] contains folders or the specified wildcard matches folders, then delete the file/folder recursively. [Not Checked]: (default) If the specified folder in [File/folder name] contains files/folders, then delete the files. If the specified wildcard matches folders, then delete nothing.	If you check [Delete bucket] or [Delete folder], then it will be disabled.

Data processing method

Item name	Required / Optional	Use of Variables	Description	Remarks
Mass data processing	Required	Not available	Select a data processing method. [Use script settings]: (default) Apply mass data processing settings of script property to adapter. [Disable]: Mass data processing is not performed. [Enable]: Mass data processing is performed.

Comment

Item name	Required / Optional	Use of Variables	Description	Remarks
Comment	Optional	Not available	You can write a short description of this adapter. The description will be reflected in the specifications.

Schema

Input Schema

None.

Output Schema

When deleting files

<?xml version="1.0"?>
<server>
  <bucket name="" status="" >
    <file name="" remotepath="" status="" type="" />
  </bucket>
</server>

Element Name	Attribute Name	Description	Remarks
server	-
bucket	-	Appear in each Bucket which is gotten.
	name	Output the name of the retrieved bucket.
	status	Output the status of the retrieved bucket. Exist: Status that the Bucket exists. ErrorBucketNotFound: The status that specified Bucket does not exist in Amazon S3. The specified [Bucket name] may be incorrect. Error: The status that because error has occurred, Bucket could not be obtained.
file	-	Appear in each deleted file/folder.
	name	Output the name of the deleted file/folder.	If status is "ErrorRemoteFolderNotFound" then the value is blank. If [Delete folder] is [Checked] then the value is blank.
	remotepath	Output file path on Amazon S3 of the deleted file/folder.
	status	Output the status of the deleted file/folder. Deleted: Status that the file/folder is successfully deleted. ErrorRemoteFolderNotFound: The status that the specified [Folder path] is not on the Amazon S3, or file is specified. ErrorRemoteFileNotFound: The status that on Amazon S3, the specified file/folder does not exist in the specified [Folder path]. Error: The status that because an error occurred file/ folder could not be deleted.
	type	Output the classification of deleted file/folder. File: File Folder: Folder

When deleting Bucket

<?xml version="1.0"?>
<server>
  <bucket name="" status="" />
</server>

Element Name	Attribute Name	Description
server	-
bucket	-	Appear in each Bucket which is gotten.
	name	Output the name of the retrieved bucket.
	status	Output the status of the retrieved bucket. Deleted: Status that Bucket is successfully deleted. ErrorBucketNotFound: The status that specified Bucket does not exist in Amazon S3. The specified [Bucket name] may be incorrect. Error: Could not delete the Bucket due to error.

Loading schema in Mapper

Schema is loaded automatically.

See "Edit Schema" for details with regards to defining a schema.

Mass data processing

Mass data processing is supported.

Transaction

Transaction is not supported.

PSP Usage

PSP is not supported.

Available component variables

Component variable name	Description	Remarks
count	Return the number which is the total number of the deleted folder number and file number.	The value defaults to null. The sum of "folder_count" and "file_count".
folder_count	Returns the number of deleted folder.	The value defaults to null.
file_count	Returns the number of deleted file.	The value defaults to null.
message_category	Stroes the category to which corresponding message code belongs to, when an error occurs.	The value defaults to null.
message_code	Stores its corresponding message code of occured error.	The value defaults to null.
message_level	Stores the severity of the corresponding message code of the occured error.	The value defaults to null.
error_type	Stores the type of the occured error.	The value defaults to null. Error is represented in the format depicted below. Example:java.io.FileNotFoundException The message may vary depending on the DataSpider Servista version.
error_message	Stores the error message for the occured error.	The value defaults to null. The message may vary depending on the DataSpider Servista version.
error_trace	Stores stack trace of the occurred error.	The value defaults to null. The message may vary depending on the DataSpider Servista version or the client application used.

Specification limitations

None.

Main exceptions

Exception name	Causes	Solution
ResourceNotFoundException Resource definition could not be found.Name: []	[Destination] is not specified.	Please specify [Destination].
ResourceNotFoundException Resource definition could not be found.Name: [<name of Global Resources>]	Resource definition selected in [Destination] is not found.	Please check the global resource specified in [Destination].
InvalidPropertyConfigurationException <Property Name> is not specified.	[<Property Name>] is not specified.	Please specify [<Property Name>].
InvalidPropertyConfigurationException "/" is included in file/folder name.	"/" is included in [File/folder name].	In [File/folder name], please specify the file/folder name which do not include "/".
java.lang.UnsupportedOperationException Root folder can not be deleted.	Attempted to delete the root folder.	If [Delete folder] is [Checked], please specify a non-root folder in [Folder path].
Status Code: 403, AWS Request ID: XXXXXXXXXXXXXXXX, AWS Error Code: InvalidAccessKeyId, AWS Error Message: The AWS Access Key Id you provided does not exist in our records., S3 Extended Request ID: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX	[Access Key ID] of global resources specified in [Destination] is not correct.	Check the settings of the global resource specified in [Destination].
Status Code: 403, AWS Request ID: XXXXXXXXXXXXXXXX, AWS Error Code: SignatureDoesNotMatch, AWS Error Message: The request signature we calculated does not match the signature you provided. Check your key and signing method., S3 Extended Request ID: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX	[Secret Access Key] of the global resources specified in [Destination] is incorrect.	Check the settings of the global resource specified in [Destination].
com.amazonaws.SdkClientException Unable to execute HTTP request: <endpoint>	[Endpoint] of the global resource specified in [Destination] is incorrect.	Check the settings of the global resource specified in [Destination].

Notes

Use HTTPS for communicating between Amazon S3 and this adapter.

If the setting of Region in which Bucket exist and endpoint is different, because it takes time to propagate the status of Bucket and file/folder, you may not get the status of latest Bucket and file/folder and fail to execute the operation.

If specifying Bucket which large number of files exist in [Bucket name], it may takes time to update list of [Folder path].
in these cases, please set the following property items and do not show items in the list.

Key	Location	Description	Remarks
itemmaxsize	$DATASPIDER_HOME/server/plugin/data_processing/modules/amazon_s3_adapter/META-INF/ adapter.properties	The setting procedure is as follows. Stop DataSpiderServer. Open adapter.properties file, add "itemmaxsize=0" and save. Start DataSpiderServer.	If you set this property, please input the path of the folder to [Folder path] directly.

When running, if an error unrelated to the connection (error which is not described in "main exception") occurs, set [Error] to status attribute of file element of the file / folder in which error occurs, then output as result data and continue the further process. When running, if an error unrelated to the connection (error which is not described in "main exception") occurs, set [Error] to status attribute of file element of the file / folder in which error occurs, then output as result data and continue the further process.