Deleting objects from a bucket after downloading

This section explains how to get transfer-related information using the get command of transfer-related information and then use that information in the CLI.

In this example, the sending-side host deletes the downloaded file (object) from the storage with the delete command in the CLI after transferring the file.

Get the following transfer information using the get command of transfer-related information, and then delete the object from the bucket using the delete command in the CLI:

  • "Bucket name" used for the transfer

  • "Object name " used for the transfer

Using the same procedure, you can get other transfer-related information to use for various jobs.

Although the example is for Amazon S3, you can register for other object storage in the same way using the command for the relevant object storage.

 

This example assumes that the following Send Storage Management Information for Amazon S3 has been registered:

Table D.18 Example of setting values for the Send Storage Management Information

Field Name in Send Storage Management Information

Setting Value

Register or Not

Send Storage Management Information ID

bucket01

✓

Bucket Name

Not specified

-

Storage Timeout

Not specified

-

Maximum number of parallels per transfer

Not specified

-

Part Size

Not specified

-

Region (*1)

us-east-1

✓

Endpoint URL (*1)

Not specified

-

Storage Authentication Information ID

Not specified

-

✓

:

Register after changing the setting value

-

:

Do not register the setting value

*1

:

This field can be set only when you use Amazon S3.

 

By registering the ID of a job that deletes a file (object) for Successful Job ID in the Send Management Information used for transferring, you can delete a file (object) downloaded from the storage after it has been transferred.

 

Perform the following procedure to register the management information.

 

  1. Create an "executable file" to be registered in the Job Information.

    The executable file to be registered in the Job Information executes the following processing:

    1. Use the get command of transfer-related information (utls3inforef) and the environment variable "FILENM" of the startup job to get the following transfer information:

      • Bucket Name

      • Object Name

    2. To delete the object, specify the "bucket name" and "object name" retrieved in step a in the CLI command.

     

    An example of the executable file is shown below.

    Windows: Example of a batch file

    @echo off
set FILENAME=%$FILENM%
for /f "usebackq delims=" %%V in (`"C:\HULFT Family\hulft\bin\plugins_utls\utls3inforef.exe" -t snd -f %FILENAME% -k bucket`) do set BUCKET=%%V
for /f "usebackq delims=" %%V in (`"C:\HULFT Family\hulft\bin\plugins_utls\utls3inforef.exe" -t snd -f %FILENAME% -k object`) do set OBJECT=%%V
C:\Program Files\Amazon\AWSCLI\bin\aws.exe s3api delete-object --bucket %BUCKET% --key %OBJECT%

    Linux: Example of a shell script

    #!/bin/sh
export PATH="$PATH":"$HULEXEP"/plugins_utls
BUCKET="$(utls3inforef -t snd -f "$FILENM" -k bucket)"
OBJECT="$(utls3inforef -t snd -f "$FILENM" -k object)"
aws s3api delete-object --bucket "$BUCKET" --key "$OBJECT"

    For details on the get command of transfer-related information for Amazon S3, refer to Get command of transfer-related information (Amazon S3).

     

  2. Register the "executable file" created in step 1 in the Job Information.

    In this example, register as follows:

    Table D.19 Example of setting values for the Job Information

    Field Name

    Setting Value

    Job ID

    OBJDELETE

    Startup Job

    The "executable file" created in step 1

    For details on registering the Job Information, refer to the following:

    HULFT8 Windows Operation Manual: Job Information

    HULFT8 UNIX/Linux Operation Manual: Job Information

     

  3. Register the job id registered in step 2 to Successful Job ID in the Send Management Information.

    In this example, register as follows:

    Table D.20 Example of setting values for the Send Management Information

    Field Name

    Setting Value

    File ID

    OBJDELETE

    Send File Name

    The "executable file" created in step 1

    Successful Job ID

    OBJDELETE

    For details on registering the Send Management Information, refer to the following:

    HULFT8 Windows Operation Manual: Send Management Information

    HULFT8 UNIX/Linux Operation Manual: Send Management Information

 

The setting is complete to delete a file (object) by the job executed after the file transfers successfully.

If you transfer file ID "DOWNLOAD01", the Send file (object) is deleted from the bucket by the job processing after transferring.

In the above example, the environment variables are as follows when transferring:

Table D.21 Values of the environment variables

Environment Variable

Setting Value

FILENM (FILENAME)

s3://bucket01/object01

BUCKET

bucket01

OBJECT

object01

 

If you want to get other field values related to transfers, refer to the following for the keys to get fields:

Get command of transfer-related information (Amazon S3)

 

For details on the command for other object storage, refer to the following: