Read Filesystem

Operation name

Read Filesystem

Function overview

Reads the data structure of files and directories in the directory.

Data model

The data model of this component is table model type.

Properties

= Remarks =

For details on use of variables, refer to Variables.

Basic settings

Item name

Required/Optional

Use of variables

Description

Remarks

Name

Required

Not available

Enter a name that is used on the script canvas.

 

Required settings

Item name

Required/Optional

Use of variables

Description

Remarks

Directory

Required

Available

Enter the directory to read.

The Browse button launches a directory chooser that allows you to select a directory.

Note

Data mode

Required

Not available

Select the data mode to extract from Directory.

Data

(Default)

Get structural information (type, path, name) for the files or directories in the specified directory and the file contents. The contents of the file are gotten as binary format.

Structure

Get structural information (type, path, name) for the files or directories in the specified directory.

 

Recursive processing

Required

Not available

Select whether to recursively extract.

false

(Default)

Don't get recursively.

true

Get recursively.

 

Data processing method

Item name

Required/Optional

Use of variables

Description

Remarks

Data processing method

Required

Not available

Select a data processing method.

Apply script settings

(Default)

The setting for mass data processing in the script properties is applied to the connector.

Do not perform mass data processing

Mass data processing isn't performed.

Perform mass data processing

Mass data processing is performed.

 

Comment

Item name

Required/Optional

Use of variables

Description

Remarks

Comment

Optional

Not available

You can write a short description of this connector.

 

Schemas

Input schema

None.

Output schema

<?xml version="1.0"?>
<table>
  <row>
    <column>filetype</column>
    <column>path</column>
    <column>name</column>
    <column>data</column>
  </row>
  <row>
    :
  </row>
</table>

Column name

Description

Remarks

row

-

One row is output per a file or a directory.

 

filetype

The type of the file or the directory retrieved is output.

file

File

directory

Directory

 

path

The absolute path of the file or directory retrieved is output.

 

name

The name of the file or the directory retrieved is output.

 

data

The contents of the retrieved file are output as binary format.

  • Output when Data mode is set to Data.

  • Output only for a file.

Loading schema in Mapper

The schema is loaded automatically.

= Remarks =

For details, refer to Edit Schema.

Mass data processing

Mass data processing is supported.

 

Large volume data is processed line by line. You can get a lot of files and directories.

 

Note

If the retrieved contents of the file are large, OutOfMemoryError might occur.

Parallel Stream Processing

PSP isn't supported.

Available component variables

Variable name

Description

Remarks

count

The number of read files/directories is stored.

  • The default value is null.

message_category

When an error occurs, the category of the message code corresponding to the error is stored.

  • The default value is null.

message_code

When an error occurs, the code of the message code corresponding to the error is stored.

  • The default value is null.

message_level

When an error occurs, the severity of the message code corresponding to the error is stored.

  • The default value is null.

error_type

When an error occurs, the error type is stored.

  • The default value is null.

  • The format of the error type is as follows.

    Example: java.io.FileNotFoundException

error_message

When an error occurs, the error message is stored.

  • The default value is null.

error_trace

When an error occurs, the trace information for the error is stored.

  • The default value is null.

Message codes, exception messages, and limitations

Connector

Message code

Exception message

Limitations

Messages and limitations of the Filesystem connector

check

check

None