Registration of resources and crawling
Addition of resources
You can add a target for data crawling.
By registering terms and tags in advance, you can link these terms and tags when you add resources.
1. Click [Resource] from the header menu.
2. Click [Add].
3. Enter the required fields, and click [Register].
-
For the endpoint to register Amazon S3, refer to the following URL:
https://docs.aws.amazon.com/ja_jp/general/latest/gr/rande.html#s3_region
-
For the connection string to register Azure Blob Storage, refer to the following URL:
Editing of resources
1. Click [Resource] from the header menu.
2. Select the resources you want to edit, and click [Edit].
3. When the screen to edit resources is displayed, enter the required information and click [Register].
Deletion of resources
1. Click [Resource] from the header menu.
2. Select the resources you want to delete, and click [Delete].
3. When the confirmation screen is displayed, click [Delete].
Crawled information
The crawled information is as follows:
For DataSpider
- Resources list (global resources)
- Project names
- Script names
- Components in scripts
- Script comments
For PosgreSQL, Oracle, SQL Server, MySQL, and Db2
-
Schema information (other than MySQL)
-
Schema names
-
-
Table information
-
Table names
-
Comments for tables (other than SQL Server)
-
-
Column information
-
Column names
-
Data types
-
Column capacity
-
Comments for columns (other than SQL Server)
-
For Amazon S3
-
Bucket information
-
Bucket names
-
-
Folder and file information
-
File names and folder names
-
Paths
-
Sizes
-
For Azure Blob Storage
-
Container information
-
Container names
-
-
Folder and file information
-
File names and folder names
-
URLs
-
Sizes
-
Method for crawling manually
To crawl at the timing of your choice, perform the following procedure:
1. Click [Resource] from the header menu.
2. Select the resources you want to crawl, and click [Crawling].
Depending on the amount of data that is crawled, it may take some time to obtain the data.
3. Reload the website.
4. Confirm that "Success" is displayed in the status column on the resources list screen.
You can also check the crawling results by checking the log files. Refer to the setup manual for details on log files.