File Delimited Data Sources

This section describes the usage of DSL APIs for creating, altering or deleting a file-delimited data source.

Considerations

Listed below are some points to consider while working with File-Delimited Data Sources:

1. When the parameters "root" and "url" are both provided, an error message is displayed.
2. During an alter operation, the properties that were previously set during a create will not be altered unless a new value or NULL is specified.

Examples

Following are examples to create a file-delimited datasource:

Create a file-delimited datasource

 

curl -d "[\"CREATE DATASOURCE /shared/examples/excelds
            BASED ON ADAPTER 'File-Delimited'
            CONNECT USING LOCAL_ROOT_PATH '/Users/shared/csv' FILE_FILTERS '*.csv'
FORMAT USING CHARACTER_SET 'utf-8'
            SET ANNOTATION 'this is a file delimited datasource created using DSL api'
        \"]"
-u "admin:admin" -X POST "http://localhost:9400/rest/execute/v1/actions/dsl/invoke"
-H "Content-Type:application/json"

Create with native properties

 

//Error: root path specified in both LOCAL_ROOT_PATH as well as native properties syntax
curl -d "[\"CREATE DATASOURCE /shared/examples/fileds BASED ON ADAPTER 'File-Delimited' SET NATIVE PROPERTIES {\\\"root\\\":\\\"/Users/Shared\\\"} CONNECT USING LOCAL_ROOT_PATH '/Users/Shared' FILE_FILTERS '*.csv,*.txt' FORMAT USING CHARACTER_SET 'utf-8' SET ANNOTATION 'this is a file delimited datasource created using DSL api'\"]" -u "admin:admin" -X POST "http://localhost:9400/rest/execute/v1/actions/dsl/invoke" -H "Content-Type:application/json"
 
'root' already specified in DSL as 'local root path'
 
//Warning logged in server_log but datasource created. root unspecified using DSL syntax but specified in native properties
curl -d "[\"CREATE DATASOURCE /shared/examples/fileds BASED ON ADAPTER 'File-Delimited' SET NATIVE PROPERTIES {\\\"root\\\":\\\"/Users/Shared\\\"} CONNECT USING FILE_FILTERS '*.csv,*.txt' FORMAT USING CHARACTER_SET 'utf-8' SET ANNOTATION 'this is a file delimited datasource created using DSL api'\"]" -u "admin:admin" -X POST "http://localhost:9400/rest/execute/v1/actions/dsl/invoke" -H "Content-Type:application/json"
 
WARN 2019-05-07 10:08:46.637 -0700 CreateOrAlterResourceFilter - 'root' could be specified in DSL instead of native property using syntax like 'CONNECT USING local_root_path'