File Delimited Data Sources
This section describes the usage of DSL APIs for creating, altering or deleting a file-delimited data source.
Considerations
Listed below are some points to consider while working with File-Delimited Data Sources:
1. When the parameters "root" and "url" are both provided, an error message is displayed.
2. During an alter operation, the properties that were previously set during a create will not be altered unless a new value or NULL is specified.
Examples
Following are examples to create a file-delimited datasource:
Create a file-delimited datasource
curl -d "[\"CREATE DATASOURCE /shared/examples/excelds
BASED ON ADAPTER 'File-Delimited'
CONNECT USING LOCAL_ROOT_PATH '/Users/shared/csv' FILE_FILTERS '*.csv'
FORMAT USING CHARACTER_SET 'utf-8'
SET ANNOTATION 'this is a file delimited datasource created using DSL api'
\"]"
-u "admin:admin" -X POST "http://localhost:9400/rest/execute/v1/actions/dsl/invoke"
-H "Content-Type:application/json"
Create with native properties
//Error: root path specified in both LOCAL_ROOT_PATH as well as native properties syntax
curl -d "[\"CREATE DATASOURCE /shared/examples/fileds BASED ON ADAPTER 'File-Delimited' SET NATIVE PROPERTIES {\\\"root\\\":\\\"/Users/Shared\\\"} CONNECT USING LOCAL_ROOT_PATH '/Users/Shared' FILE_FILTERS '*.csv,*.txt' FORMAT USING CHARACTER_SET 'utf-8' SET ANNOTATION 'this is a file delimited datasource created using DSL api'\"]" -u "admin:admin" -X POST "http://localhost:9400/rest/execute/v1/actions/dsl/invoke" -H "Content-Type:application/json"
'root' already specified in DSL as 'local root path'
//Warning logged in server_log but datasource created. root unspecified using DSL syntax but specified in native properties
curl -d "[\"CREATE DATASOURCE /shared/examples/fileds BASED ON ADAPTER 'File-Delimited' SET NATIVE PROPERTIES {\\\"root\\\":\\\"/Users/Shared\\\"} CONNECT USING FILE_FILTERS '*.csv,*.txt' FORMAT USING CHARACTER_SET 'utf-8' SET ANNOTATION 'this is a file delimited datasource created using DSL api'\"]" -u "admin:admin" -X POST "http://localhost:9400/rest/execute/v1/actions/dsl/invoke" -H "Content-Type:application/json"
WARN 2019-05-07 10:08:46.637 -0700 CreateOrAlterResourceFilter - 'root' could be specified in DSL instead of native property using syntax like 'CONNECT USING local_root_path'