Write
You can use the Write activity to write data to a specified file in HDFS.
To append data to the same file or overwrite data in the same file in HDFS while running multiple processes simultaneously, you have to turn on the single thread by adding the following properties to the VM arguments in TIBCO Business Studio:
-Dcom.tibco.plugin.bigdata.write.singlethread=true
-Dbw.application.job.flowlimit.project_name.application=1
To access the VM arguments in TIBCO Business Studio, click Run Configurations dialog, and then click the Argument tab in the right panel of the dialog.
, expand in the left panel of theAlso, for deployment of the Write activity, you have to add the following properties to the Config.ini file:
com.tibco.plugin.bigdata.write.singlethread=true
bw.application.job.flowlimit.project_name.application=1
The Config.ini file is located in the TIBCO_HOME/bw/version_number/domains/domain_name/appnodes/appspace_name/appnode_name directory.
General
In the General tab, you can specify the activity name in the process, establish a connection to HDFS, and decide wether to append data to an existing file or overwrite data in an existing file.
The following table lists the configurations in the General tab of the Write activity:
Field | Module Property? | Description |
---|---|---|
Name | No | The name to be displayed as the label for the activity in the process. |
HDFSConnection | Yes | The HDFS Connection shared resource that is used to create a connection between the plug-in and HDFS. Click
to select an HDFS Connection shared resource.
If no matching HDFS Connection shared resources are found, click Create Shared Resource to create one. For more details, see Creating an HDFS Connection. |
Append | No | If you want to append the data to an existing file, you can select this check box. |
WriteType | No | The format of the data to be written. Select a format from the list: |
Overwrite | Yes | If you want to overwrite the existing data in the specified file, you can select this check box.
This check box is displayed only when you clear the Append check box. |
Input
In the Input tab, you can configure the append or overwrite operation that you select in the General tab. The input elements of the Write activity vary depending on the data format that you select in the General tab.
The following table lists all the possible input elements in the Input tab of the Write activity:
Input Item | Data Type | Description |
---|---|---|
fileName | String | The path of the file to be written in. |
fileContent | String | The content to be written in the file.
This element is displayed only when you select text from the WriteType list in the General tab. |
binaryData | Base64
Binary |
The binary data to be written in the file.
This element is displayed only when you select binary from the WriteType list in the General tab. |
sourceFile
Path |
String | The path of the file to be written.
This element is displayed only when you select file from the WriteType list in the General tab. |
inputStream
Object |
Object | The streaming object to be written.
This element is displayed only when you select StreamObject from the WriteType list in the General tab. |
overwrite | Boolean | You can specify whether you want to overwrite the existing data in the source file, 1 (true) or 0 (false). |
blockSize | Long | The block size of the file. The value in this field must be greater than 0. |
replication | Short | The number of replications of the file. The value in this field must be greater than 0. |
permission | Integer | The permission of the file. The value in this field must be in the range 0 - 777. |
buffersize | Integer | The size of the buffer that is used in transferring data. The value in this field must be greater than 0. |
timeout | Long | The amount of time, in milliseconds, to wait for this activity to complete.
By default, this activity does not time out if you do not specify a value. |
Output
In the Output tab, you can view the operation status of the Write activity.
The following table lists the output elements in the Output tab of the Write activity:
Fault
In the Fault tab, you can view the error code and error message of the Write activity. See Error Codes for more detailed explanation of errors.
The following table lists the error schema elements in the Fault tab of the Write activity:
Error Schema Element | Data Type | Description |
---|---|---|
msg | String | The error message description that is returned by the plug-in. |
msgCode | String | The error code that is returned by the plug-in. |
exception | String | The exception occurs when the plug-in has internal errors. |
message | String | The error message that is returned by the server. |
javaClassName | String | The name of the Java class where an error occurs. |