Copyright © Cloud Software Group, Inc. All Rights Reserved

Parse Data

Activity

The Parse Data activity takes a text string or input from a file and processes it, turning it into a schema tree based on the specified Data Format shared configuration.

You can use any mechanism to obtain or create a text string for processing. For example, you can use the Read File activity to obtain text from a file, or you can retrieve a text field from an adapter message. You can also specify a text file to read using this activity.

You might use this activity in a number of situations. For example, you may have a file that consists of multiple lines of comma-separated values (as in data obtained from a spreadsheet). You may also want to insert that data into a database table. To do this, read and parse the file into a data schema with the Parse Data activity. Then use a JDBC Update activity to insert the data schema into a database table.

Configuration

The Configuration tab has the following fields.

Field	Global Var?	Description
Name	No	The name to appear as the label for the activity in the process definition.
Description	No	Short description of the activity.
Data Format	No	The Data Format shared configuration to use when parsing the text input. See Data Format for more information.
Input Type	No	Specifies the type of input for this activity. Input can either be a text string or a file. If the input is a text string, supply the string to the textString input item. If the input is a file, supply the file name and location to the fileName input item.
Encoding	Yes	The encoding of the input file. This field is only available when "File" is chosen in the Input Type field. Any valid Java encoding name can be used.
Skip Blank Lines	No	Skips any empty records when parsing the text input. When this field is unchecked, parsing stops at the first blank line encountered in the input.
Manually Specify Start Record	No	Allows you to specify the record in the input where you want to start parsing. This is useful if you have a large number of records and you want to read the input in parts (to minimize memory usage). Checking this check box causes the startRecord input item to appear. See Parsing a Large Number of Records for more information on how to read the input stream in parts.
Strict Validation	No	Validates every input line for the specified number of fields for the "Fixed Format" text. For example, if the format states that there are three fields per line and this field is checked, all lines in the input must contain three fields.
Continue on Error	No	Continues parsing the next record in the input after encountering an error, if any. If an error occurs, the error information is separated from the output of the successfully parsed records and is provided in the output schema of the activity. See Output for more information on how the error information for the records that failed parsing is provided. Error information is provided only for records specified as Required. If the data item is specified as Optional, the process completes parsing the input successfully. Irrespective of whether this field is checked or not, the Parse Data activity quits when any data validation errors occur.

Input

See TIBCO ActiveMatrix BusinessWorks Process Design for more information about mapping and transforming input data.

The input for the activity is the following.

Input Item	Datatype	Description
text	string	The text string to parse. This input item is only available when "String" is specified in the Input Type field on the Configuration tab.
fileName	string	The location and name of the file to read. The file’s contents is used as the input text string for this activity. This input item is only available when "File" is specified in the Input Type field on the Configuration tab.
startRecord	number	The line number of the input stream where you want to begin parsing. All lines before the specified line are ignored. This input item is only available if the Manually Specify Start Record field on the Configuration tab is checked. The input stream begins with line number 1 (one). This is useful if you want to read the input stream in parts to minimize memory usage. See Parsing a Large Number of Records for more information.
noOfRecords	number	The number of records to read from the input stream. Specify "-1" if you want to read all records in the input stream. This is useful if you want to read the input stream in parts to minimize memory usage. See Parsing a Large Number of Records for more information.
SkipHeaderCharacters	integer	The number of characters to skip when parsing. This allows you to skip over any file headers or other unwanted information.

Output

The output for the activity is the following.

Output Item	Datatype	Description
Rows	complex	This output item contains the list of parsed lines from the input. This is useful to determine the number of records parsed by this activity. The schema specified by the Data Format resource is contained in this output item.
schema	complex	The schema containing the data from the parsed input text. This output item contains zero or more parsed records.
ErrorRows		This output item is available when Continue on Error is checked, and error(s) while parsing the records in the input. <<raw input data is put in the error string>> This field contains the list of error lines for the records from the input that failed parsing. See the example below explaining successful parsing of records specified as Optional and Mandatory. During parsing, if any of the Required data items are missing, the records are displayed in the Error Row. If there are missing Optional items, the records are displayed in the Output Row.
EOF	boolean	true if no more records are available for parsing. false if there are more records available. This output item is useful for checking if there are no more records in the input stream when you are reading the input in parts to preserve memory. See Parsing a Large Number of Records for more information.

Example

The example contains two input records containing five data items, ID, FName, LName, Address and Age. All the items are declared mandatory.

,FN1,LN1,Pune,99

2,FN2,LN2,Pune,33

Shown below is the output of the two records, after parsing them.

As the field "ID" is missing from the first record, the entire record appears in the ErrorRows element.

 

<?xml version = "1.0" encoding = "UTF-8"?>

<Output>

      <Rows>

           <root>

                <ID>2</ID>

                <FName>FN2</FName>

                <LName>LN2</LName>

                <Address>Pune</Address>

                <Age>33</Age>

           </root>

       </Rows>

      <ErrorRows>

           <ErrorString>,FN1,LN1,Pune,99</ErrorString>

      </ErrorRows>

      <EOF>true</EOF>

</Output>

 

When the ID field is declared Optional, and is missing, it is not an error. The records appear in the output string :

 

<?xml version = "1.0" encoding = "UTF-8"?>

<Output>

      <Rows>

          <root>

               <FName>FN1</FName>

               <LName>LN1</LName>

               <Address>Pune</Address>

               <Age>99</Age>

          </root>

          <root>

               <ID>2</ID>

               <FName>FN2</FName>

               <LName>LN2</LName>

               <Address>Pune</Address>

               <Age>33</Age>

         </root>

   </Rows>

   <ErrorRows/>

   <EOF>true</EOF>

</Output>

Error Output

The Error Output tab lists the possible exceptions that can be thrown by this activity. See TIBCO ActiveMatrix BusinessWorks Error Codes for more information about error codes and corrective action to take.

Exception	Thrown When...
FileNotFoundException	The specified input file cannot be located.
BadDataFormatException	The input format is not valid.

Parsing a Large Number of Records

The input for this activity is placed in a process variable and takes up memory as it is being processed. If you are reading a large number of records from a file, the process may consume significant machine resources. To avoid using too much memory, you may want to read the input in parts, parsing and processing a small set of records before moving on to the next set.

To process a large number of records, follow this procedure:

1.	Create a Parse Data activity.

2.	Specify the fields on the Configuration tab. Select the check box for the Manually Specify Start Record field.

3.	Select the Parse Data activity and click the group icon on the tool bar to create a group containing the Parse Data activity. See TIBCO ActiveMatrix BusinessWorks Process Design for more information about working with groups.

4.	Specify "Repeat Until True Loop" as the Group action, and specify an index name (for example, "i").

5.	The loop should exit when the EOF output item for the Parse Data activity is set to true. For example, the condition for the loop could be set to the following: string($ParseData/Output/done) = string(true())

6.	Set the noOfRecords input item for the Parse Data activity to the number of records you want to process for each execution of the loop.

7.	If you do not check the Manually Specify Start Record field on the Configuration tab of the Parse Data activity, the loop processes the specified noOfRecords with each iteration until there are no more input records to parse.

You can optionally check the Manually Specify Start Record field to specify the startRecord on the Input tab. If you do this, you must create an XPath expression to properly specify the starting record to read with each iteration of the loop. For example, the count of records in the input starts at zero, so the startRecord input item could be set to the current value of the loop index minus one. For example, $i - 1.

The procedure above is a general guideline for creating a loop group for parsing a large set of input records in parts. You may want to modify the procedure to include additional processing of the records, or you may want to change the XPath expressions to suit your business process.

Copyright © Cloud Software Group, Inc. All Rights Reserved