GetFragment Activity

The GetFragment activity provides a way to process a large stream of XML fragment-by-fragment, based on a configuration.

Warning: This activity must be used in a group. It outputs next fragment each time the group executes.

For more information on how the GetFragment activity fragments the large XML, refer to XML Split Elements.

The GetFragment activity also provides the following features:

  • Memory Optimization: When the GetFragment activity is used in a loop, it is ensured that only the fragment being processed is loaded into memory. The earlier ones are cleared from memory.
  • Smart recovery: Recovery can be done using the Checkpoint activity of TIBCO ActiveMatrix BusinessWorks. In case the engine crashes during processing in a group, on re-starting, the processing would start from the Checkpoint activity from the fragment where it stopped.
    Note: The recovery is not supported if GetFragment activity Input Source is selected as input stream.

    For more information on the Checkpoint activity, refer to TIBCO ActiveMatrix BusinessWorks Palette Reference.

  • Metadata Information: This activity outputs metadata information such as fragment number, number of records, and total number of fragments processed for every fragment generated. This metadata is useful to corelate the fragments with each other and with the parent large XML.

    This metadata information is not provided by the XMLSplitter activity.

  • In-line Fragment Processing: Processes each XML fragment in-line while the XML is being fragmented and still guarantees only one fragment in memory at any time.

    With the XMLSplitter activity, you need to wait for the activity to complete before you can apply your business logic.

General Tab

The General tab has the following fields.

Field Literal Value/ Module Property/ Process Property? Description
Name No The name to be displayed in the process definition
Description No A short description of the activity
Input XML Source
(Mandatory) No The format of the input XML:
  • file name
  • input stream
  • output stream
  • tcp stream

Dynamic Input Source (Optional)

No

This check box is enabled only when the Input XML Source is selected as file name or input stream.

With a Dynamic Input Source, the GetFragment activity can get fragments from a file or input stream while the file or input stream is being written to. That is, the file or input stream, is incomplete.

You can specify the following values:

  • Read Timeout for Dynamic Source (Msec): The time to wait, in milliseconds, for a dynamic stream. The default value is 10000.
  • Retry Count: The number of times the Read Timeout for Dynamic Source (Msec) should be retried before an error or exception is thrown. The default is 1.

The Read Timeout for Dynamic Source (Msec) and Retry Count must to be adjusted to ensure that the dynamic stream is handled correctly.

Timeout(msec) Yes

This field is enabled only when the Input XML Source is selected as tcp stream.

The time to wait, in milliseconds, for a TCP stream over a TCP connection.

Output Source
(Mandatory) No The format of the output:
  • output stream
  • xml
  • binary
Fragment Criteria
(Mandatory) No The criteria used for fragmenting:
  • size of fragment - split size in bytes
  • number of records
Fragment Criteria Value Yes The value of Fragment Criteria.

Note: You can also specify the fragment criteria in the Input tab (fragmentSize and noOfRecords). The fragment criteria specified in the Input tab overrides the Fragment Criteria specified in the General tab.

Input Tab

The input for the activity is the following.

Input Item Datatype Description
fileName String The path and name of XML file.

This field is available only when file name is selected as the Input XML Source on the General tab.

inputStream ObjectReference The stream mapped from output of FileToStream activity.

This field is available only when input stream is selected as the Input XML Source on the General tab.

outputStream ObjectReference The Output stream ObjectReference mapped from the GetFragment activity.

It is available only when Output Stream is selected as the Input XML source on the General tab.

elementName String The name of the XML element to be used for splitting the input XML document into fragments.

If you want to consider parent elements during fragmenting, specify the split element name as an expression. For example:

/root/parent/split element

NOTE: The path must begin with a forward slash (/).

For more information on how the GetFragment activity fragments the large XML, refer to XML Split Elements.
fragmentSize Integer Specifies the maximum size of the fragment in bytes of each fragment.

Available only if size of fragment is selected as the Fragment Criteria on the General tab.

Note: The fragment criteria value specified in the Input tab overrides the Fragment Criteria Value specified in the General tab.

noOfRecords Integer Specifies the maximum number of records in each fragment.

Available only if number of records is selected as the Fragment Criteria on the General tab.

Note: The Fragment Criteria Value specified in the Input tab overrides the Fragment Criteria Value specified in the General tab.

Output Tab

The output of the activity is the following.

Output Item Datatype Description
fragmentOutputStream Object Reference

The object reference of stream which can be used with StreamToFile, TransformXML, and XMLValidation activities.

Available if output stream is selected as the Output Source on the General tab.

XML String

Available when xml is selected in the Output Source field on the General tab. This item contains the fragmented XML string.

binaryContent Binary The fragment content in binary format.

Available when binary is selected as the Output Source on the General tab.

endofFile Boolean This field shows whether the GetFragment activity has finished processing the given XML file or XML stream. If the end of the file has reached, the value is true. Otherwise, the value is false.

This can be used to put a condition in the Loop Group activity to run the iteration till the condition is not true.

For example:

true()=$GetFragment/endOfFile or $GetFragment/endOfFile=true()

fragmentNumber Integer The sequential number of the currently generated fragment
noOfrecords Integer The number of records contained in the current fragment
totalProcessedFragments integer The total number of records processed

Fault Tab

The Fault tab lists the possible exceptions that can be thrown by this activity. See TIBCO ActiveMatrix BusinessWorks Error Codes for more information about the error codes and corrective actions to take.

Exception Thrown When...
GetFragmentException This is a Generic GetFragment activity exception. This exception provides errors related to fragment processing.
FileNotFoundException Cannot find the file provided in the fileName parameter.
FragmentCriteriaException This exception provides errors related to fragment criteria.

The excessSplitElement in FragmentCriteriaException holds the excess split element in binary form. You can handle these errors to process the excess split element separately and still continue splitting the large XML file.

Sample

The TIBCO_HOME/bw/palettes/lx/<version>/samples folder contains samples and a readme file that explains how to import and use the sample.

Contents
Search Results