GetFragment Activity

The GetFragment activity provides a way to process a large stream of XML fragment-by-fragment, based on a configuration.

Warning: This activity must be used in a group. It outputs next fragment each time the group executes.

For more information on how the GetFragment activity fragments the large XML, refer to XML Split Elements.

The GetFragment activity also provides the following features:

  • Memory Optimization: When the GetFragment activity is used in a loop, it is ensured that only the fragment being processed is loaded into memory. The earlier ones are cleared from memory.
  • Smart recovery: Recovery can be done using the Checkpoint activity of TIBCO ActiveMatrix BusinessWorks. In case the engine crashes during processing in a group, on re-starting, the processing would start from the Checkpoint activity from the fragment where it stopped.
    Note: The recovery is not supported if GetFragment activity Input Source is selected as input stream.

    For more information on the Checkpoint activity, refer to TIBCO ActiveMatrix BusinessWorks Palette Reference.

  • Metadata Information: This activity outputs metadata information such as fragment number, number of records, and total number of fragments processed for every fragment generated. This metadata is useful to correlate the fragments with each other and with the parent large XML.

    This metadata information is not provided by the XMLSplitter activity.

  • In-line Fragment Processing: Processes each XML fragment in-line while the XML is being fragmented and still guarantees only one fragment in memory at any time.

    With the XMLSplitter activity, you need to wait for the activity to complete before you can apply your business logic.

General Tab

The General tab has the following fields.

Field Literal Value/ Module Property/ Process Property? Description
Name No The name to be displayed in the process definition
Description No A short description of the activity
Input XML Source
(Mandatory) No The format of the input XML:
  • file name
  • input stream
  • output stream
  • tcp stream

Dynamic Input Source (Optional)

No

This check box is enabled only when the Input XML Source is selected as file name or input stream.

With a Dynamic Input Source, the GetFragment activity can get fragments from a file or input stream while the file or input stream is being written to. That is, the file or input stream, is incomplete.

You can specify the following values:

  • Read Timeout for Dynamic Source (Msec): The time to wait, in milliseconds, for a dynamic stream. The default value is 10000.
  • Retry Count: The number of times the Read Timeout for Dynamic Source (Msec) should be retried before an error or exception is thrown. The default is 1.

The Read Timeout for Dynamic Source (Msec) and Retry Count must to be adjusted to ensure that the dynamic stream is handled correctly.

Timeout(msec) Yes

This field is enabled only when the Input XML Source is selected as tcp stream.

The time to wait, in milliseconds, for a TCP stream over a TCP connection.

Output Source
(Mandatory) No The format of the output:
  • output stream
  • xml
  • binary
Fragment Criteria
(Mandatory) No The criteria used for fragmenting:
  • size of fragment - split size in bytes
  • number of records
Fragment Criteria Value Yes The value of Fragment Criteria.

Note: You can also specify the fragment criteria on the Input tab (fragmentSize and noOfRecords). The fragment criteria specified on the Input tab overrides the Fragment Criteria specified on the General tab.

Fragment Element Name No The name of the XML element used for splitting the input XML document into fragments. If the elementName is specified on the Input tab, it takes preference over the Fragment Element Name, specified on the General tab.

If you want to consider the parent elements during fragmenting, specify the child element name.

For example: child_element

For more information on how the GetFragment activity fragments the large XML, refer to XML Split Elements.

Fragment Element Level No This field denotes whether the fragment elements are at the same level or at different levels. This field is available only when the Fragment Element Name has a relative path, without slashes.

You need to select Same level, if the fragment elements are all at the same hierarchical level. If they are at different levels, you need to select Different Level.

Note: If you open your project in a version earlier than ActiveMatrix BusinessWorks Plug-in for Large XML 6.1.3, you need to provide a relative path in Fragment Element Name to use the Fragment Element Level field.

Input Tab

The input for the activity is the following.

Input Item Datatype Description
fileName String The path and name of XML file.

This field is available only when file name is selected as the Input XML Source on the General tab.

inputStream ObjectReference The stream mapped from output of FileToStream activity.

This field is available only when input stream is selected as the Input XML Source on the General tab.

outputStream ObjectReference The Output stream ObjectReference mapped from the GetFragment activity.

It is available only when Output Stream is selected as the Input XML source on the General tab.

elementName String The name of the XML element used for splitting the input XML document into fragments. If the elementName is specified on the Input tab, it takes preference over the Fragment Element Name, specified on the General tab.

If you want to consider the parent elements during fragmenting, specify the child element name.

For example: child_element

For more information on how the GetFragment activity fragments the large XML, refer to XML Split Elements.
fragmentSize Integer Specifies the maximum size of the fragment in bytes of each fragment.

Available only if size of fragment is selected as the Fragment Criteria on the General tab.

Note: The fragment criteria value specified on the Input tab overrides the Fragment Criteria Value specified on the General tab.

noOfRecords Integer Specifies the maximum number of records in each fragment.

Available only if number of records is selected as the Fragment Criteria on the General tab.

Note: The Fragment Criteria Value specified on the Input tab overrides the Fragment Criteria Value specified on the General tab.

Output Tab

The output of the activity is the following.

Output Item Datatype Description
fragmentOutputStream Object Reference

The object reference of stream which can be used with StreamToFile, TransformXML, and XMLValidation activities.

Available if output stream is selected as the Output Source on the General tab.

XML String

Available when xml is selected in the Output Source field on the General tab. This item contains the fragmented XML string.

binaryContent Binary The fragment content in binary format.

Available when binary is selected as the Output Source on the General tab.

endofFile Boolean This field shows whether the GetFragment activity has finished processing the given XML file or XML stream. If the end of the file has reached, the value is true. Otherwise, the value is false.

This can be used to put a condition in the Loop Group activity to run the iteration till the condition is not true.

For example:

true()=$GetFragment/endOfFile or $GetFragment/endOfFile=true()

fragmentNumber Integer The sequential number of the currently generated fragment
noOfrecords Integer The number of records contained in the current fragment
totalProcessedFragments integer The total number of records processed

Fault Tab

The Fault tab lists the possible exceptions that can be thrown by this activity. See TIBCO ActiveMatrix BusinessWorks Error Codes for more information about the error codes and corrective actions to take.

Exception Thrown When...
GetFragmentException This is a Generic GetFragment activity exception. This exception provides errors related to fragment processing.
FileNotFoundException Cannot find the file provided in the fileName parameter.
FragmentCriteriaException This exception provides errors related to fragment criteria.

The excessSplitElement in FragmentCriteriaException holds the excess split element in binary form. You can handle these errors to process the excess split element separately and still continue splitting the large XML file.