Streams and Streams Modules
This section provides a very brief overview of some of the features of Streams.
Streams is a framework that supports modular processing of data. The unit of data in a streams application is a message, a user-defined type that inherits from a base message in Streams message type.
A stream is comprised of modules in Streams modules connected by links. A module performs some business task. Modules are implemented using action language and are derived from a base module type. Modules have zero, one or many output ports in Streams output ports and exactly one input port. Each port and module supports a particular message type and all its subtypes.
A link connects an output port of one module to the input port of another. Any number of links can be connected to an input port. Ports on each side of a link must support matching message types.
Messages are processed by a module and then passed to downstream modules by putting the message on an output port. A module’s response to a put request is implemented in the virtual operation onPut. A module calls put without any knowledge of the downstream modules, and typically implements onPut without any knowledge of upstream modules.
The first module in a stream is a provca::ClientAdapter a module to collect output from external sources and inject this output into the stream.
Midstream modules parse, process, and format the messages and can implement flow control, data aggregation, or many other tasks.
All streams end in a terminator in a Streams terminator module. This is the only module that deletes messages. Any number of output ports from other modules can be plumbed to a single instance of the terminator module.
The Streams framework includes numerous pre-built modules and module base classes. In a typical streams application, you configure pre-built modules for common tasks like file parsing, file writing, flow control, and so forth, and you develop custom modules for application-specific tasks.