System Container and Control Streams

When an EventFlow engine process starts up, an EventFlow container named system is always included alongside the container, usually named default, that holds the top-level EventFlow module. The system container emits continuous streams of information and statistics about the currently running server engine.

There are several system streams emitted from server, described in sections below:

You can see the containers in a running fragment by issuing an sbc list command or

epadmin --ad=adminport display container

In general, do not make a long-running direct connection to a system stream with an sbc dequeue command. (The epadmin command does not allow dequeuing from system streams.) To monitor control streams, use a container connection between EventFlow containers with an asynchronous connection type, or use the StreamBaseMonitor API for Java or .NET.

Do not design a StreamBase application architecture that assumes you can make long-term connections to any system stream with the synchronous connection type.

StreamBase provides a number of ways to connect to and interpret system streams, especially the busy system.statv2 stream:

Emits a tuple every time a client connects to or disconnects from this server. The schema of the system.connections stream is the following:

Tuples are emitted from the control stream in response to certain system-level events. These tuples have the following schema:

The use and meaning of fields in the control stream schema differ according to which control stream subsystem sends the tuple. The subsystems are: CONTAINER, CLUSTER, HEARTBEAT, and HA (deprecated). You can identify the subsystem type from the string sent in the first field.

name = "clientapilistenersettings"
version = "1.0.0"
type = "com.tibco.ep.streambase.configuration.sbclientapilistener"
configuration = {
  ClientAPIListener = {
    apiListenerAddress = {
      portNumber = 10000
    }
    ...
    clientHeartbeatIntervalMilliseconds = 60000
  }
}

The system.subscriptions stream emits a tuple every time a client starts or stops dequeuing from any stream on this server. The schema of the system.subscriptions stream is the following:

The system.error stream is part of the EventFlow try-catch error streams system. This system and the schema of this stream are described on Using Error Ports and Error Streams.

The system.statv2 stream continuously emits tuples containing EventFlow engine monitoring statistics for operators, queues, threads, client connections, and memory use in the running server.

As with all system streams, do not make a synchronous container connection to the system.statv2 stream. The statistics tuples are not designed to be human readable or parseable; use one of the tools listed in Connecting to System Streams to interpret this stream.

For reference when developing a custom monitor app, the following is the schema of the statv2 stream:

The system.statv3 stream is an optional alternative to the statv2 stream described above. An EventFlow engine emits either a system.statv2 or system.statv3 stream, and must be configured to emit the latter. Without configuration, the statv2 stream is the default.

The statv3 stream reports the same statistics information as statv2, but in a different, binary format that is not human readable. All of the StreamBase tools that interpret a statv2 stream, listed in Connecting to System Streams above, can detect and interpret a statv3 stream, if encountered. The statv3 stream can result in around a ten times improvement in network bandwidth usage, especially with large LiveVew projects.

To enable the statv3 stream for an EventFlow engine, set the Java system property streambase.appgen.stats.schema.version equal to 3. The following example configures to use the statv3 stream and also slows down the frequency of each statistics tuple emission:

name = "sbengine"
type = "com.tibco.ep.streambase.configuration.sbengine"
version = "1.0.0"
configuration = { 
  StreamBaseEngine = { 
    systemProperties {
      "streambase.appgen.stats-schema-version" = "3" 
    }   
    streamBase = { 
      engineMonitor = { 
        statsFrequencyMilliseconds = 10000
      }   
    }   
  }
}

For reference, the schema of the statv3 stream has three of the same fields as statv2, but adds a blob field to contain the binary statistics information: