Container Overview

StreamBase containers are the building blocks for organizing and connecting multiple StreamBase applications. There is one StreamBase application per container, and each container name serves as a handle for that application's resources. Multiple containers and applications can be run in a single StreamBase Server process, and you can share resources between applications.

Container Overview

Containers enable a number of useful features. You can:

  • Dynamically add or remove StreamBase applications from the running StreamBase Server process.

  • Use containers to run multiple applications in a single server process, without necessarily connecting the applications' resources.

  • Connect the output of one StreamBase application to the inputs or outputs of one or more other StreamBase applications, all running within the same server.

  • Establish container connections even for servers running on separate physical servers.

  • Dynamically modify an existing container in a running application, to add or change a container connection, or to start, stop, or pause enqueuing or dequeuing.

  • Specify containers and container connections from the command line with the sbadmin command, or through a deployment file, or with the StreamBaseAdminClient class.

The following is an example of command-line container commands:

sbadmin addContainer c1 newapp.sbapp
sbadmin addContainer c2 myapp-same-schema.sbapp c2.InputStream=c1.OutputStream

This example defines two containers, c1 and c2, and associates each with an application. The last expression specifies that the output of the c1 container is to be consumed by the input stream of the c2 container. Both containers are running in the same StreamBase Server process.

Each container has its own thread of execution within the containing StreamBase Server. Containers behave much like operators or modules marked with the Run this component in a parallel region setting on the Concurrency tab of the Properties view. On a multi-core machine, one container does not block the execution of another container.

The following set of conceptual diagrams illustrates the ideas behind containers. The first diagram illustrates a configuration running multiple unrelated StreamBase applications in a single sbd process:

In the next diagram, we add two reader applications (B and C) to a running sbd process that was already hosting application A:

In the next diagram, we add two writer applications (B and C) to a running sbd process that was already hosting application A. Then we added application D to write its output stream to the output stream of application A.

Notes

  • In the diagrams above, the StreamBase applications are shown for simplicity as having the same components. This is for illustration only; applications in containers are not required to have the same set of components. However, the streams used in container connections between applications must have the same schema.

  • When removing containers and applications from a running server process, try to remove them in the reverse order that they were added.

Container Name Standards

The name you assign to containers must follow the StreamBase identifier naming rules, as described in Identifier Naming Rules.

Containers and StreamBase Path Notation

StreamBase's formal path notation is designed to avoid naming conflicts when you run multiple applications in the same server, when you use modules, and when operators are run in separate threads.

StreamBase path notation uses the following syntax, with a period separator between each element:

[containerName.][moduleRefName[[.moduleRef2Name].[..]]entityName

The container name is shown as optional because in most contexts the default container is used and need not be specified.

For additional information, see StreamBase Path Notation.

Containers Specified in Deployment Files

In StreamBase deployment files, the <application> element has two attributes:

  1. module — the name of an application module on the project's module search path.

  2. container — the container's name. The deployment file must specify one container named default, which should contain the primary, top-level application module. Other container names are up to you.

You specify container connections with <container-connection> child elements of the <container-connections> element of deployment files.

See StreamBase Deployment File XML Reference for details on container and container connection syntax. See Deployment File Overview to understand deployment files.

In StreamBase releases before 7.0, the <application> element was part of the syntax of server configuration files, and <container-connection> was a child element of <application>. This syntax is still supported for backwards compatibility.

The System Container

When an sbd process starts up — with or without an initial application loaded — a container named system is always present and running. The system container has the following output streams:

error

Emits a tuple containing each StreamBase Server runtime error, if any.

  • The schema of the system.error stream is described in Schema of the StreamBase Error Tuple.

  • Starting with StreamBase 7.2.0, the system.error stream is included by default in the list of streams shown in the Application Output view in Studio's Test/Debug perspective.

control

Emits messages from this StreamBase Server in response to certain system-level events. The control stream is discussed in more detail in Using Control Stream Features.

statv2

Emits tuples containing StreamBase monitoring statistics for operators and queues in the running server. (In StreamBase releases before 7.0, the statistics stream was named stat.) See Profiling for the tools that use this stream.

connections

Emits a tuple every time a client connects to or disconnects from this server. The schema of the system.connections stream is the following:

Field Data Type Comments
connectionid string A global ID for this connection. This is first field returned from an sbadmin listConnections command, and is used as the argument for the getClientIP() function.
connect bool Shows true for a connection event or false for a disconnection event.
time timestamp The date and time of the connection or disconnection event.
subscriptions

Emits a tuple every time a client starts or stops dequeuing from any stream on this server. The schema of the system.subscriptions stream is the following:

Field Data Type Comments
connectionid string A global ID for the connection, the same ID described for the connections stream.
path string The path to the stream that had a dequeue event, using StreamBase path notation. Examples: system.control, default.outputstream
logicalPath string The path to the stream specified in a filtered subscription, or the path field repeated, if not filtered. See Narrowing Dequeue Results with Filtered Subscribe.
subscribe bool Shows true when a stream starts dequeuing or false when a stream stops dequeuing.
time timestamp The date and time of the dequeue event.

Containers and StreamBase URI Syntax

The URI syntax for addressing StreamBase applications allows you to connect to a particular container in a running application. The URI syntax is:

sb://host:port/container-name

For example:

sb://localhost:10000/holder2

If you leave off the container-name portion of the URI, the container named default is assumed. Thus, the following commands are equivalent:

sbc -u sb://localhost:10000/default
sbc -u sb://localhost:10000

When you specify a container name in the URI, you can omit the container name from any subsequent stream or operator names. For example, the following commands are equivalent:

sbc -u sb://localhost/holder2 enqueue InputStream1
sbc -u sb://localhost enqueue holder2.InputStream1

If you start the server without specifying an application name, the sbd process includes a system container, but no default container. In this case, you may see errors when the default container name is assumed in commands. For example, type the following command in a terminal window or a StreamBase Command Prompt:

sbd -p 9999

This starts an instance of the server on port 9999 without an application running. In another window, type the following command:

sbc -p 9999 list

The response is:

container       system
sbc: Container: default does not exist

See sburi for the details of StreamBase URI syntax.

Multiple URI Syntax

Some commands that accept StreamBase URIs can specify a comma-separated list of URIs to address two or more StreamBase Servers at once. This feature is used primarily to support a cluster of StreamBase Servers that are using high availability features.

See Multiple URI Syntax for HA Scenarios for a discussion of this subject.

Addressing Multiple Containers Independently

If a server process has multiple containers, you can enqueue to or dequeue from multiple containers at the same time. For example, you can dequeue with a command like the following:

sbc dequeue holder1.OutputStream1 holder2.OutputStream1

If you specify a container name in the URI, you can still dequeue or enqueue from different containers. For example:

sbc -u sb://localhost/holder1 dequeue OutputStream1 holder2.OutputStream1

OutputStream1 will be dequeued from the application in container holder1, while holder2.OutputStream1 will be dequeued from the container holder2.

Because tuples are queued between applications, there is overhead introduced when tuples cross modules.

Adding Containers

There are several ways to add a container to a running StreamBase Server process:

  • In StreamBase deployment files.

  • Use the sbadmin command, targeting a running server process. The format is:

    sbadmin addContainer container-name application-name.[ssql | sbapp | sbar]
    
  • Use the StreamBaseAdminClient class in a custom client application.

Removing Containers

You can remove containers from a running sbd server process with the sbadmin command (or with the StreamBaseAdminClient class). For example:

sbadmin removeContainer container-name

When a container is removed:

  • Enqueuer clients that were accessing the application in that container receive an exception:

    ContainerManager$NonExistentContainerException

  • Dequeuer clients receive an EOF if they are dequeuing only from that container. If they are connected to multiple containers, then they continue to receive tuples from the other containers.

  • Containers that depend on a removed container for input or output continue to function. Their input or output will disappear.

Getting a List of Containers

The sbc list command (and the underlying Client library) allows you to show all the containers in a running server. The following shows example output from sbc list:

stream ....
schema ...
operator ...
container   default
container   holder2
container   system

The above example shows three containers. Two are defined by the user, and the third is provided by StreamBase.

Getting More Information About Containers

The sbc describe command shows you the container name and any connections specified when that container was added.

Managing Running Applications in Containers

You can use the sbadmin command to manage multiple container-bound applications running in a single sbd process. The relevant sbadmin commands are:

sbadmin {shutdown | suspend | resume} container-name

You can use sbadmin modifyContainer to:

  • add a connection between running containers

  • remove a container connection

  • start, stop, or pause an enqueue to or dequeue from a container

See sbadmin for details on the modifyContainer command.

Monitoring Container Information

StreamBase Manager and StreamBase Monitor can display statistics from application running in containers. While running sbmonitor, press c to toggle the display of container names in the Operator Name column.

See StreamBase Monitor and TIBCO StreamBase Manager for more information on the StreamBase monitoring utilities.

Enabling and Disabling Enqueue or Dequeue

Using either the sbadmin addContainer or sbadmin modifyContainer commands, you can dynamically start, stop, and pause enqueuing or dequeuing in containers.

Both commands take the following syntax:

--enqueue=STREAMSTATUS
--dequeue=STREAMSTATUS

The values for STREAMSTATUS can be:

  • ENABLED. This is the default.

  • DISABLED. The enqueue or dequeue will actively refuse any enqueue or dequeue requests, and throw an exception.

  • DROP_TUPLES. The enqueue or dequeue will silently drop tuples.

When you interrupt queuing for an application running in a container, you also interrupt queuing for any external clients, input adapters, and output adapters that were communicating with that application. The following table explains how various combinations are affected:

Client or Adapter Type ENABLED DROP_TUPLES DISABLED
Input adapter Can enqueue Off Off
Enqueuing client Can enqueue Tuples silently dropped Connections actively refused, with exception thrown
Output adapter Can dequeue off Off
Dequeuing client Can dequeue Tuples silently dropped Connections actively refused, with exception thrown

Client Library Support for Containers

The StreamBase Client libraries for Java, C++, .NET, and Python supports containers. You can write your own applications to connect to and interact with StreamBase Servers and the containers therein.

See API Guide for more information.

Unsupported Features

The following StreamBase features are not supported with containers:

  • Multiple applications per container

  • Ad-Hoc queries

Related Topics