- Index

< Previous |  | Next >

# EventFlow Container Overview

StreamBase containers are the building blocks for organizing and connecting multiple StreamBase applications. There is one StreamBase application per container, and each container name serves as a handle for that application's resources. Multiple containers and applications can be run in a single StreamBase Server process, and you can share resources between applications.

## Container Overview

Containers enable a number of useful features. You can:

- Dynamically add or remove StreamBase applications from the running StreamBase Server process.

- Use containers to run multiple applications in a single server process, without necessarily connecting the applications' resources.

- Connect the output of one StreamBase application to the inputs or outputs of one or more other StreamBase applications, all running within the same server.

- Establish container connections even for servers running on separate physical servers.

- Dynamically modify an existing container in a running application, to add or change a container connection, or to start, stop, or pause enqueuing or dequeuing.

- Specify containers and container connections from the command line with the sbadmin command, or through a deployment file, or with the `StreamBaseAdminClient` class.

The following is an example of command-line container commands:

    sbadmin addContainer c1 newapp.sbapp
    sbadmin addContainer c2 myapp-same-schema.sbapp c2.InputStream=c1.OutputStream

This example defines two containers, `c1` and `c2`, and associates each with an application. The last expression specifies that the output of the `c1` container is to be consumed by the input stream of the `c2` container. Both containers are running in the same StreamBase Server process.

Each container has its own thread of execution within the containing StreamBase Server. Containers behave much like operators or modules marked with the Run this component in a parallel region setting on the Concurrency tab of the Properties view. On a multi-core machine, one container does not block the execution of another container.

The following set of conceptual diagrams illustrates the ideas behind containers. The first diagram illustrates a configuration running multiple unrelated StreamBase applications in a single sbd process:

In the next diagram, we add two reader applications (B and C) to a running sbd process that was already hosting application A:

In the next diagram, we add two writer applications (B and C) to a running sbd process that was already hosting application A. Then we added application D to write its output stream to the output stream of application A.

### Notes

- In the diagrams above, the StreamBase applications are shown for simplicity as having the same components. This is for illustration only; applications in containers are not required to have the same set of components. However, the streams used in container connections between applications must have the same schema.

- When removing containers and applications from a running server process, try to remove them in the reverse order that they were added.

## Container Name Standards

The name you assign to containers must follow the StreamBase identifier naming rules, as described in Identifier Naming Rules.

## Containers and StreamBase Path Notation

StreamBase's formal path notation is designed to avoid naming conflicts when you run multiple applications in the same server, when you use modules, and when operators are run in separate threads.

StreamBase path notation uses the following syntax, with a period separator between each element:

    [containerName.][moduleRefName[[.moduleRef2Name].[..]]entityName

The container name is shown as optional because in most contexts the default container is used and need not be specified.

For additional information, see StreamBase Path Notation.

## The System Container

When an sbd process starts up — with or without an initial application loaded — a container named `system` is always present and running. The system container has the following output streams:

error

  Emits a tuple containing each StreamBase Server runtime error, if any.

  - The schema of the `system.error` stream is described in Schema of the StreamBase Error Tuple.

  - Starting with StreamBase 7.2.0, the `system.error` stream is included by default in the list of streams shown in the Output Streams view in Studio's Test/Debug perspective.

control

  Emits messages from this StreamBase Server in response to certain system-level events. The control stream is discussed in more detail in Using Control Stream Features.

statv2

  Emits tuples containing StreamBase monitoring statistics for operators and queues in the running server. (In StreamBase releases before 7.0, the statistics stream was named `stat`.) See Profiling for the tools that use this stream.

connections

  Emits a tuple every time a client connects to or disconnects from this server. The schema of the `system.connections` stream is the following:

  Field | Data Type | Comments
  --- | --- | ---
  connectionid | string | A global ID for this connection. This is first field returned from an sbadmin listConnections command, and is used as the argument for the getClientIP() function.
  connect | bool | Shows `true` for a connection event or `false` for a disconnection event.
  time | timestamp | The date and time of the connection or disconnection event.

subscriptions

  Emits a tuple every time a client starts or stops dequeuing from any stream on this server. The schema of the `system.subscriptions` stream is the following:

  Field | Data Type | Comments
  --- | --- | ---
  connectionid | string | A global ID for the connection, the same ID described for the `connections` stream.
  path | string | The path to the stream that had a dequeue event, using StreamBase path notation. Examples: `system.control`, `default.outputstream`
  logicalPath | string | The path to the stream specified in a filtered subscription, or the `path` field repeated, if not filtered. See Narrowing Dequeue Results with Filtered Subscribe.
  subscribe | bool | Shows `true` when a stream starts dequeuing or `false` when a stream stops dequeuing.
  time | timestamp | The date and time of the dequeue event.

## Containers and StreamBase URI Syntax

The URI syntax for addressing StreamBase applications allows you to connect to a particular container in a running application. The URI syntax is:

    sb://host:port/container-name

For example:

    sb://localhost:10000/holder2

If you leave off the `container-name` portion of the URI, the container named `default` is assumed. Thus, the following commands are equivalent:

    sbc -u sb://localhost:10000/default
    sbc -u sb://localhost:10000

When you specify a container name in the URI, you can omit the container name from any subsequent stream or operator names. For example, the following commands are equivalent:

    sbc -u sb://localhost/holder2 enqueue InputStream1
    sbc -u sb://localhost enqueue holder2.InputStream1

If you start the server without specifying an application name, the sbd process includes a `system` container, but no `default` container. In this case, you may see errors when the default container name is assumed in commands. For example, type the following command in a terminal window or a StreamBase Command Prompt:

    sbd -p 9999

This starts an instance of the server on port 9999 without an application running. In another window, type the following command:

    sbc -p 9999 list

The response is:

    container       system
    sbc: Container: default does not exist

See sburi for the details of StreamBase URI syntax.

## Multiple URI Syntax

Some commands that accept StreamBase URIs can specify a comma-separated list of URIs to address two or more StreamBase Servers at once. This feature is used primarily to support a cluster of StreamBase Servers that are using high availability features.

See Multiple URI Syntax for HA Scenarios for a discussion of this subject.

## Addressing Multiple Containers Independently

If a server process has multiple containers, you can enqueue to or dequeue from multiple containers at the same time. For example, you can dequeue with a command like the following:

    sbc dequeue holder1.OutputStream1 holder2.OutputStream1

If you specify a container name in the URI, you can still dequeue or enqueue from different containers. For example:

    sbc -u sb://localhost/holder1 dequeue OutputStream1 holder2.OutputStream1

`OutputStream1` will be dequeued from the application in container `holder1`, while `holder2.OutputStream1` will be dequeued from the container `holder2`.

Because tuples are queued between applications, there is overhead introduced when tuples cross modules.

## Adding Containers

There are several ways to add a container to a running StreamBase Server process:

- Use the sbadmin command, targeting a running server process. The format is:

      sbadmin addContainer container-name application-name.[ssql | sbapp | sbar]

- Use the `StreamBaseAdminClient` class in a custom client application.

## Removing Containers

You can remove containers from a running sbd server process with the sbadmin command (or with the `StreamBaseAdminClient` class). For example:

sbadmin removeContainer `container-name`

When a container is removed:

- Enqueuer clients that were accessing the application in that container receive an exception:

  ContainerManager$NonExistentContainerException

- Dequeuer clients receive an EOF if they are dequeuing only from that container. If they are connected to multiple containers, then they continue to receive tuples from the other containers.

- Containers that depend on a removed container for input or output continue to function. Their input or output will disappear.

## Getting a List of Containers

The sbc list command (and the underlying Client library) allows you to show all the containers in a running server. The following shows example output from sbc list:

    stream ....
    schema ...
    operator ...
    container   default
    container   holder2
    container   system

The above example shows three containers. Two are defined by the user, and the third is provided by StreamBase.

## Getting More Information About Containers

The `sbc describe` command shows you the container name and any connections specified when that container was added.

## Managing Running Applications in Containers

You can use the sbadmin command to manage multiple container-bound applications running in a single sbd process. The relevant sbadmin commands are:

    sbadmin {shutdown | suspend | resume} container-name

You can use sbadmin modifyContainer to:

- add a connection between running containers

- remove a container connection

- start, stop, or pause an enqueue to or dequeue from a container

See sbadmin for details on the `modifyContainer` command.

## Monitoring Container Information

StreamBase Manager and StreamBase Monitor can display statistics from application running in containers. While running sbmonitor, press c to toggle the display of container names in the Operator Name column.

See StreamBase Monitor and StreamBase Manager for more information on the StreamBase monitoring utilities.

## Enabling and Disabling Enqueue or Dequeue

Using either the `sbadmin addContainer` or `sbadmin modifyContainer` commands, you can dynamically start, stop, and pause enqueuing or dequeuing in containers.

Both commands take the following syntax:

    --enqueue=STREAMSTATUS
    --dequeue=STREAMSTATUS

The values for STREAMSTATUS can be:

- ENABLED. This is the default.

- DISABLED. The enqueue or dequeue will actively refuse any enqueue or dequeue requests, and throw an exception.

- DROP_TUPLES. The enqueue or dequeue will silently drop tuples.

When you interrupt queuing for an application running in a container, you also interrupt queuing for any external clients, input adapters, and output adapters that were communicating with that application. The following table explains how various combinations are affected:

Client or Adapter Type | ENABLED | DROP_TUPLES | DISABLED
--- | --- | --- | ---
Input adapter | Can enqueue | Off | Off
Enqueuing client | Can enqueue | Tuples silently dropped | Connections actively refused, with exception thrown
Output adapter | Can dequeue | off | Off
Dequeuing client | Can dequeue | Tuples silently dropped | Connections actively refused, with exception thrown

## Client Library Support for Containers

The StreamBase Client libraries for Java, C++, .NET, and Python supports containers. You can write your own applications to connect to and interact with StreamBase Servers and the containers therein.

See API Guide for more information.

## Unsupported Features

The following StreamBase features are not supported with containers:

- Multiple applications per container

- Ad-Hoc queries

## Related Topics

- sbc

- sbadmin

- API Guide