Unicode Support

Configure StreamBase Server to process Unicode characters in streams by setting three Java system property settings to UTF-8 for the JVM that runs the server. You can make this change in the engine configuration file, by adding systemProperties properties, as shown in the following example:

name = "sbengine1"
version = "1.0.0"
type = "com.tibco.ep.streambase.configuration.sbengine"
configuration = {
  StreamBaseEngine = {
    systemProperties = {"file.encoding" = "UTF-8",
      "sun.jnu.encoding"="UTF-8",  
      "streambase.tuple-charset" = "UTF-8" }
  }
}

(Notice that the separator between streambase and tuple is a period, while the separator between tuple and charset is a hyphen.)

To enforce this change while running applications in Studio, the configuration file must be placed in the src/main/configurations folder of your Studio project.

Configure StreamBase Studio to process and display Unicode characters in streams by setting the Java system property streambase.tuple-charset to UTF-8 for the JVM that runs Studio. In this case, you must make the change using the environment variable STREAMBASE_STUDIO_VMARGS.

STREAMBASE_STUDIO_VMARGS=-Xms512M -Xmx1024M

Configure this environment variable globally for your system, or temporarily in the UNIX terminal or StreamBase Command Prompt environment from which you run the sbstudio command. Use a command like the following for Windows. This example is shown on two lines for publication clarity, but should be typed as one long line:

set STREAMBASE_STUDIO_VMARGS=-Xms512M -Xmx1024M
    -Dstreambase.tuple-charset=UTF-8

Use a line like the following example for the Bash shell in Linux:

export STREAMBASE_STUDIO_VMARGS=-Xms512M -Xmx1024M \
    -Dstreambase.tuple-charset=UTF-8

For complete Unicode support, you must configure both ends of any communication with StreamBase Server. This applies to StreamBase client applications written with any of the StreamBase Client Libraries.

For clients written with the StreamBase Java API, you must configure the JVM that runs your client code. You can do this in any of three ways:

For clients written with the StreamBase C++, .NET, or Python APIs, you must set the environment variable STREAMBASE_TUPLE_CHARSET=UTF-8 in the environment that runs your client application.

You can perform expression language operations such as substr() on Unicode strings. Unicode strings on input streams are canonicalized to UTF-8 NFC (Normalization Form C, as described in Unicode Normalization Forms).

With Unicode support enabled as described above, some of the expression language functions that deal with strings have different behavior than in the default configuration with Unicode disabled. For example, for some functions, characters are counted as a number of graphemes with Unicode enabled, but as a number of bytes with Unicode disabled. Each affected function is noted on the Expressions page.

EventFlow applications are saved in an XML format that specifies UTF-8 encoding, which ensures that any Unicode strings you use in expressions in operators are preserved. In addition, all other files created and saved by StreamBase Studio, such as server configuration files, are saved as Unicode-compliant files with UTF-8 encoding.

The UTF-8 encoding of files created by StreamBase Studio occurs by default, and is independent of the Server, Studio, or client configuration settings described in the first three sections of this page.

Back to Top ^