Using StreamBase Logging

For all supported operating systems, StreamBase uses the industry-standard SLF4J logging system with the Logback framework. See Windows Event Logging for additional considerations for Windows.

All subsystems in EventFlow and LiveView fragments use Logback logging. You can configure the level of logging information emitted, and can direct logs to different appenders using standard Logback configuration files in either XML or Groovy formats.

Log files are written by default into a subdirectory named logs in a fragment's node directory:

To configure a custom Logback configuration files see Custom Configuration and StreamBase Logback Extensions below. The default logs subdirectory location can be overridden as described in StreamBase Logback Extensions.

The StreamBase Runtime processes log configuration in the following order:

Logging configurations do not merge. That is, a later configuration in the processing order sequence overrides an earlier one. For this reason, any custom logging configuration file should specify settings for both RootFileAppender and RootConsoleAppender to avoid error messages.

Logging settings are ultimately specified per engine. If your fragment has more than one engine, each engine can inherit the settings made for the fragment and application as a whole.

The logging level and logging configuration can be changed at runtime per engine using the logging target of the epadmin command, as described in Managing Logging at Runtime.

In addition to the logging messages emitted by EventFlow and LiveView as a whole, you can add logging instructions to any EventFlow module with the Log Output adapter. The adapter has a sample described in Log Output Adapter Sample that illustrates formatting options you can perform with the Log Output adapter.

Do not confuse the Java logging facility discussed here with the StreamBase error stream feature. Logging is for writing your own logging messages and exception stack traces to one or more log files. By contrast, the error stream feature is like a module-level throw that ascends up the call stack to the containing module, and from there to the top-level EventFlow module instance running the fragment. StreamBase sends an error tuple to the error stream when it encounters an exception during a tuple processing method. See Using Error Ports and Error Streams for further information.

If you write custom operators or adapters, you can use an import org.slf4j.Logger statement in your code to define a logger for your operator or adapter. The base class, Operator, provides the convenience method getLogger(), which returns an org.slf4j.Logger object that should thereafter be used for logging messages and exceptions in your code. For the full list of methods available on Logger objects, refer to the Logger class in the SLF4J Javadoc.

The StreamBase installation includes a logging sample that illustrates the difference between using a Log Output adapter and using getLogger() methods in your custom code. This sample compares both methods with a third module that does not log.

StreamBase uses the standard log level meanings provided by the Logback framework, as defined in the Logger API.

The following table maps the log level integer settings used in StreamBase releases before 10.2.0 to Logback message levels.

Most StreamBase adapters allow you to set a log level independently for the adapter than for the hosting EventFlow module:

For example, an adapter might set the log level to WARN or ERROR for messages emitted from the adapter while the hosting fragment is still set to its default INFO level. This allows you to troubleshoot adapter connectivity issues without having to wade through hundreds of verbose server messages from the fragment.

StreamBase's default settings for Logback logging are as follows:

For fragments launched from Studio, the logs subdirectory includes three log file types:

The components of the default log message pattern have the meanings shown in the following table.

Consult the Logback documentation for PatternLayout for the available log message pattern components.

You can override any of the default settings with a custom Logback configuration.

To configure Logback logging, specify a Logback configuration file. StreamBase accepts files in either the traditional XML format, often named logback.xml, or in the alternative Groovy format with .groovy extension. In either case, place your configuration files in the src/main/resources folder of your Studio project.

You can also use the src/test/resources folder for a logging configuration you only need when running Maven tests. To have one configuration used both for standard fragment launches and tests, use src/test/resources.

The StreamBase logging sample includes an example logback.xml configuration file that illustrates a setup for both a console appender and a rolling file appender.

The Groovy language format for Logback configuration files is described on https://logback.qos.ch/manual/groovy.html. One of the advantages of defining your configuration using Groovy is that you can introduce conditional statements. For example, you can specify a certain console appender format when running on all development hosts, but not on production hosts.

The Logback site provides an XML to Groovy translator at https://logback.qos.ch/translator/asGroovy.html.

In general, StreamBase Logback configuration files use standard Logback patterns as described in the Logback documentation for PatternLayout. StreamBase provides the following extensions to the standard Logback patterns:

You can define different values for these variables in your configuration file:

<variable name="com.tibco.ep.dtm.logging.logDirectory" value="/Users/sbuser/logs"/>
<variable name="com.tibco.ep.dtm.logging.logFileName" value="testrun"/>
...
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
  <file>${com.tibco.ep.dtm.logging.logDirectory}/${com.tibco.ep.dtm.logging.logFileNamePrefix}.log>/file>
  <encoder>
. ...

Search for "Variable substitution" on this Logback documentation page to learn more about Logback variables.

StreamBase creates entries in an audit log for every epadmin command issued per node, stored by default in the same logs subdirectory of the node directory.

The audit log:

Command audit messages are saved only in the audit log, and are not duplicated in the other node logs.