- Index

< Previous |  | Next >

# HOCON Syntax Reference

## Overview

This article describes the HOCON configuration syntax.

## Configuration Syntax

The configuration syntax for StreamBase is Human-Optimized Configuration Object Notation (HOCON). HOCON is a superset of JavaScript Object Notation (JSON), enhanced for readability. The specification for HOCON syntax is found here. Features in HOCON that are unchanged from JSON are described here.

TIBCO StreamBase extends standard HOCON in several ways, including:

- The first three lines of each configuration file must contain the `name`, `version`, and `type` keywords as a header, which together uniquely identify the configuration realm and version of an individual configuration file.

- StreamBase HOCON does not support the `include` keyword defined in the HOCON specification. Instead, StreamBase HOCON recognizes a multi-line array syntax within square brackets as a way to embed a subordinate configuration specification within another. Multi-line array syntax is bounded by three double-quote symbols; the brackets define an array:

      [
      """
      embedded configuration
      """
      ]

Further differences between standard HOCON format and configuration file syntax are described below.

## Terminology

These terms are used to describe configuration data:

Root Object

  Root objects are the first-level children in the configuration envelope for a type. Each type defines one or more root objects. A configuration envelope — that is, a single .`conf` file — can contain zero or one root objects for each schema defined. For example, LocalAuthenticationRealm and TrustedHosts are root objects in this configuration.

      configuration = {
        LocalAuthenticationRealm = { ... }
        TrustedHosts = { ... }
      }

Object

  Objects are descendants of a root object and their contents are delimited with braces. For example, `nodes` and `availabilityZones` are objects in this configuration.

      configuration = {
        NodeDeploy =  {
          nodes = { ... }
            availabilityZones = { ... }
        }
      }

Array

  Arrays are a keyword whose contents are delimited with brackets, such as `jvmArgs` in this configuration:

      configuration = {
        JavaEngine =
        {
           jvmArgs =
           [
              "-Xmx1024m"
              "-Xms512m"
              "-XX:MaxPermSize=768m"
              "-Xdebug"
              "-Xnoagent"
              "-Djava.compiler=NONE"
              "-Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=5005"
            ]
        }
      }

Property

  Properties are fields that do not contain an object or an array. For example, `minimumDispatchThreads` is a property in this configuration.

      configuration = {
        JavaEngine = {
          jvm = {
            minimumDispatchThreads = 10
          }
        }
      }

## File Names and Encoding

StreamBase `runtime`, non-runtime StreamBase, and Live Datamart configuration files must have a suffix of `.conf`. The basename of the file can be any valid name supported by the underlying operating system. There is no special significance associated with the file name; that is, configuration files can be named anything that reminds you of their contents.

Certain configuration file types are generated by StreamBase Studio with conventional names, such as `engine.conf` and `node.conf`. These file names are not required, but are suggested starting points.

A configuration file's character encoding can be UTF-8, UTF-16, or UTF-32 as defined by the JSON specification.

## Headers and Envelope

Every configuration file must start with a header that contains the configuration type, name, and version identifiers. These identifiers are all string values. Following the header, a configuration envelope must be specified in which the configuration data is contained. The configuration data allowed within the configuration envelope is defined by the configuration type specified in the header.

The supported configuration types are:

StreamBase Runtime configuration types, to specify and manage StreamBase Runtime applications, nodes, engines, cluster monitoring, and the node-to-node security realm.

com.tibco.ep.dtm.configuration.application

  See Runtime Application Configuration

com.tibco.ep.dtm.configuration.node

  See Runtime Node Configuration

com.tibco.ep.dtm.configuration.javaengine

  See Runtime JavaEngine Configuration

com.tibco.ep.dtm.configuration.security

  See Runtime Security Configuration

com.tibco.ep.dtm.configuration.clustermonitor

  See Runtime ClusterMonitor Configuration

com.tibco.ep.dtm.configuration.servicediscoveryadapter

  See Runtime Service Discovery Configuration

StreamBase configuration types to specify and manage features of StreamBase EventFlow Fragments running within a StreamBase Runtime node.

com.tibco.ep.streambase.configuration.sbengine

  See StreamBase Engine Configuration

com.tibco.ep.streambase.configuration.sbclientapilistener

  See StreamBase Client API Listener Configuration

com.tibco.ep.streambase.configuration.jdbcdatasource

  See StreamBase JDBC DataSource Configuration

com.tibco.ep.streambase.configuration.adapter

  See StreamBase JMS Adapter Configuration

com.tibco.ep.streambase.configuration.ads

  See StreamBase Artifact Distribution Service Configuration

Live Datamart Configuration types to specify and manage features of Live Datamart EventFlow fragments running in a StreamBase Runtime node.

com.tibco.ep.ldm.configuration.ldmclientapilistener

  See Live Datamart Client API Listener Configuration

com.tibco.ep.ldm.configuration.ldmengine

  See Live Datamart Engine Configuration

com.tibco.ep.ldm.configuration.ldminternalcredentials

  See Live Datamart Internal Credentials Configuration

### Configuration Header and Envelope Example

Header values:

    name = "integration-test-application"
    version = "1.0"
    type = "com.tibco.ep.dtm.configuration.application"

Configuration envelope:

    configuration =
    {
    //your configuration data specific to configuration type
    }

## Comment Characters

As in standard JSON and standard HOCON, any text between `//` or `#` and the next newline is considered a comment and is ignored (unless the `//` or `#` is inside a quoted string).

Prior to StreamBase 10.3.0, substitution variables commented AND inside a quoted string were ignored. As of 10.3.0 they are no longer ignored.

## Substitution Variables

Configuration files provide substitution variable support, where the substitution variable values come from (in priority order):

1. The `substitutions=` parameter of the epadmin install node or epadmin load configuration commands.

2. A substitution file specified using the `substitutionfile=` parameter of the same two epadmin commands.

For example, if the same substitution variable value is specified in a substitution file using the install node `substitutionfile=` parameter and also in the `substitutions=` parameter, the value specified in the `substitutions=` parameter is used.

There is no support for using an environment variable value as a substitution variable value. This is different behavior than standard HOCON, which allows environment variable values to be used as a substitution variable value.

Substitution variables must follow these rules:

- Defined using ${`variable-name`[:-`default-value`]}. This defines a substitution variable named `variable-name` with an optional default value of `default-value`.

- `variable-name` can only contain characters specified by this regular expression [a-zA-Z0-9\\.-_]+.

- `variable-name` cannot contain the HOCON escape character (\).

- Substitution variable values, and `default-value` expressions, cannot contain unescaped newlines. Escaped newlines (`\\n`) are permitted, and will result in a newline character appearing in the substituted value.

- Nested substitution variables are not allowed (these are allowed by standard HOCON). For example:

      foo = bar
      bar = biz
      bar = biz ${${foo}} // returns an error instead of the value biz

- Substitution variables are supported anywhere in a configuration file, including the left-hand side of an assignment. This is an extension to standard HOCON.

- Substitution variables on the left-hand side of an assignment that contain the "." character must be surrounded with double quotes to prevent the value from being interpreted as a HOCON path. This is an extension to standard HOCON.

- Substitution variables are supported in quoted strings. This is an extension to standard HOCON.

### Substitution Variable Example

The following is an example of a substitution variable in the configuration file header as well as in the body of the file.

### Note

When using the HOCON Editor in StreamBase Studio, substitution variables MUST be specified with default values in order to pass Studio typechecking. Nevertheless, configuration files like the sample below (without default values) can be run as shown from the epadmin command line.

    //
    // Setting substitution variables on command line
    //
    epadmin load configuration source=my.conf substitutions=
      "NODE_NAME=A.X,PROJECT_VERSION=1.0"

    //
    // Use substitution variable in configuration file header
    //
    name     = "integration-test-application"
    version  = ${PROJECT_VERSION}
    type     = "com.tibco.ep.dtm.configuration.node"
    configuration =
    {
      NodeDeploy =
      {
        nodes =
          {
            //
            //  Use substitution variable on left-hand side
            //  of an assignment in a quoted string
            //
            "${NODE_NAME}" = { ... }
          }
      }
    }

### Default Substitution Variable Value Example

The following example shows a configuration file that specifies a default for its substitution variable using the colon-hyphen (`:-`) syntax.

### Note

When using the HOCON Editor in StreamBase Studio, substitution variables MUST be specified with default values in order to pass Studio typechecking. Nevertheless, configuration files without default values can be successfully run from the epadmin command line.

    configuration =
    {
      ApplicationDefinition =
      {
        execution
        {
          dataTransport =
            {
              //
              //    Default value of 10 if not specified
              //
              nodeActiveTimeoutSeconds = ${TIME_OUT_VALUE:-10}
            }
         }
      }
    }

## Escaping Characters

Per the standard HOCON specification, the \ escape character is used to escape the next character in the file. In all cases, except one, the escape character is significant and must follow the standard HOCON parsing rules. The one exception is escape characters in front of substitution variable definitions are ignored. For example:

    configuration =
    {
        name = \${value}
    }

    //
    //  Processed as (escape character stripped)
    //
    configuration =
    {
        name = ${value}
    }

To escape the $ character that starts a substitution variable definition use:

    configuration =
    {
        name = \\\${value}
    }

    //
    //  Processed as (substitution variable escaped)
    //
    configuration =
    {
        name = \${value}
    }

## Built-in Substitution Variables

Built-in substitution variables are available. You can use built-in substitution variables anywhere in a configuration file where a standard substitution variable can be used. Built-in substitution variables do not need to specified. Built-in substitution variables include:

- `EP_NODE_NAME` — node name.

The `EP_NODE_NAME` variable makes it easier to create "elastic" node deployment files such as:

    name = "MyApplication"
    version = "2.0"
    type = "com.tibco.ep.dtm.configuration.node"
    configuration = {
      NodeDeploy = {
        nodes = {
          "${EP_NODE_NAME}" = {
          }
        }
        availabilityZones = {
          zone1 = {
            dataDistributionPolicy = "dynamic"
          }
        }
      }
    }

## Data Unit Syntax

The values in configuration objects follow the standard HOCON syntax, including support for unit formatting (for example: ms, bytes, MB, and so on). The complete list of the standard supported unit formatting can be found here.

## Date Syntax

The supported date formats for date fields are:

- ISO8601 combined date and time — yyyy-MM-dd'T'HH:mm:ss

- ISO8601 with milliseconds — yyyy-MM-dd'T'HH:mm:ss.SSS

- Time only — HH:mm:ss.

Example:

    //   Date value with timezone
    //
    dateValue = "1970-01-30T12:34:00"

    //
    //    Date value with milliseconds
    //
    dateValue = "1970-01-30T12:34:00.100"

    //
    //   Time-only date value
    //
    dateValue = "12:34:56"

## Configuration File Includes

HOCON configuration files can be embedded in another configuration file. The following example shows a complete node configuration file defined by its header lines at the top, that embeds an entire sbclientapilistener configuration file starting just below the fragmentIdentifier property.

The embedded configuration file is bounded by a pair of triple quotes (`"""`) within a pair of square brackets.

    name = "NodeDeployment"
    version = "1.0"
    type = "com.tibco.ep.dtm.configuration.node"
    configuration = {
      NodeDeploy = {
        nodes = {
          "B.sbuser" = {
            engines = {
              engine1 = {
                fragmentIdentifier = "com.example.sample_operator"
                configuration =
                [
                  """
                  name = "myapilistenersettings"
                  version = "1.0.0"
                  type = "com.tibco.ep.streambase.configuration.sbclientapilistener"
                  configuration = {
                    ClientAPIListener = {
                      apiListenerAddress = {
                        portNumber = 10000
                      }
                    }
                  }
                  """
                ]
              }
            }
          }
        }
      }
    }

## HOCON Configuration File Editor

The HOCON Configuration File Editor is a validating HOCON editor that is aware of the structure that defines the HOCON syntax of StreamBase configuration files.

See HOCON Configuration File Editor for instructions on using the Editor.