Using the Apache Kudu Operators

This topic describes how to use the TIBCO StreamBase® for Apache Kudu operators to interact with an Apache Kudu database, and explains how to configure the operators' Properties views.

The Apache Kudu connectivity solution is implemented as a suite of five global Java operators that allows a StreamBase application to connect to a Kudu database and access its data.

Apache Kudu is a distributed database management system designed to provide a combination of fast inserts/updates and efficient columnar scans.

In addition to four write operators dedicated to insert, delete, update and upsert operations, a Scanner operator allows read operations. The operation of all five operators is described in this document.

The different Kudu operators share a connection to the same database, provided they are configured to do so. Each operator lists the clusters available in the a combo box (see Properties: Operator Properties Tab). The list's values are specified in a dedicated section of the application's Kudu.conf file. Here is an example of such a section, containing all supported settings (long lines wrap to the next, for clarity):

name = "Kudu.conf"
type = "com.tibco.ep.streambase.configuration.adapter"
version = "1.0.0"
configuration = {
        
// An adapter group type defines a collection of EventFlow adapter configurations, indexed by adapter type.
  AdapterGroup = {
        
// A collection of EventFlow adapter configurations, indexed by adapter type. This object 
// is required and must contain at least one configuration.
    adapters = {
        
// The root section for an EventFlow adapter configuration.
      kudu = {
        
// Section list. This array is optional and has no default value.
        sections = [
        
// A configuration for an EventFlow adapter named section.
          {
        
// Section name. The value does not have to be unique; that is, you can have multiple sections with
// the same name in the same array of sections. This property is required.
            name = "master-definition"
        
// Section for setting adapter properties. All values must be strings. This object is optional 
// and has no default value.
              settings = {
                hosts = "kuduserver1:7051;kuduserver2:7051"
                id = "Test Cluster"
                bossCount = "0"
                workerCount = "0"
                disableStatistics = "false"
            }
          }
        ]
      }
    }
  }
}

A best practice is to define your clusters before placing operator instances on the canvas, so that the list is already available in the Properties view and the operators can be configured quickly.

This section describes the properties you can set for the each of the Kudu operators, using the various tabs of the Properties view in StreamBase Studio.

By default each operator contains only one input port, which is used to receive commands describing an operation to be performed on the Kudu cluster. In all cases, any field found in the input schema that does not match what is described below is ignored and passed through directly in the result output tuples.

In addition, you can configure the Insert, Update, Upsert and Delete operators to have one more input port to receive Connect, Disconnnect and Flush commands (in MANUAL_FLUSH flush mode).

Every operator contains two fixed output ports. The first is used to deliver the results of executing operator commands and the second emits status tuples on interesting events, such as errors and connection events.

When exchanging tuples between StreamBase and Kudu, the fields and field types of these tuples are expected to match in the following ways:

The table below lists all the translation types that the operator supports.

The StreamBase installation comes with a sample demonstrating the use of this operator. To load the sample in StreamBase Studio, select File>Import Samples and Community Content and search the Extending StreamBase section for an entry called Kudu Operator.

Back to Top ^