Kafka Admin Adapter

Contents

Introduction
Adapter Properties
Command Input Port
Topics Output Port
Brokers Output Port
Status Output Port
Related Topics

Introduction

The Spotfire Streaming Output Adapter for Apache Kafka Admin allows StreamBase applications to perform various administration requests to an Apache Kafka broker.

The Apache Kafka adapter suite is implemented against the version of the Kafka libraries listed on the Supported Configurations page. Note however that this version of the Apache Kafka client API supports connections to older Apache Kafka brokers back to v2.8.

Adapter Properties

This section describes the properties you can set for this adapter, using the various tabs of the Properties view in StreamBase Studio.

General Tab

Name: Use this required field to specify or change the name of this instance of this component. The name must be unique within the current EventFlow module. The name can contain alphanumeric characters, underscores, and escaped special characters. Special characters can be escaped as described in Identifier Naming Rules. The first character must be alphabetic or an underscore.

Adapter: A read-only field that shows the formal name of the adapter.

Class name: Shows the fully qualified class name that implements the functionality of this adapter. If you need to reference this class name elsewhere in your application, you can right-click this field and select Copy from the context menu to place the full class name in the system clipboard.

Start options: This field provides a link to the Cluster Aware tab, where you configure the conditions under which this adapter starts.

Enable Error Output Port: Select this checkbox to add an Error Port to this component. In the EventFlow canvas, the Error Port shows as a red output port, always the last port for the component. See Using Error Ports to learn about Error Ports.

Description: Optionally, enter text to briefly describe the purpose and function of the component. In the EventFlow Editor canvas, you can see the description by pressing Ctrl while the component's tooltip is displayed.

Adapter Properties Tab

Property Type Description
Bootstrap Servers string A comma-separated list of address:ports specifying one or more Kafka bootstrap servers to connect to for issuing commands. The default value is localhost:9092
Enable Topics Port check box When selected, specifies that the adapter sends topic information on its topics port when the adapter receives a topics command. The default value is selected.
Enable Brokers Port check box When selected, specifies that the adapter sends broker information on its brokers port when the adapter receives a brokers command. The default value is selected.
Enable Read Value Port check box When selected, specifies that the adapter outputs value information on its read value Port based on a readValue command. The default value is selected.
Enable Status Port check box When selected, the adapter sends out informational data on its Status Port about various states of the adapter. The default value is cleared.
Session Timeout (MS) int The Zookeeper session timeout in milliseconds. Default: 6000 milliseconds.
Connection Timeout (MS) int The Zookeeper connection timeout in milliseconds. Default: 6000 milliseconds.
Log Level INFO Controls the level of verbosity the adapter uses to issue informational traces to the console. This setting is independent of the containing application's overall log level. Available values, in increasing order of verbosity, are: OFF, ERROR, WARN, INFO, DEBUG, TRACE.

Connection Settings Tab

This tab contains a table used to specify any properties to use when establishing a connection to the bootstrap server using a Kafka Admin object. This is where, for example, any SLL-related settings can be specified. Values for these settings may be enciphered for security.

Cluster Aware Tab

Use the settings in this tab to enable this operator or adapter for runtime start and stop conditions in a multi-node cluster. During initial development of the fragment that contains this operator or adapter, and for maximum compatibility with releases before 10.5.0, leave the Cluster start policy control in its default setting, Start with module.

Cluster awareness is an advanced topic that requires an understanding of StreamBase Runtime architecture features, including clusters, quorums, availability zones, and partitions. See Cluster Awareness Tab Settings on the Using Cluster Awareness page for instructions on configuring this tab.

Concurrency Tab

Use the Concurrency tab to specify parallel regions for this instance of this component, or multiplicity options, or both. The Concurrency tab settings are described in Concurrency Options, and dispatch styles are described in Dispatch Styles.

Caution

Concurrency settings are not suitable for every application, and using these settings requires a thorough analysis of your application. For details, see Execution Order and Concurrency, which includes important guidelines for using the concurrency options.

Command Input Port

The Command input port is always enabled by default. Use the command port to send action commands to the Kafka Admin adapter.

The minimum schema for the command input port is a single field named command of type string. A command port with this schema can accept the brokers and topics commands, to generate output on the brokers and topics output ports.

The suggested full schema for the command input port is the following:

  • command, string. The command to send to the adapter. Valid (case-insensitive) values are: createTopic, deleteTopic, brokers (to list available brokers on the cluster; results issued on the Brokers output port), topics (to list available topics on the cluster; results issued on the Topics output port).

  • topic (optional), string. The topic to which to subscribe or unsubscribe.

  • partition (optional), integer. The partition to subscribe to. Leave null for default of 0.

  • replication (optional), integer. Defaults to 1.

  • properties, list(tuple), with the tuple's schema being {key (string), value (string)}. Properties used when running a createTopic command.

Topics Output Port

Description

The topics port, when enabled in the Adapter Properties tab of the Properties view, emits tuples in response to a topics command.

Topics Port Schema

  • topics, list(tuple). One list member for each topic. The schema of the tuple is:

    • topic, string. The name of this topic.

    • properties, list(tuple). Where the tuple's schema is key (string), value (string).

    • partitions, list(integer) —

Brokers Output Port

Description

The brokers port, when enabled in the Adapter Properties tab of the Properties view, emits tuples in response to a brokers command.

Brokers Port Schema

  • topics, list(tuple). One list member for each broker. The schema of the tuple is:

    • host, string —

    • port, integer —

    • id, integer —

Status Output Port

Description

The status port is used to send status information tuples downstream to inform the user of changes.

The status port, when enabled in the Adapter Properties tab of the Properties view, emits status information tuples downstream to inform the user of changes.

Status Port Schema

  • type, string. The type of status information emitted on this port. Status types are:

    • Error—This message relates to an error that occurred.

    • Warn —This message relates to a warning that the user should be aware of.

    • Info—This message relates to extra status information.

  • action, string. A specific action related to this status message.

  • object, string. This value may be null. If it is not null, it contains a value relevant to the status message.

  • message, string. This is a formatted human readable message that explains the status message.

  • time, timestamp. The time the status message occurred.

Related Topics