Kafka Channel

To send and receive messages from a Kafka broker, use the TIBCO BusinessEvents Kafka channel. The Kafka channel converts the incoming Kafka messages to BusinessEvents events and transforms BusinessEvents events as outgoing Kafka messages.

Before You Begin

Before setting up the Kafka channel, go through the Kafka documentation for information (concepts, architecture, demos, APIs, and so on) about them. The following table lists the URLs that you can see for information about Kafka.

Channel Type	Documentation URL
Kafka	Kafka Documentation

Kafka Channel Serializers

The Kafka channel provides the following serializers to handle payloads:

KafkaMapSerializer

The KafkaMapSerializer serializer (com.tibco.cep.driver.kafka.serializer.KafkaMapSerializer) serializes and deserializes a BusinessEvents event along with its payload into (or from) HashMap. The KafkaMapSerializer is used to send and receive events between BusinessEvents instances. For incoming messages the serializer converts the bytes sequence to event and its payload. For outgoing events the serializer converts the event and its payloads into HashMap as bytes.

KafkaJsonSerializer

The KafkaJsonSerializer serializer (com.tibco.cep.driver.kafka.serializer.KafkaJsonSerializer) serializes and deserializes a BusinessEvents event along with its payload into (or from) JSON. For incoming messages, the KafkaXmlSerializer decodes the text from the message as an JSON string and deserializes it to an event. For outgoing events, the serializer converts the event and its payloads into an JSON string. The KafkaXmlSerializer serializer is useful for processing or sending messages between BusinessEvents and external systems.

KafkaXmlSerializer

The KafkaXmlSerializer serializer (com.tibco.cep.driver.kafka.serializer.KafkaXmlSerializer) serializes and deserializes a BusinessEvents event along with its payload into (or from) a JSON string that does not have any qualifiers. For incoming messages, the KafkaXmlSerializer decodes the text from the message as an XML string and deserializes it to an event. For outgoing events, the serializer converts the event and its payloads into an XML string. The KafkaXmlSerializer serializer is useful for processing or sending messages between BusinessEvents and external systems.

KafkaTextPayloadSerializer

The KafkaTextPayloadSerializer serializer (com.tibco.cep.driver.kafka.serializer.KafkaTextPayloadSerializer) serializes and deserializes a BusinessEvents event along with its payload into (or from) text ignoring event properties. This serializer ignores the event properties even if they are defined. For incoming messages, the KafkaTextPayloadSerializer reads the text from the message and deserializes it to an event. For outgoing events, the serializer converts the event payload into a Kafka message value.

Note: If you are migrating a project from earlier version which contains KafkaStringPayloadSerializer then the project still works in this version. However, the KafkaStringPayloadSerializer serializer is not available for a new Kafka channel.

KafkaAvroSerializer

The KafkaAvroSerializer serializer (com.tibco.cep.driver.kafka.serializer.KafkaAvroSerializer) serializes and deserializes a BusinessEvents event along with its payload into (or from) Avro data format. For incoming messages, the KafkaAvroSerializer reads the data from the Avro message and deserializes it to an event. For outgoing events, the serializer converts the event payload into a Avro message value.

Adding a New Channel

See the following topics for more information about new Kafka channel:

Adding a Kafka Channel in TIBCO BusinessEvents Application

Sample TIBCO BusinessEvents Applications

The following sample TIBCO BusinessEvents application is provided with TIBCO BusinessEvents for Kafka channel to help you understand the channel better:

Channel Type	Sample TIBCO BusinessEvents Application
Kafka	BE_HOME/examples/standard/KafkaChannel

Catalog Functions

You can send and receive events with payload in Kafka channel using Event.sendEvent(), and Event.routeTo(). Currently, Kafka does not support synchronous request-reply of messages; thus, the functions Event.requestEvent() and Event.replyEvent() are not supported.

Kafka Channel Advanced Properties

You can specify Kafka channel advanced properties in the CDD file by adding appropriate prefixes to the channel. By using the same prefix, you can also override an existing Kafka channel property, which has been exposed through the TIBCO BusinessEvents Studio channel editor. For example, for specifying the default.replication.factor Kafka property, use be.channel.kafka.default.replication.factor.

Channel Type	Prefix	Description
Kafka	`be.channel.kafka`	Used for Kafka properties that are applicable for the Kafka channel and all its destinations.
Kafka	`be.channel.kafka.<destination_name>`	Used for Kafka properties that are applicable for only the specified destination of the Kafka channel.

Support for Kafka Message Parameters

The Kafka message parameters, such as key and offset, can be accessed through the TIBCO BusinessEvents event properties. You can define event properties for each Kafka parameter and use them in rule or rulefunction. The deserializer fills the values in these event property fields when creating a TIBCO BusinessEvents event instance for a received Kafka message. For details, see Event Property and Rules and Functions.

Following properties can be added to events for Kafka message parameters.

Event Property	Description
`kafka_message_key`	Placeholder for a key of the Kafka message.
`kafka_message_offset`	Placeholder for an offset of the Kafka message.
`kafka_message_partition`	Placeholder for a partition number of the Kafka message.
`kafka_message_timestamp`	Placeholder for a time stamp of the Kafka message.
`kafka_message_timestamptype`	Placeholder for a type of the time stamp field of the Kafka message.

Subtopics

Did you find this helpful?

Yes No