Configuring Apache Kafka Connect

The bridge relies on the Apache Kafka Connect framework. You must configure the Apache Kafka Connect properties file.

Supply the properties file as the first argument in the Apache Kafka Connect command line. (For examples, see Running the Connectors, where the properties file appears as config/connect-standalone.properties.)

Procedure

  1. Configure the locations of the Apache Kafka brokers.
    Set the bootstrap.servers parameter to a list of host:port locations of Apache Kafka brokers.
    For example: 
    bootstrap.servers=localhost:9092
  2. Configure the key and value converters. For example, to convert between FTL messages and Avro messages: 
    value.converter=com.tibco.messaging.kafka.avro.AvroConverter
key.converter=com.tibco.messaging.kafka.avro.AvroConverter
    The following table describes the compatible converters.
    Converter Description
    com.tibco.messaging.kafka.avro.AvroConverter The Avro converter stores and retrieves Avro messages on disk at the Apache Kafka broker.

    Use this converter to bridge between FTL applications and Apache Kafka applications that use Avro messages. You can also use this converter to convert between Avro messages and Apache Kafka Connect in-memory representation.

    The Avro converter requires the FTL Realm service to be running (see step 3).

    org.apache.kafka.connect.storage.StringConverter The string converter stores and retrieves messages in JSON string representation on disk at the Apache Kafka broker.

    Use this converter to bridge between FTL applications and Apache Kafka applications that use messages formatted as JSON strings without attached schemas.

    org.apache.kafka.connect.json.JsonConverter The JSON converter stores and retrieves JSON messages, optionally with schemas attached.

    Use this converter to bridge between FTL applications and Apache Kafka applications that use JSON-formatted messages with schemas attached.

    To attach a schema to each message, see step 4.

  3. If you use the Avro converter, configure the locations of FTL realm servers (if running pre-6.x versions of TIBCO FTL) or the FTL Server (when running 6.x or later versions of TIBCO FTL) and the schema repository. (These parameters are not relevant to other converters.) For example, 
    value.converter.ftl.realmservers = http://localhost:8585
key.converter.ftl.realmservers = http://localhost:8585

key.converter.schema.registry.url = http://localhost:8081/schema/v1
value.converter.schema.registry.url = http://localhost:8081/schema/v1
  4. If you use the JSON converter, configure Apache Kafka Connect to store the schema with each converted message. (These parameters are not relevant to other converters.) Storing schemas with JSON messages enables interoperability with some 3rd-party sink connectors. For example: 
    value.converter.schemas.enable=true
key.converter.schemas.enable=true
  5. Configure the plugin path.
    Set the plugin.path parameter to a directory containing the bridge connector archive file, tibftl-kafka-connect-2.3.0.jar. For example: 
    plugin.path=TIBCO_HOME/akd/bridge/2.3/lib