Configuring Kafka Connect

The bridge relies on the Kafka Connect framework. You must configure the Kafka Connect properties file.

Supply the properties file as the first argument in the Kafka Connect command line. (For examples, see Running the Connectors, where the properties file appears as config/connect-standalone.properties.)

Procedure

  1. Configure the locations of the Kafka brokers.
    Set the bootstrap.servers parameter to a list of host:port locations of Kafka brokers.
    For example: 
    bootstrap.servers=localhost:9092
  2. Configure the key and value converters. For example, to convert between FTL messages and Avro messages: 
    value.converter=com.tibco.messaging.kafka.avro.AvroConverter
key.converter=com.tibco.messaging.kafka.avro.AvroConverter
    The following table describes the compatible converters.
    Converter Description
    com.tibco.messaging.kafka.avro.AvroConverter The Avro converter stores and retrieves Avro messages on disk at the Kafka broker.

    Use this converter to bridge between FTL applications and Kafka applications that use Avro messages. You can also use this converter to convert between Avro messages and Kafka Connect in-memory representation.

    The Avro converter requires the realm server (see step 3).

    org.apache.kafka.connect.storage.StringConverter The string converter stores and retrieves messages in JSON string representation on disk at the Kafka broker.

    Use this converter to bridge between FTL applications and Kafka applications that use JSON strings without attached schemas.

    org.apache.kafka.connect.json.JsonConverter The JSON converter stores and retrieves JSON messages, optionally with schemas attached.

    Use this converter to bridge between FTL applications and Kafka applications that use JSON messages with schemas attached.

    To attach a schema to each message, see step 4.

  3. If you use the Avro converter, configure the locations of FTL realm servers and the schema repository.
    (These parameters are not relevant to other converters.)
    For example, 
    value.converter.ftl.realmservers = http://localhost:8080
key.converter.ftl.realmservers = http://localhost:8080

key.converter.schema.registry.url = http://localhost:8081/schema/v1
value.converter.schema.registry.url = http://localhost:8081/schema/v1
  4. If you use the JSON converter, configure Kafka Connect to store the schema with each converted message.
    (These parameters are not relevant to other converters.)
    Storing schemas with JSON messages enables interoperability with some 3rd-party sink connectors.
    For example, 
    value.converter.schemas.enable=true
key.converter.schemas.enable=true
  5. Configure the plugin path.
    Set the plugin.path parameter to a directory containing the bridge connector archive file, tibftl-kafka-connect-1.0.0.jar.
    For example, 
    plugin.path=TIBCO_HOME/akd/bridge/1.0/lib