Apache Kafka Schema Registry

You can use the Schema Registry to validate Kafka messages against a registered schema. Schema Registry ensures data consistency and increases data quality. Schema Registry supports schema versioning so that the different versions of the schema can be used simultaneously without causing compatibility issues. TIBCO BusinessEvents supports Confluent Schema Registry and TIBCO Schema Registry.

Configuring the Schema Registry

See Kafka Destination Configuration Properties for information about configuring the Schema Registry.

Subject Naming Strategies

Confluent Schema Registry

When the schema is registered, the name of the subject is decided by the subject name strategy. Subject name strategy can be set using Kafka destination properties. By default the subject name strategy is TopicNameStrategy. The subject name strategy can be configured by using the following property:

be.channel.kafka/Channels/<channelname>/<destinationname>.value.subject.name.strategy=<Subject Name Strategy Class>

You can set one of the following subject-named strategy classes:

• io.confluent.kafka.serializers.subject.TopicNameStrategy

• io.confluent.kafka.serializers.subject.RecordNameStrategy

• io.confluent.kafka.serializers.subject.TopicRecordNameStrategy

TIBCO Schema Registry

For the TIBCO Schema Registry, you can set the subject name by using the property value.subject.name with a few "%" template variables. The default value for this property is %t-value.

The application automatically replaces the special characters %t, %r, and %% with topic, name of schema, and literal % character respectively.

You can set the value for the value.subject.name as TIBCO BusinessEvents property for each destination.

Example:

be.channel.kafka/Channels/KafkaChannel/orderDest.value.subject.name=%r-%t-value

The preceding property configures value.subject.name for "orderDest" destination. The subject name is "RecordName-TopicName-Value".

See Apache Avro Client Library Configuration Options for more information.

Schema Compatibility

When schemas evolve and change, you must add (register) different schema versions. Before adding a version of a schema, the Schema Registry checks whether the new schema is compatible with the previously registered schema under the subject. This compatibility is managed through the Schema Registry compatibility parameter. See Confluent documentation for more information about setting schema compatibility.

Schema Registration Scenarios

Refer to the following table for different scenarios about how schema is registered in the Schema Registry based on whether Subject and Schema fields are specified or not.

Subject field specified?	Schema field specified?	How Schema is registered?
Yes	Yes	If the schema subject is valid, TIBCO BusinessEvents fetches the latest schema from the Schema Registry associated with the subject. If the subject name is not as per the subject name strategy, then TIBCO BusinessEvents registers the latest schema under subject that is as per the subject name strategy. If the schema subject is not valid, TIBCO BusinessEvents registers the schema configured in the Kafka destination properties with the subject name as per the subject name strategy. If the schema is already registered, then TIBCO BusinessEvents updates the schema in the Schema Registry.
No	Yes	Registers the schema configured in the Kafka destination properties. If the schema is already registered, then TIBCO BusinessEvents updates the schema in the Schema Registry.
Yes	No	If the schema subject is valid, TIBCO BusinessEvents fetches the latest schema from the Schema Registry associated with the subject. If an associated subject name is not as per the subject name strategy, then TIBCO BusinessEvents registers the latest schema under subject that is as per the subject name strategy. If schema subject is not valid, TIBCO BusinessEvents creates schema using event properties and payload associated with the destination and registers it as per the subject name strategy
No	No	If schema subject is not valid, TIBCO BusinessEvents creates schema by using event properties and payload associated with the destination and registers it as per the subject name strategy.

Top