Kafka Connection Shared Resource

Kafka Connection resource is used to specify the configuration details of the Kafka server hosted across nodes. Typically, a producer would publish the messages to a specific topic hosted on a server node of a Kafka cluster and consumer can subscribe to any specific topic to fetch the data.

General

On the General tab, you can specify the required parameters before you use this activity. The General tab contains the following fields:

Field Description
Package Package path to be added
Name The name to be displayed as the label for the resource
Description A short description of the shared resource

Kafka Connection Resource Configuration

The Kafka Connection Configuration section has the following fields:

Field Literal Value/Module Property? Description
Bootstrap Servers Yes A list of host/port pairs to use for establishing the initial connection to the Kafka cluster.
Reconnect Backoff(msec) Yes This is the amount of time to wait before attempting to reconnect to a given host. This avoids repeatedly connecting to a host in a tight loop.

The default value is 50.

Retry Backoff(msec) Yes The amount of time to wait before attempting to retry a failed fetch request to a given topic partition. This avoids repeated fetching-and-failing in a tight loop.

The default value is 100.

Request Timeout(msec) Yes The maximum amount of time the client waits for the response of a request. If the response is not received before the timeout elapses the client resends the request if necessary or the request fails the retries are exhausted.

The default value is 30000.

Schema Registry

The Schema Registry section has the following fields:

Condition Applicable Field Literal Value/Module Property? Description
N/A Schema Registry Vendor No The vendor that provides schema management through schema registry.
There are two types of Schema Registry Vendors:
  • TIBCO Schema Registry
  • Confluent Schema Registry
Note: To use TIBCO Schema Registry, for an existing application module, right-click the project name in the Project Explorer view and select Configure > Convert to Java project. Add

tibftl-kafka-avro-<version>. jar

into your application module lib folder from

<TIBCO_HOME>\akd\repo\<version>\lib.

Available only when Schema Registry Vendor is selected. Schema Registry URL Yes The URL for the Schema Registry.
Note: For Tibco Schema Registry use the FTL realm URL.
Available only when Schema Registry Vendor is selected. Username Yes The username to access the Schema Registry
Available only when Schema Registry Vendor is selected. Password Yes The password to access the Schema Registry
Available only when Confluent Schema Registry is selected. SSL No This check box is available when Confluent Schema Registry is selected as the Schema Registry Vendor.

Select this check box to use the SSL authentication to verify the user and the schema registry server.

This flag enables or disables transport layer security for the connection.

Available only when SSL check box is selected. SSL Client Configuration No This field is available when the SSL checkbox is selected.

Select the SSL Client Shared Resource to setup the connection from the plug-in to the schema registry.

Available only when Schema Registry Vendor is selected. Test Connection No Click this button to validate the connection to the schema registry.

Security

Add required SSL properties in the server.properties file to enable SSL.

The Security section has the following fields:

Field Literal Value/Module Property Description
SSL Yes Select this check box to use the SSL authentication to verify the user and the server.
Keystore Provider as Trust Store None This field is used to create KeystoreProviderResource and then provide trust store URL and password. Available only when SSL check box is selected.
Enable Mutual Authentication None Select this check box to provide two-way SSL authentication. Available only when SSL check box is selected.
Identity Store Provider None This field is used to create KeystoreProviderResource and then provide key store URL and password. Available only when Enable Mutual Authentication check box is selected.
Key Password Yes Specify the key password. Available only when Enable Mutual Authentication check box is selected.

Authentication

In the Authentication section, you can select and configure an authentication type based on the level of security and encryption needed. The Kerberos security authentication can be combined with SSL to provide SASL_SSL.

Note: Before enabling Kerberos authentication, ensure that krb5.conf orkrb5.ini file (depending on your operating system) is placed in a default location. If you want to place the krb5.conf orkrb5.ini file in a custom location, configure java.security.krb5.conf system property. For more information, see "SECURITY" section in Apache Kafka Documentation.
Note: Kafka plug-in accepts "kafka" as a service principal name to connect with kerberized server. If the user creates a service name other than "kafka", the following error message is displayed:

Authentication failure

You can overwrite the Kerberos service name by adding the following property in Kafka producer/consumer "Advanced Properties" section:

sasl.kerberos.service.name=<service-name>.

The Authentication section has the following fields:

Condition Applicable Field Literal Value/Module Property Description
N/A Authentication Yes Select this check box to use the authentication to authenticate a user to connect to a server.
Available only when Authentication check box is selected. Authentication Type Yes There are following four types of authentication you can use:
  • Simple Username & Password (PLAIN)
Note: It is a good practice to use SSL with Simple Username & Password (PLAIN) type of authentication.
  • Modern Username & Password with Challenge (SCRAM-SHA-256)
  • Modern Username & Password with Challenge (SCRAM-SHA-512)
  • Kerberos (GSSAPI)
  • OAuthBearer
Available only when OAuth Bearer is selected in the Authentication Type field. JAAS Config File Yes Select the JAAS configuration file using the Resource Picker.

JAAS configuration example, 

KafkaClient {
                 org.apache.kafka.common.securit
y.oauthbearer.OauthBearerLoginModulerequired 
                 unsecuredLoginStringClaim_sub="
alice";
                     };
Available only when OAuth Bearer is selected in the Authentication Type field. Login Callback Handler Yes Select a class which provides implementation of "org.apache.kafka.common.security.auth.AuthenticateCallbackHandler" interface.

Please refer Kafka documentation for more details.

Not available when you select the authentication type as Kerberos (GSSAPI). User name Yes

Enter the user name for authentication.

Password Yes

Enter the password for authentication.

Available only when you select authentication type as Kerberos (GSSAPI). Use Ticket Cache Yes

Select this check box to use the ticket, which is present in the Kerberos cache. Else, select Keytab File and Principal Name options.

Note: Ensure that latest version of JCE Policy is installed, to use Use Ticket Cache option.
Keytab File Yes Select the Keytab file for authentication by using Resource Picker.
Principal Name Yes Fill the principal name such as admin@KAFKASECURE, depending on the environment setup.