Kafka Connection Shared Resource

Kafka Connection resource is used to specify the configuration details of the Kafka server hosted across nodes. Typically, a producer would publish the messages to a specific topic hosted on a server node of a Kafka cluster and consumers can subscribe to any specific topic to fetch the data.

General

On the General tab, you can specify the required parameters before you use this activity. The General tab contains the following fields:

Field Description
Package Package path to be added
Name The name to be displayed as the label for the resource
Description A short description of the shared resource

Kafka Connection Configuration

The Kafka Connection Configuration section has the following fields:

Note: To set the application status to StartFailed when invalid credentials are detected, configure the JVM argument as follows:
com.tibco.bw.palette.kafka.runtime.validateCredentials=true
Field Literal Value/Module Property? Description
Bootstrap Servers Yes A list of host/port pairs to use for establishing the initial connection to the Kafka cluster.
Reconnect Backoff(msec) Yes This is the amount of time to wait before attempting to reconnect to a given host. This avoids repeatedly connecting to a host in a tight loop.

The default value is 50.

Retry Backoff(msec) Yes The amount of time to wait before attempting to retry a failed fetch request to a given topic partition. This avoids repeated fetching and failing in a tight loop.

The default value is 100.

Request Timeout(msec) Yes The maximum amount of time the client waits for the response to a request. If the response is not received before the timeout elapses the client resends the request if necessary or the request fails the retries are exhausted.

The default value is 30000.

Schema Registry

The Schema Registry section has the following fields:

Condition Applicable Field Literal Value/Module Property? Description
N/A Schema Registry Vendor No The vendor that provides schema management through a schema registry.

There are two types of Schema Registry Vendors:

  • TIBCO Schema Registry
  • Confluent Schema Registry

Available only when Schema Registry Vendor is selected. Schema Registry URL Yes The URL for the Schema Registry. For the Confluent Schema Registry, the list of registry server URLs is separated by a comma (,) and for the TIBCO Schema Registry, they are separated by a pipe (|)
Note: For TIBCO Schema Registry use the FTL realm URL.
Available only when Schema Registry Vendor is selected. Username Yes The username to access the Schema Registry
Available only when Schema Registry Vendor is selected. Password Yes The password to access the Schema Registry
Available for both Confluent Schema Registry and TIBCO Schema Registry. SSL No

Select this checkbox to use the SSL authentication to verify the user and the schema registry server.

This flag enables or disables the transport layer security for the connection.

Note: Only trust store configuration is used while connecting with the TIBCO Schema Registry. You can import the FTL trust certificate (PEM file) in the configured trust store.

Available only when the SSL checkbox is selected. SSL Client Configuration No This field is available when the SSL checkbox is selected.

Select the SSL Client Shared Resource to set up the connection from the plug-in to the schema registry.

Available only when Schema Registry Vendor is selected. Test Connection No Click this button to validate the connection to the schema registry.

Security

Add required SSL properties to the server.properties file to enable SSL.

The Security section has the following fields:

Field Literal Value/Module Property Description
SSL Yes Select this checkbox to use the SSL authentication to verify the user and the server.
Keystore Provider as Trust Store None This field is used to create KeystoreProviderResource and then provide the trust store URL and password. Available only when the SSL checkbox is selected.
Enable Mutual Authentication None Select this checkbox to provide two-way SSL authentication. Available only when the SSL checkbox is selected.
Identity Store Provider None

This field is used to create KeystoreProviderResource and then provide the key store URL and password.

Available only when the Enable Mutual

Authentication checkbox is selected.
Key Password Yes Specify the key password. Available only when the Enable Mutual Authentication checkbox is selected.

Authentication

In the Authentication section, you can select and configure an authentication type based on the level of security and encryption needed. You can combine the Kerberos security authentication with SSL to provide SASL_SSL.

Note: Before enabling Kerberos authentication, ensure that the krb5.conf orkrb5.ini file (depending on your operating system) is placed in a default location. If you want to place the krb5.conf orkrb5.ini file in a custom location, configure java.security.krb5.conf system property. For more information, see the "SECURITY" section in Apache Kafka Documentation.
Note: Kafka plug-in accepts "kafka" as a service principal name to connect with kerberized servers. If the user creates a service name other than "kafka", the following error message is displayed:

Authentication failure

You can overwrite the Kerberos service name by adding the following property in Kafka producer/consumer "Advanced Properties" section:

sasl.kerberos.service.name=<service-name>.

The Authentication section has the following fields:

Condition Applicable Field Literal Value/Module Property Description
N/A Authentication Yes Select this checkbox to use the authentication to authenticate a user to connect to a server.
Available only when the Authentication checkbox is selected. Authentication Type Yes There are following four types of authentication you can use:
  • Simple Username & Password (PLAIN)
Note: It is a good practice to use SSL with a Simple Username & Password (PLAIN) type of authentication.
  • Modern Username & Password with Challenge (SCRAM-SHA-256)
  • Modern Username & Password with Challenge (SCRAM-SHA-512)
  • Kerberos (GSSAPI)
  • OAuthBearer
Available only when OAuth Bearer is selected in the Authentication Type field. JAAS Config File Yes Select the JAAS configuration file using the Resource Picker.

JAAS configuration example, 

KafkaClient {
                 org.apache.kafka.common.securit
y.oauthbearer.OauthBearerLoginModulerequired 
                 unsecuredLoginStringClaim_sub="
alice";
                     };

Available only when OAuth Bearer is selected in the Authentication Type field. Login Callback Handler Yes Select a class that provides implementation of "org.apache.kafka.common.security.auth.AuthenticateCallbackHandler" interface.

Please refer to Kafka documentation for more details.

Not available when you select the authentication type as Kerberos (GSSAPI). User name Yes

Enter the user name for authentication.

Password Yes

Enter the password for authentication.

Available only when you select authentication type as Kerberos (GSSAPI). Use Ticket Cache Yes

Select this checkbox to use the ticket, which is present in the Kerberos cache. Else, select Keytab File and Principal Name options.

Note: Ensure that the latest version of JCE Policy is installed, to use the Use Ticket Cache option.
Keytab File Yes Select the Keytab file for authentication by using Resource Picker.
Principal Name Yes Fill in the principal name such as admin@KAFKASECURE, depending on the environment setup.