Kafka Connection Shared Resource
Kafka Connection resource is used to specify the configuration details of the Kafka server hosted across nodes. Typically, a producer would publish the messages to a specific topic hosted on a server node of a Kafka cluster and consumers can subscribe to any specific topic to fetch the data.
General
On the General tab, you can specify the required parameters before you use this activity. The General tab contains the following fields:
Field | Description |
---|---|
Package | Package path to be added |
Name | The name to be displayed as the label for the resource |
Description | A short description of the shared resource |
Kafka Connection Configuration
The Kafka Connection Configuration section has the following fields:
com.tibco.bw.palette.kafka.runtime.validateCredentials=true
Field |
|
Description |
---|---|---|
Bootstrap Servers | Yes | A list of host/port pairs to use for establishing the initial connection to the Kafka cluster. |
Reconnect Backoff(msec) | Yes | This is the amount of time to wait before attempting to reconnect to a given host. This avoids repeatedly connecting to a host in a tight loop.
The default value is 50. |
Retry Backoff(msec) | Yes | The amount of time to wait before attempting to retry a failed fetch request to a given topic partition. This avoids repeated fetching and failing in a tight loop.
The default value is 100. |
Request Timeout(msec) | Yes | The maximum amount of time the client waits for the response to a request. If the response is not received before the timeout elapses the client resends the request if necessary or the request fails the retries are exhausted.
The default value is 30000. |
Schema Registry
The Schema Registry section has the following fields:
Condition Applicable | Field | Literal Value/Module Property? | Description |
---|---|---|---|
N/A | Schema Registry Vendor | No | The vendor that provides schema management through a schema registry.
There are two types of Schema Registry Vendors:
|
Available only when Schema Registry Vendor is selected. | Schema Registry URL | Yes | The URL for the Schema Registry. For the Confluent Schema Registry, the list of registry server URLs is separated by a comma (,) and for the TIBCO Schema Registry, they are separated by a pipe (|)
Note: For TIBCO Schema Registry use the FTL realm URL.
|
Available only when Schema Registry Vendor is selected. | Username | Yes | The username to access the Schema Registry |
Available only when Schema Registry Vendor is selected. | Password | Yes | The password to access the Schema Registry |
Available for both Confluent Schema Registry and TIBCO Schema Registry. | SSL | No |
Select this checkbox to use the SSL authentication to verify the user and the schema registry server. This flag enables or disables the transport layer security for the connection.
Note: Only trust store configuration is used while connecting with the TIBCO Schema Registry. You can import the FTL trust certificate (PEM file) in the configured trust store.
|
Available only when the SSL checkbox is selected. | SSL Client Configuration | No | This field is available when the SSL checkbox is selected.
Select the SSL Client Shared Resource to set up the connection from the plug-in to the schema registry. |
Available only when Schema Registry Vendor is selected. | Test Connection | No | Click this button to validate the connection to the schema registry. |
Security
Add required SSL properties to the server.properties file to enable SSL.
The Security section has the following fields:
Field | Literal Value/Module Property | Description |
---|---|---|
SSL | Yes | Select this checkbox to use the SSL authentication to verify the user and the server. |
Keystore Provider as Trust Store | None | This field is used to create KeystoreProviderResource and then provide the trust store URL and password. Available only when the SSL checkbox is selected. |
Enable Mutual Authentication | None | Select this checkbox to provide two-way SSL authentication. Available only when the SSL checkbox is selected. |
Identity Store Provider | None |
This field is used to create KeystoreProviderResource and then provide the key store URL and password. Available only when the Enable Mutual Authentication checkbox is selected. |
Key Password | Yes | Specify the key password. Available only when the Enable Mutual Authentication checkbox is selected. |
Authentication
In the Authentication section, you can select and configure an authentication type based on the level of security and encryption needed. You can combine the Kerberos security authentication with SSL to provide SASL_SSL.
Authentication failure
You can overwrite the Kerberos service name by adding the following property in Kafka producer/consumer "Advanced Properties" section:
sasl.kerberos.service.name=<service-name>
.
The Authentication section has the following fields:
Condition Applicable | Field | Literal Value/Module Property | Description |
---|---|---|---|
N/A | Authentication | Yes | Select this checkbox to use the authentication to authenticate a user to connect to a server. |
Available only when the Authentication checkbox is selected. | Authentication Type | Yes | There are following four types of authentication you can use:
Note: It is a good practice to use SSL with a Simple Username & Password (PLAIN) type of authentication.
|
Available only when OAuth Bearer is selected in the Authentication Type field. | JAAS Config File | Yes | Select the JAAS configuration file using the Resource Picker.
JAAS configuration example, KafkaClient { org.apache.kafka.common.securit y.oauthbearer.OauthBearerLoginModulerequired unsecuredLoginStringClaim_sub=" alice"; }; |
Available only when OAuth Bearer is selected in the Authentication Type field. | Login Callback Handler | Yes | Select a class that provides implementation of "org.apache.kafka.common.security.auth.AuthenticateCallbackHandler" interface.
Please refer to Kafka documentation for more details. |
Not available when you select the authentication type as Kerberos (GSSAPI). | User name | Yes |
Enter the user name for authentication. |
Password | Yes |
Enter the password for authentication. |
|
Available only when you select authentication type as Kerberos (GSSAPI). | Use Ticket Cache | Yes |
Select this checkbox to use the ticket, which is present in the Kerberos cache. Else, select Keytab File and Principal Name options. Note: Ensure that the latest version of JCE Policy is installed, to use the Use Ticket Cache option.
|
Keytab File | Yes | Select the Keytab file for authentication by using Resource Picker. | |
Principal Name | Yes | Fill in the principal name such as admin@KAFKASECURE, depending on the environment setup. |