Kafka Connection Shared Resource

Kafka Connection resource is used to specify the configuration details of the Kafka server hosted across various nodes. Typically, a producer would publish the messages to a specific topic hosted on a server node of a Kafka cluster and consumer can subscribe to any specific topic to fetch the data.

General

On the General tab, you can specify the required parameters before you use this activity. The General tab contains the following fields:

Field Description
Package Package path to be added
Name The name to be displayed as the label for the resource
Description A short description of the shared resource

Kafka Connection Resource Configuration

The Kafka Connection Configuration section has the following fields:

Field Literal Value/Module Property? Description
Bootstrap Servers Yes A list of host/port pairs to use for establishing the initial connection to the Kafka cluster.
Reconnect Backoff(msec) Yes This is the amount of time to wait before attempting to reconnect to a given host. This avoids repeatedly connecting to a host in a tight loop.

The default value is 50.

Retry Backoff(msec) Yes The amount of time to wait before attempting to retry a failed fetch request to a given topic partition. This avoids repeated fetching-and-failing in a tight loop.

The default value is 100.

Request Timeout(msec) Yes The amount of time in milliseconds after which metadata is refreshed forcefully even though there are no changes in the partition leadership, to proactively discover any new brokers or partitions.

The default value is 40000.

SSL

Add required SSL properties in the server.properties file to enable SSL.

The SSL section has the following fields:

Field Literal Value/Module Property Description
SSL Yes Select this check box to use the SSL authentication to verify the user and the server.
Keystore Provider as Trust Store None This field is used to create KeystoreProviderResource and then provide trust store URL and password. Available only when SSL check box is selected.
Enable Mutual Authentication None Select this check box to provide two-way SSL authentication. Available only when SSL check box is selected.
Identity Store Provider None This field is used to create KeystoreProviderResource and then provide key store URL and password. Available only when Enable Mutual Authentication check box is selected.
Key Password Yes Specify the key password. Available only when Enable Mutual Authentication check box is selected.

Authentication

In the Authentication section, you can select and configure an authentication type based on the level of security and encryption needed. The Kerberos security authentication can be combined with SSL to provide SASL_SSL.

Note: Before enabling Kerberos authentication, ensure that krb5.conf orkrb5.ini file (depending on your operating system) is placed in a default location. If you want to place the krb5.conf orkrb5.ini file in a custom location, configure java.security.krb5.conf system property. For more information, see "SECURITY" section in the Apache Kafka Documentation.

The Authentication section has the following fields:

Condition Applicable Field Literal Value/Module Property Description
N/A Authentication Yes Select this check box to use the authentication to authenticate a user to connect to a server.
Available only when Authentication check box is selected. Authentication Type Yes There are following four types of authentication you can use:
  • Simple Username & Password (PLAIN)
Note: It is a good practice to use SSL with Simple Username & Password (PLAIN) type of authentication.
  • Modern Username & Password with Challenge (SCRAM-SHA-256)
  • Modern Username & Password with Challenge (SCRAM-SHA-512)
  • Kerberos (GSSAPI)
Not available when you select the authentication type as Kerberos (GSSAPI). User name Yes

Enter the user name for authentication.

Password Yes

Enter the password for authentication.

Available only when you select authentication type as Kerberos (GSSAPI). Use Ticket Cache Yes

Select this check box to use the ticket which is present in the Kerberos cache. Else, select Keytab File and Principal Name options.

Note: Ensure that latest version of JCE Policy is installed, to use Use Ticket Cache option.
Keytab File Yes Select the Keytab file for authentication by using Resource Picker.
Principal Name Yes Enter the principal name such as admin@KAFKASECURE, depending on the environment setup.