Kafka Producer

Apache Kafka producer activity sends a record to a specified topic or channel in the Kafka cluster.

Settings

On the Settings tab, you can define the Apache Kafka connection and its details as given in the following table:

Condition Applicable Field Description
N/A Apache Kafka Connection Select the connection that you want to use from the dropdown list.
N/A Acks Mode Select one of the following acknowledgment modes from the dropdown list:
  • None: To receive no acknowledgment on record delivery
  • Leader: To receive an acknowledgment on record delivery from the leader
  • All: To receive acknowledgment on record delivery from leaders and all in-sync replicas
Applicable only when All is selected in the Ack Mode field. Ack Timeout The amount of waiting time in milliseconds to receive confirmation.
N/A Partitioner

Select one of the following partitioners from the dropdown list:

  • Hash: This is the default mode. When you select the Hash mode and the key field is not set on the Input tab, then a random partition is selected, otherwise, a key hash is used for calculating the partition to ensure all records with the same key are sent to the same partition.
  • Manual: When you select the Manual mode, the value configured in the partition field of the Input tab is used to select the partition.
Note: If the Manual mode is selected, then the partition field is mandatory on the Input tab.
N/A Compression Type Select a compression type: None, GZIP, or LZ4.
N/A Value Serializer Select the type of record value to be sent:
  • String
  • JSON
  • Avro
Applicable only when Avro is selected in the Value Serializer field. Subject

A list of all registered subjects in your Schema Registry. A subject refers to the name under which a schema is registered.

Select the subject to be used.

Applicable only when Avro is selected in the Value Serializer field. Version Version of the subject (registered name of schema) registered.

Select the version of the subject to be used.

N/A Max Request Size The maximum size of buffered records that can be sent in one request.

Default value: 1048576 bytes

N/A Max Messages The maximum number of records that can be sent in a single broker request.
N/A Frequency The frequency of sending buffered records in milliseconds.

Default value: 1000

Input Settings

Condition Applicable Field Description
N/A Headers

Header record to be sent. Only the String datatype value is supported.

Note: Headers are supported in the Apache Kafka version 0.11.0 and later.
Applicable only when JSON is selected in the Value Serializer field on the Settings tab. Schema for JSON value The JSON schema for the Apache Kafka record value.
Applicable only when Avro is selected in the Value Serializer field on the Settings tab. Schema for Avro Value

The Avro schema for the Apache Kafka record value. Depending on the Subject and Version selected, the schema is displayed here.

Note: This field is read-only if Use Schema Registry in the Apache Kafka Client Configuration dialog is set to True. Otherwise, you can provide the schema using this editor.

Input

Condition Applicable Field Description
N/A topic Name of the topic.
N/A partition Existing partition number where the record is to be sent.
N/A key Optional key value.
Applicable only when String is selected in the Value Serializer field. stringValue String value to be sent.
Applicable only when JSON is selected in the Value Serializer field. jsonValue Data structure based on the JSON schema that you have configured on the Input Settings tab.
Applicable only when Avro is selected in the Value Serializer field. avroData Data structure based on the Avro schema that you have configured on the Input Settings tab.
N/A headers Header value to be sent.

Output

Condition Applicable Field Description
N/A topic Name of the topic.
N/A partition Partition number of the record to send.
N/A offset Offset of the record.

Loop

For information on the Loop tab, see the Using the Loop Feature in an Activity section in the TIBCO Flogo® Enterprise documentation.

Retry on Error

Using the Retry on Error tab, you can set the number of times the flow tries to run the activity on encountering an error that can be fixed on retrial. The errors such as waiting for a server to start, intermittent connection failures, or connection timeout can be fixed on retrial.

You can set the count and the interval in one of the following ways:

  • Manually type the value in the mapper.

  • Map the value from the previous Activity.

  • Select a function from the list of functions.

  • Map app property to override the values.

Field Description
Count

The number of times the flow should attempt to run the activity. This value must be an integer.
Interval (in millisecond)

The time to wait in between each attempt to run the activity. This value must be an integer.
Note: The Count and Interval fields are mandatory. By default, the values are set to 0.