Apache Pulsar Producer Activity

Apache Pulsar Producer activity can be used to map a string or a JSON object to a message. You can use the Producer activity to map the properties such as string:stringpairs and an optional key. You can send messages to a single topic and compress them by all the supported algorithms.

Settings

On the Settings tab, configure the following settings:

Field Required Description
Pulsar Connection Yes Name of the connection.
Topic Name Yes The name of the topic to which the message is published. The Topic Name field has the following format :persistent://public/default/foo

The first segment is the type of topic. The topic can be persistent or nonpersistent. With persistent topics, all the messages are persisted to the disk. The second segment is the name of the domain or tenant. The third segment is the namespace within the domain and the fourth segment is the name of the individual topic.

An abbreviated form of the topic name can be used only when the rightmost segment is provided. If the other segments of the topic name are needed then the complete topic name must be used.

Send Mode Yes

The message can be sent to the brokers using the Sync and Async mode.
Send Timeout No

Set the SendTimeout (in milliseconds) to specify how long to wait for the server to acknowledge the message. If the server doesn't respond within this time, the operation will fail with a SendTimeout error. To disable this feature, use -1. The default value is -1.
Compression Type No The following options are available for the compression type:
  • None
  • LZ4
  • ZLIB
  • ZSTD
Chunking No

Select the Chunking checkbox to enable the chunking feature.

Note: If you set Chunking to True, Batching automatically gets disabled.
ChunkMaxMessage Size Yes

The maximum chunk size of a message that can be sent in bytes.

Default: 5242880.
Batching No

Select the Batching checkbox to enable batching. The batching at the

producer happens only when the Send Mode is Async on the General tab.

Note: If you set Batching to True, Chunking automatically gets disabled.
Batching Max Messages Yes

Set the maximum number of messages that can be sent in a particular batch.

Default: 1000
Batching Max Size Yes

This specifies maximum number of bytes permitted in a batch.

Default: 128000
Batching Max Publish Delay Yes Specifies the maximum time in milliseconds within which the messages sent can be batched. Default value: 10 ms
Enable Replication No

Enables the replication for this message.
Replication Clusters No Provide the comma separated list of cluster names.

The producer created has a name in the following format that can be seen by using the Pulsar Admin REST API: <ApplicationName>-<AppVersion>-<FlowName>-<ActivityName>-<HostName>

Input Settings

The Input Settings tab has the following fields:

Field Required Description
Message Format Yes The following options are available for the message format:
  • String: A simple string can be mapped on the Input schema.
  • JSON: The Input schema is updated to reflect the structure of the JSON document and flow data can be mapped to the fields.
Message Properties No You can use the Message Properties field to add property value to the properties field presented on the Input schema. Each additional property is presented as a named string that is mapped.
Schema for JSON Value No You can see the Schema for JSON Value field only if the Message Format on the Input Settings tab is JSON. This is a free form text editor, which accepts any valid JSON document that is presented on the Input schema. This JSON document must also be used by the receiving app to decode the message.

Input

The Input tab has the following fields:

Field Description
payload The message to be published to a topic. Either a simple string or a representation of the JSON document provided on the Input Settings tab.
properties An object with a string value for each of the named properties from the Input Settings tab.
key A string value used by the topic compaction function of the broker. In the shared subscriber mode subscribers are bound to specific keys so that the repeat keys are always processed by the same consumer.

Output Settings

The Output Settings tab has the following field:

Field Description
msgid A string representing the message id.

Output

The Output tab displays the output schema of the activity as a tree structure. The output is read-only. The information in the schema varies based on the fields selected on the Settings tab. The properties that are displayed in the schema correspond to the output of this activity and can be used as input by subsequent activities in the flow.

Output Settings

The Output Settings tab has the following field:

Field Description
msgid A string representing the message id.

Using the Loop feature in an activity

If you want this activity to iterate multiple times within the flow, enter an expression that evaluates the iteration details. Select a type of iteration from the Type menu. The default type is None, which means the activity does not iterate. For more information, refer to the "Using the Loop Feature in an Activity" topic".

Using Retry on error feature in an activity

Using the Retry on Error tab, you can set the number of times the flow tries to run the activity on encountering an error that can be fixed on retrial. The errors such as waiting for a server to start, intermittent connection failures, or connection timeout can be fixed on retrial.

You can set the count and the interval in one of the following ways:

  • Manually type the value in the mapper.

  • Map the value from the previous Activity.

  • Select a function from the list of functions

  • Map app property to override the values

Field Description
Count The number of times the flow should attempt to run the activity. This value must be an integer.
Interval (in millisecond) The time to wait in between each attempt to run the activity. This value must be an integer.
Note: The Count and Interval fields are mandatory. By default, the values are set to 0.