Publisher Mode and Send Policy

There are two parameters that drive the behavior of publishers: the publisher mode and the send policy.

Administrators set the publisher's mode when configuring a store. The publisher mode is either store_confirm_send, which offers maximum delivery assurance, or store_send_noconfirm, which offers minimal delivery assurance. For details, see Publisher Send Mode.

Applications may set the publisher's send policy when creating a publisher object. In addition, administrators may set a default send policy in the realm configuration.

The send policy may be inline or non-inline. In general inline send offers better latency, while non-inline send offers better throughput.

When the publisher mode is store_send_noconfirm, the send call returns immediately, regardless of the send policy. The application is not notified as to whether the message was persisted or lost. In this case, the non-inline send policy may offer somewhat improved throughput in exchange for higher latency.

When the publisher mode is store_confirm_send, the FTL library will attempt to persist the message for the publisher's retry duration. The retry duration can be set by the application. Administrators may set a default value in the realm configuration. See the Administration guide, Realm Properties Details Panel, sectionRealm Properties Details Panel.

If the publisher mode is store_confirm_send, and the send policy is inline, then the send call will block until the message is persisted, or the retry duration has elapsed. If the FTL library could not ensure that the message was persisted, an exception is raised by the send call. This is a synchronous send call, where performance is tied directly to the latency to the persistence service. It may be necessary to use the batched send call (tibPublisher_SendMessages), or multiple publishers in parallel, to achieve acceptable performance.

If the publisher mode is store_confirm_send, and the send policy is non-inline, the send call may return immediately, provided that the maximum batch count has not yet been reached. The application may set the maximum batch count. See Configuring Publisher Send Policy.

When the send policy is non-inline, the FTL library attempts to persist the message in the background for the publisher's retry duration. If the latency to the persistence service is large, a non-inline send may offer a substantial improvement in throughput compared to inline send. However, failures are signaled to the application asynchronously, and handling failures may require a more sophisticated application.

If using a non-inline send, it is critical that the application flushes or closes the publisher before exiting. If the application merely exits, the send call may not be complete, and the message may be lost.

Configuring Publisher Send Policy

Send policy is configurable via the realm properties and at the API level. The API property set for inline or non-inline sends during publisher creation takes precedence over the realm property.

Dynamic update of the send policy realm property is allowed. See Realm Properties Details Panel. However it applies only to the newly created publishers.

Applications may override the send policy when creating a publisher object (TIB_PUBLISHER_PROPERTY_INT_SEND_POLICY).

When using non-inline send, the application may specify a maximum batch count (TIB_PUBLISHER_PROPERTY_INT_MAX_BATCH_COUNT) when creating the publisher. The maximum batch count is an upper limit on how many messages the FTL library may have that are not yet persisted (in other words, how many messages are in flight to the persistence service). If this limit is reached, the send call will block. This limit may also be used to determine which messages might have been lost in the event of a failure.

If using non-inline send, a situation may occur where an application closes a publisher shortly after making a send call. Since the send may not be complete, the FTL library will linger for some amount of time while the send is retried. If the message cannot be persisted within the linger period it may be lost. The application may override the publisher close linger period when creating the publisher (TIB_PUBLISHER_PROPERTY_DOUBLE_PERSISTENCE_CLOSE_LINGER).

FTL provides APIs that an application may use to determine the send policy and max batch count of a publisher object.

Detecting Failures with Non-Inline Send

When the send policy is non-inline, and a message cannot be persisted (perhaps because the retry duration has elapsed), the failure may occur in the background. The application will be notified, possibly after the fact, in one of two ways:

• An exception is generated by an ongoing or future send call

• An exception is generated by an ongoing or future flush call

However if the failure occurs during the publisher close linger period (for example, if a close call is made while the FTL library has messages that are not yet persisted), no exception is generated. Applications may wish to make a flush call before the close call.

When an exception is generated, the application must assume that up to maximum batch count messages might have been lost, in addition to the message that was passed to a failed send call. For example, if the maximum batch count is 100, and the send call fails, the application should assume that the current message and the previous 100 messages might have been lost. Applications may wish to keep information about the most recent messages for diagnostic or other purposes. Re-sending the most recent messages is possible, but may result in duplicates.

If the application does not make send or flush calls frequently, it may subscribe to an advisory for non-inline send loss (TIB_ADVISORY_NAME_NON_INLINE_SEND_LOSS). When this advisory is received, it is a hint to the application that an error has occurred in the background, and that a future flush or send call will generate an exception.