Introduction : Explicit Confirmation Mode (ECM)

Explicit Confirmation Mode (ECM)
You can use TIBCO Adapter for Files z/OS to publish or subscribe to data using TIBCO Rendezvous RV (reliable), RVCM (certified messaging), or ECM (explicit confirmation mode). RV and RVCM modes are discussed in detail in the TIBCO Rendezvous documentation set.
ECM is available for communication between the Adapter and TIBCO ActiveMatrix Adapter for Files using RV, and between the Adapter and TIBCO Adapter for Files i5/OS.
This section discusses the following ECM-related topics:
Introduction to ECM
ECM provides FTP-like functionality with error handling and recovery. While RVCM supports certified messaging, it can be slow when handling large data sets. ECM provides a simplified confirmation mechanism that is suitable for handling large data sets and allows the Adapter to handle data sets of any size.
RVCM versus ECM
By default, the Adapter uses TIBCO Rendezvous Certified Messaging (RVCM) to guarantee message delivery. With RVCM, all messages are held in the publisher's ledger file until they are confirmed by all subscribers. While RVCM guarantees message delivery, its major drawback is that it requires significantly more disk space and memory. RVCM works well in environments that do not require high volume data rates. Its main strength is that it sends messages asynchronously and handles confirmations in the ledger without the application having to deal with it.
In block transfer mode, the Adapter must support high volumes of data (files of 200MB or more), and its performance in transferring files is critical. If RVCM were used to transfer such large data sets, it would require an impractical and unfeasible amount of system resources. Also the data that the Adapter is publishing is already held on disk in the input file, so storing it again in the RVCM ledger adds significantly more overhead.
The following differences between ECM and RVCM are notable:
With RVCM, since data is held in the ledger along with each subscriber's state, subscribers can confirm independently. However, with RVCM, slow consumers would grow FilePublisher's memory too much. In contrast, memory stays constant with ECM. The only trade-off is that all subscribers must confirm each block since the state is maintained per file type, not per subscriber.
Since FilePublisher is aware of when the confirmations arrive, it can send the next set of blocks as soon as all the confirmations arrive. This significantly improves the performance of Explicit Confirmation Mode, and it can operate at the speed of the subscriber. In that scenario, the transactionDelay option in this mode does not dictate when FilePublisher publishes. It only determines the time to detect that a confirmation has not arrived. This is the case when noWaitAfterConfirmations is set to true.
ECM Protocol Basics
The Explicit Confirmation Mode for block transfer requires that each subscriber confirm each block message. The publisher sends out a set of blocks containing file data, and it does not send the next set until it has received confirmations from each subscriber. If the publisher does not receive a block confirmation within a specified time limit, the publisher resends the block message (or messages). This mechanism ensures that the process memory does not grow based on how fast the subscribers can process the data. Also, because the block data is already persisted in the input file and the publisher (FilePublisher) maintains its state, guaranteed delivery and recovery are achieved.
_TIBCO_AE_ADAPTER_FAFT_SUBTOPUB.<endpoint subject name>
_TIBCO_AE_ADAPTER_FAFT_PUBTOSUB.<endpont subject name>
File Transfer Using ECM
ECM is well suited for transferring files without having to define schemas. Both publisher and subscriber must be set up to use ECM for successful transfer. The following two sections discuss in detail how publisher and subscriber interact. For information on setting up your adapter for ECM, see Setting FilePublisher for Explicit Confirmation Mode (ECM) and Setting FileSubscriber for Explicit Confirmation Mode.
Publication Service
A publication service in ECM performs the following tasks:
Upon starting for the first time, the publication service sends out an administration message to ECM subscribers prespecified in its configuration file. This process is initiated only when the publication service starts for the first time.
Upon restart, the active configuration information is retrieved from the progress file. Resynchronized handshakes are performed as needed to re-establish communication between the publisher and its subscribers.
When any inactive subscription service requests activation, the publication service marks the subscription service to be activated on a new file boundary.
Subscription Service
A subscription service in ECM performs the following tasks:
Upon starting the subscription service for the first time, the subscriber waits for an administration message from the publisher. Once the handshake has been established, the subscription service creates a progress file.
If no file transfer is active, the subscription service waits for an administration message from the publisher and starts receiving file transfers on a new file boundary.
ECM Mode Error/Restart Handling
Publication Service Error Handling
The publisher keeps track of the last block published in the progress file for restart purposes.
When the Adapter encounters an error, it publishes a message using the error subject established in the configuration file (ERROR_SUBJECT).
Subscription Service Error Handling
The subscriber keeps track of the last block of data received and written in the progress file for restart purposes.
When a subscription service detects an error (such as out of disk space) in the middle of a file transfer, the subscription service converts the working file to an error file.
When the Adapter encounters an error, it publishes a message using the error subject established in the configuration file (ERROR_SUBJECT).
Subjects Used by ECM
The ECM administration messages between the publication and subscription services are exchanged with the following subject names:
_TIBCO_AE_ADAPTER_FAFT_PUBTOSUB.<endpoint subject name>
_TIBCO_AE_ADAPTER_FAFT_SUBTOPUB.<endpont subject name>
Administrators can listen to these subjects at runtime.