Deferred Collection (Collect Later)

The LATER mode indicates that the client can submit and update data to the Service Session. It does not collect the results, however, as another instance attaches to the Service Session to collect the results. None of the results are removed from the Service Session until it is destroyed by the collector. The LATER mode cannot be used from a Service proxy.

There are two reasons to use this method:

The architecture requires different applications for submission and results processing.

To recover from a failure in the application that embeds the Driver. Since results are not removed until the Session is destroyed, if the application undergoes a failure it can recollect the results when it restarts.

Deferred collection Services require that the submitting Driver call destroy on the Service to indicate that submission is complete. If you are using the submitting Driver in such a way that it exits after submitting the tasks and calling destroy, do not call System.exit or exit from the ServiceLifecycleHandler, as this stops the destroy message from getting to the Broker. Also note that if you are exiting the submitting Driver immediately, you must set DirectDataTransfer to false in the driver.properties file.

After creating a Service with the deferred collection, use ServiceFactory.getService() to retrieve results. After you collect all results, call destroy to indicate to the Broker:

•

that the instance has collected all outputs and

•

to destroy the Session

You can create multiple collectors, but keep in mind that if a collector calls destroy(), the Service is destroyed and no other collectors can finish collecting outputs.

The following is an example of how to use the LATER mode with recovery:

// Creates a new Session and submits requests to the Session
// @param serviceType The type
// @param methods The list of methods
// @param args The list of arguments
// @return The id of the Session
// @throws Exception on error.
private String submitService(String serviceType, String methods[], Object[][] args) throws Exception {
    // create the session as a Collection.LATER type
    Properties props = new Properties();
    props.setProperty(Options.COLLECTION_TYPE, Options.Collection.LATER);
    Service cs = ServiceFactory.getInstance().createService(serviceType,
        null, props, null);
    // Submit all requests. 
    // Note that the handler must be null because this Instance cannot collect.
    for (int i = 0; i < args.length; i++) {
        cs.submit(methods[i], args[i], null);
        }
    String id = cs.getId();
    // destroy to indicate that submission is complete, and to free local
    // resources
    cs.destroy();
    // save this ID to a file, for recovery purposes
    saveServiceForRecovery(id);
    return id;
}
// Starts collection of results from a Collection.LATER Session
// @param id The id of the session
// @param handler The invocation handler
// @throws Exception on error.
private void collectService(final String id, ServiceInvocationHandler handler) throws Exception {
    // create a handler that removes this Service ID from the list in the file
    // when it is finished
    ServiceLifecycleHandler slc = new ServiceLifecycleHandler() {
        public void destroyed() {
            removeServiceFromRecovery(id);
        }
        public void destroyed(ServiceException e) {
            removeServiceFromRecovery(id);
        }
    };
    // get an instance of the session, which starts collecting results
    Service cs = ServiceFactory.getInstance().getService(id, handler, slc);
    // set the Service to be destroyed when it finishes collecting all output
    cs.destroyWhenInactive();
}
// Runs a Service by first creating a Collection.LATER Session, submitting all
// requests, then getting the collection instance to collect the results.
// @param serviceType The type
// @param methods The list of methods
// @param args The list of arguments
// @param handler The invocation handler
// @throws Exception on error.
private void runService(String serviceType, String methods[], Object[][] args, ServiceInvocationHandler handler) throws Exception{
    String id = submitService(serviceType, methods, args);
    collectService(id, handler);
}
// Recovers from an application failure by starting collection of Sessions that 
// did not complete collection prior to failure
// @param handler The invocation handler
// @throws Exception on error.
//
private void recoverAll(ServiceInvocationHandler handler) throws Exception {
    String[] recovered = getAllRecoveryServices();
    for (int i = 0; i < recovered.length; i++) {
        collectService(recovered[i], handler);
    }
}

Deferred Collection Failover

While other Sessions manage failover by resubmitting all outstanding tasks to a Failover Broker, in this case the submitting Session has already been destroyed, and the Driver might even be offline. So, to handle failover in this case, you must do the following:

Procedure

Every standard Broker must have one Failover Broker associated with it.

Configure a shared filesystem that can be accessed from each Broker pair.

Go to Admin > System Admin > Manager Configuration > Services, and under the DataTransfer heading, change the DataTransfer Base Directory to the location of the shared file system for each pair. Each pair must have its own directory.

If the submitting Driver does not use an external web server and it disconnects after submission, disable DDT. The data is then served by the Broker from the shared file system.

Since the collecting Driver cannot auto-resubmit when the results cannot be collected from a Daemon that is offline, it’s recommended that DDT is also disabled on Engines.

The submitting and collecting Drivers must be allowed to log in to both Brokers in the pair.

If a Broker fails while the Service is running, the Failover Broker takes over by recreating the task queues. If the original Broker comes back up, the Failover relinquishes control and the original recreates the task queues.

When the collecting Driver starts to collect, if the original is down, it collects results from the Failover. If that then comes up, it migrates to the original and continues.

Did you find this helpful?

Yes No