Engine and Job Tuning

The engine, job tuning and checkpointing properties are specified in the config.ini file for each AppNode and alternatively, at the AppSpace level. The properties specified in the AppSpace config.ini file apply to all AppNodes associated with the AppSpace; however the properties specified in the AppNode config.ini file only apply to a specific AppNode and furthermore, they overwrite any property specified in the AppSpace config.ini file.

The TIBCO ActiveMatrix BusinessWorks™ engine is a multi-threaded engine. When events that trigger the execution of a process occur concurrently, the engine executes the same process multiple times, concurrently, once for each event. Each process execution, referred to as a process instance, provides an execution scope for the activities that are a part of the process.

Execution of a component process is called a job. When the business logic is spread across multiple processes, a process instance is created for each of these processes and executed in conjunction with a particular event. Even though these are separate process instances they are all working together and can be executed as part of the same job. A job can spawn multiple process instances and can provide the execution context for activities that are part of multiple processes. The engine always executes a job in one engine thread; however, it is not guaranteed that the same engine thread will be used for the entirety of the job.

Engine Tuning

The rate at which the ActiveMatrix BusinessWorks™ engine can execute and complete processes depends on the ThreadCount and StepCount engine properties.

ThreadCount (bw.engine.threadCount): The process instances in memory are executed by the engine. The number of process instances that can be executed concurrently by the engine is limited by the maximum number of threads, indicated by the ThreadCount property. This property specifies the size of the job thread pool and is applied to all the AppNodes in the AppSpace. Threads execute a finite number of tasks or activities uninterrupted and then yield to the next process instance that is ready. Engine threads are shared by all the applications deployed on the same AppNode. The CPU and memory resources should be measured under a typical processing load to determine if the default ThreadCount is suitable for your environment. By default, the thread count is eight. See Setting Engine and Job Tuning Properties for instructions on how to change the default value.

If the engine throughput has reached a plateau, but the CPU and memory are not fully utilized, you can increase the thread count to have a positive effect on the throughput.
Caution: If the engine thread count value is too high, it can cause CPU thrashing, or an increase in latency caused by a large number of messages in the queue.If the engine thread count value is too low, it can cause higher memory use and lower engine throughput as some CPU resources remain unutilized.

The process instances created by the engine are typically held in memory. However, this may not be the case if the FlowLimit and PageThreshold application properties are set. The number of process instances that can be created in memory is also limited by the memory available on the machine and the memory allocated to the JVM on which the engine executes.

StepCount (bw.engine.stepCount): The engine StepCount property determines the number of activities that are executed by an engine thread, without any interruption, before yielding the engine thread to another job that is ready in the job pool. This value is applied to all the AppNodes in the AppSpace. Exceptions to StepCount can occur when the job is in a transaction, is blocked, or is waiting for an asynchronous activity to complete. When a job is in a transaction, the thread is not released until the transaction is complete, even when the StepCount is exceeded. However, if a job is blocked or waiting for an asynchronous activity to complete, the thread can be yielded even when the StepCount has not been reached.

The default value of this property is -1. When the values is set to -1, the engine can determine the necessary StepCount value. A low StepCount value can degrade engine performance due to frequent thread switches. A high StepCount value may cause less concurrency in executing jobs and hence, result in an inefficient use of CPU.

Job Tuning

Job tuning is done at the application level. Tuning can be narrowed to a specific application version, and a specific component within the application. Job tuning is set by the FlowLimit, PageThreshold, and Priority application properties. When setting these properties, specify the application name. The application version and component name are optional. If the version or component name is not specified, then the property value applies to all versions or components in the application. To dynamically push any of these properties to runtime, stop the application, update the property in the AppNode or AppSpace config.ini file, and restart the application.

Flow Limit (bw.application.job.flowlimit): The FlowLimit property specifies the flow limit value for an application's process starters or service bindings and is applicable to all the AppNodes in an AppSpace. Flow limit is useful when the engine needs to be throttled as it specifies the maximum number of jobs that can be started before suspending the process starter. Thus ensuring that the incoming requests do not overwhelm the engine performance and the CPU and memory is preserved. If the number of jobs being created exceeds the FlowLimit, the engine suspends the creation of new jobs but continues executing the jobs in memory. The engine resumes creating new jobs when sufficient resources are available. There is no default FlowLimit value and it is not enforced by the engine unless the FlowLimit property is specified for an application.
Note: Only set the bw.application.job.flowlimit property for applications using non-HTTP-based transports, for example, JMS. If applications are using HTTP-based transports, ensure you set the Max QTP Threads value of the HTTP Connector shared resource to apply Flow Limit.

PageThreshold (bw.application.job.pageThreshold): The PageThreshold property specifies the job page threshold value for an application's process starters or service bindings and is applicable to all the AppNodes in an AppSpace. It specifies the maximum number of jobs that can concurrently be loaded into memory, thus limiting the number of running jobs in the memory. Jobs are paged out of memory on the basis of paging strategy selected after the PageThreshold value is reached. The default value of strategy is PageOnDelete.

There is no default PageThreshold value and it is not enforced by the engine unless the PageThreshold property is specified for an application. The PageThreshold feature requires that the engine persistent mode property (bw.engine.persistenceMode) is set to datastore or group. For more information, see Engine Persistence Modes.

Paging Strategy (bw.application.job.paging.strategy): The Paging Strategy property specifies how paging should take place.

It supports two options: PageOnIdle and PageOnDelete.

When you set the value of the property to PageOnIdle, and when the PageThreshold is reached, jobs that are idle are moved out of memory, and are paged-out to the engine database. The new or old jobs are loaded back into the memory in place of idle jobs when they are created or scheduled.

When you set the value of the property to PageOnDelete, all new jobs created after the PageThreshold value is reached are temporarily paged out to the engine database. These jobs are moved back into the memory when the number of jobs in the memory is less than the PageThreshold value.

Priority (bw.application.job.priority): The Priority property specifies the application job's priority. The option for this property is one of low, normal, or high. The default value is normal. Engine threads process lower priority jobs only when higher priority jobs are all blocked from continuing. Lower priority jobs are not preempted while in execution.

Checkpointing

Note: To dynamically push any of these properties to runtime, stop the application, update the property in the AppNode or AppSpace config.ini file, and restart the application.
Retain Faulted Job (bw.application.checkpoint.retainFaultedJob) : This is an optional property and specifies whether to enable a failed process recovery. The supported values are true and false. The default value is false. The Application Name must be a included as a part of this property, however , the Application Version is optional. After setting these properties in the AppNode config.ini file, they can be modified and pushed to runtime by restarting the application.

Recover On Restart (bw.application.checkpoint.recoverOnRestart) :This is an optional property and specifies whether the checkpointed process instances should automatically restart when a process engine restarts. The supported values are true and false. The default value is true. The Application Name must be included as part of this property, and the Application Version is optional.

Duplicate key Timeout (bw.application.checkpoint.dupKeyTimeout): This is an optional property and specifies the amount of time in minutes to keep duplicate keys stored after the chekpointed job finished executing. By default, the timeout is 0 minutes, and indicates that the duplicate key is deleted as soon as the checkpointed job completes execution.

The ActiveMatrix BusinessWorks application name must be included as part of this property. However the application version and component name are optional. If the component name is not specified, the value is applied to all components in the ActiveMatrix BusinessWorks application. 

bw.application.checkpoint.dupKeyTimeout.<UsersBWApplicationName> [.<UsersBWApplicationVersion>][.<UsersBWComponentName>]=0

When the property bw.application.checkpoint.retainFaultedJob is set to true for an application, the job is not automatically removed after a failure. So the duplicateKey remains as long as the job remains. Such a job can be restarted or purged later.

You can also configure the periodic interval in which expired dupKeys should be purged from the database by configuring bw.engine.checkpoint.expired.dupkey.purge.interval property. It specifies the default interval for the background thread to poll for expired duplicate keys.

The numerical value can be preceded either by "P" or "D". A value "P60" indicates the background thread polls after every 60 minutes whereas a value "D2" indicates the background thread polls after every two days.

The default value is "P30".