Engine and Job Tuning

The engine, job tuning, and checkpointing properties are specified in the config.ini file for each AppNode and alternatively, at the AppSpace level. The properties specified in the AppSpace config.ini file apply to all AppNodes associated with the AppSpace. However the properties specified in the AppNode config.ini file only apply to a specific AppNode and furthermore, they overwrite any property specified in the AppSpace config.ini file.

The BWEngine is a multi-threaded engine. When events that trigger the execution of a process occur concurrently, the engine runs the same process multiple times, concurrently, once for each event. Each process execution, referred to as a process instance, provides an execution scope for the activities that are a part of the process.

Execution of a component process is called a job. When the business logic is spread across multiple processes, a process instance is created for each of these processes and run with a particular event. Even though these are separate process instances that they are all working together and can be run as part of the same job.

A job can spawn multiple process instances and can provide the execution context for activities that are part of multiple processes. The engine always runs a job in one engine thread. However, it is not guaranteed that the same engine thread is used for the entirety of the job.

Engine Tuning

The rate at which the BWEngine can run and complete processes depends on the ThreadCount and StepCount engine properties.

Property Description

Property	Description
Thread count (bw.engine.threadCount)	The process instances in memory are run by the engine. The number of process instances that can be run concurrently by the engine is limited by the maximum number of threads, indicated by the ThreadCount property. This property specifies the size of the job thread pool and is applied to all the AppNodes in the AppSpace. Threads run a finite number of tasks or activities uninterrupted and then yield to the next process instance that is ready. Engine threads are shared by all the applications deployed on the same AppNode. The CPU and memory resources should be measured under a typical processing load to determine if the default ThreadCount is suitable for your environment. By default, the thread count is eight. For instructions on how to change the default value, see Setting Engine and Job Tuning Properties. If the engine throughput has reached a plateau, but the CPU and memory are not fully used, you can increase the thread count to have a positive effect on the throughput. Caution: If the engine thread count value is too high, it can cause CPU thrashing, or an increase in latency caused by a large number of messages in the queue. If the engine thread count value is too low, it can cause higher memory use and lower engine throughput as some CPU resources remain unutilized. The process instances created by the engine are typically held in memory. However, this may not be the case if the `FlowLimit` and `PageThreshold` application properties are set. The number of process instances that can be created in memory is also limited by the memory available on the machine and the memory allocated to the JVM on which the engine runs.
Step count (bw.engine.stepCount)	The engine `StepCount` property determines the number of activities that are run by an engine thread, without any interruption, before yielding the engine thread to another job that is ready in the job pool. This value is applied to all the AppNodes in the AppSpace. Exceptions to `StepCount` can occur when the job is in a transaction, is blocked, or is waiting for an asynchronous activity to complete. When a job is in a transaction, the thread is not released until the transaction is complete, even when the `StepCount` is exceeded. However, if a job is blocked or waiting for an asynchronous activity to complete, the thread can be yielded even when the `StepCount` has not been reached. The default value of this property is `-1`. When the values are set to `-1`, the engine can determine the necessary `StepCount` value. A low `StepCount` value can degrade engine performance due to frequent thread switches. A high `StepCount` value may cause less concurrence in running jobs and hence, result in an inefficient use of CPU.

Thread count

(bw.engine.threadCount)

The process instances in memory are run by the engine. The number of process instances that can be run concurrently by the engine is limited by the maximum number of threads, indicated by the ThreadCount property. This property specifies the size of the job thread pool and is applied to all the AppNodes in the AppSpace. Threads run a finite number of tasks or activities uninterrupted and then yield to the next process instance that is ready. Engine threads are shared by all the applications deployed on the same AppNode.

The CPU and memory resources should be measured under a typical processing load to determine if the default ThreadCount is suitable for your environment. By default, the thread count is eight.

For instructions on how to change the default value, see Setting Engine and Job Tuning Properties.

If the engine throughput has reached a plateau, but the CPU and memory are not fully used, you can increase the thread count to have a positive effect on the throughput.

Caution: If the engine thread count value is too high, it can cause CPU thrashing, or an increase in latency caused by a large number of messages in the queue. If the engine thread count value is too low, it can cause higher memory use and lower engine throughput as some CPU resources remain unutilized.

The process instances created by the engine are typically held in memory. However, this may not be the case if the FlowLimit and PageThreshold application properties are set. The number of process instances that can be created in memory is also limited by the memory available on the machine and the memory allocated to the JVM on which the engine runs.

Step count

(bw.engine.stepCount)

The engine StepCount property determines the number of activities that are run by an engine thread, without any interruption, before yielding the engine thread to another job that is ready in the job pool. This value is applied to all the AppNodes in the AppSpace.

Exceptions to StepCount can occur when the job is in a transaction, is blocked, or is waiting for an asynchronous activity to complete.

When a job is in a transaction, the thread is not released until the transaction is complete, even when the StepCount is exceeded. However, if a job is blocked or waiting for an asynchronous activity to complete, the thread can be yielded even when the StepCount has not been reached.

The default value of this property is -1. When the values are set to -1, the engine can determine the necessary StepCount value. A low StepCount value can degrade engine performance due to frequent thread switches. A high StepCount value may cause less concurrence in running jobs and hence, result in an inefficient use of CPU.

Job Tuning

Job tuning is done at the application level. Tuning can be narrowed to a specific application version, and a specific component within the application. Job tuning is set by the FlowLimit, PageThreshold, and Priority application properties. When setting these properties, specify the application name. The application version and component name are optional. If the version or component name is not specified, then the property value applies to all versions or components in the application. To push FlowLimit, PageThreshold, or Priority properties to runtime, stop the application, update the property in the AppNode or AppSpace config.ini file, and restart the application.

Property	Description
Flow limit bw.application.job.flowlimit	The Flow limit property specifies the Flow limit value for an application's process starters or service bindings and is applicable to all the AppNodes in an AppSpace. Flow limit is useful when the engine needs to be throttled as it specifies the maximum number of jobs that can be started before suspending the process starter. Thus ensuring that the incoming requests do not overwhelm the engine performance and the CPU and memory is preserved. If the number of jobs being created exceeds the Flow limit, the engine suspends the creation of new jobs but continues running the jobs in memory. The engine resumes creating jobs when sufficient resources are available. There is no default Flow limit value, and it is not enforced by the engine unless the Flow limit property is specified for an application. Note: Only set the bw.application.job.flowlimit property for applications using non-HTTP-based transports, for example, JMS. If applications are using HTTP-based transports, ensure you set the Max QTP Threads value of the HTTP Connector shared resource to apply the Flow limit. Get the logs for the component states such as Start, Stop, and Resume based on whether the `FlowLimit` is breached or complied by enabling the core runtime logger `com.tibco.bw.core` at INFO level. Note: Flow limit updated via the REST API resets in case of any application/appnode/appspace configuration changes.
Page Threshold (bw.application.job.pageThreshold)	The page threshold property specifies the job page threshold value for an application's process starters or service bindings and is applicable to all the AppNodes in an AppSpace. It specifies the maximum number of jobs that can concurrently be loaded into memory, thus limiting the number of running jobs in the memory. Jobs are paged out of memory on the basis of a paging strategy selected after the page threshold value is reached. The default value of the strategy is PageOnDelete. There is no default page threshold value and it is not enforced by the engine unless the PageThreshold property is specified for an application. The page threshold feature requires that the engine persistent mode property (bw.engine.persistenceMode) is set to datastore or group. For more information, see Engine Persistence Modes.
Paging Strategy (bw.application.job.paging.strategy)	The paging strategy property specifies how paging should take place. It supports two options: PageOnIdle and PageOnDelete. When you set the value of the property to PageOnIdle, and when the PageThreshold is reached, jobs that are idle are moved out of memory, and are paged-out to the engine database. The new or old jobs are loaded back into the memory in place of idle jobs when they are created or scheduled. When you set the value of the property to PageOnDelete, all new jobs created after the PageThreshold value is reached are temporarily paged out to the engine database. These jobs are moved back into the memory when the number of jobs in the memory is less than the PageThreshold value.
Priority (bw.application.job.priority)	The priority property specifies the application job's priority. The option for this property is one of low, normal, or high. The default value is normal. Engine threads process lower priority jobs only when higher priority jobs are all blocked from continuing. Lower priority jobs are not preempted while in execution.

You can update the Flow limit value dynamically without restarting an application from the Admin UI. Additionally, you can use the following REST API:

Base path for all REST APIs exposed is http://<host or IP address>:<port>/ where the port is of running AppNode.

bw/app.json/updateflowlimit/
Method	POST
Description	Update the Flow limit without restarting an application.
Path Parameters	None
Query Parameters	`parameter: flowLimit` `Type: Integer` (Mandatory) `Description:` The new value of the Flow limit. `parameter: name` `Type: String` `Description: Application name`. This property is mandatory for BW 6.x applications but it is optional when using for BWCE or TCI applications. `parameter: version` `Type: Integer` `Description: Application version`. This property is mandatory for BW 6.x applications but it is optional when using for BWCE or TCI applications. `parameter: component` `Type: String` (Optional) `Description: Component name of an application`.
For example	`http://<host or IP address>:<port>/bw/app.json/updateflowlimit?flowLimit=<new_flow_limit>&name=<app_name>.application&version=<app_version>&component=<component_name>`

bw/app.json/flowlimit/
Method	GET
Description	Get the latest Flow limit applied to the application or the component without restarting an application.
Path Parameters	None
Query Parameters	`parameter: name` `Type: String` `Description: Application name`. This property is mandatory for BW 6.x applications but it is optional when using for BWCE or TCI applications. `parameter: version` `Type: Integer` `Description: Application version`. This property is mandatory for BW 6.x applications but it is optional when using for BWCE or TCI applications. `parameter: component` `Type: String` (Optional) `Description: Component name of an application`.
For example	`http://<host or IP address>:<port>/bw/app.json/flowlimit?name=<app_name>.application&version=<app_version>&component=<component_name>`

Checkpointing

Property	Description
Retain faulted job bw.application.checkpoint.retainFaultedJob	This is an optional property and specifies whether to enable a failed process recovery. The supported values are `true` and `false`. The default value is `false`. The Application Name must be included as a part of this property, however, the Application Version is optional. After setting these properties in the AppNode config.ini file, they can be modified and pushed to runtime by restarting the application. When the property bw.application.checkpoint.retainFaultedJob is set to true for an application, the job is not automatically removed after a failure. So the duplicate key remains as long as the job remains. Such a job can be restarted or purged later.
Recover on restart bw.application.checkpoint.recoverOnRestart	This is an optional property and specifies whether the checkpointed process instances should automatically restart when a process engine restarts. The supported values are `true` and `false`. The default value is `true`. The Application Name must be included as part of this property, and the Application Version is optional.
Duplicate key timeout bw.application.checkpoint.dupKeyTimeout	This is an optional property and specifies the amount of time in minutes to keep duplicate keys stored after the checkpointed job finished running. By default, the timeout is 0 minutes, and indicates that the duplicate key is deleted as soon as the checkpointed job completes execution. The ActiveMatrix BusinessWorks application name must be included as part of this property. However the application version and component name are optional. If the component name is not specified, the value is applied to all components in the ActiveMatrix BusinessWorks application. bw.application.checkpoint.dupKeyTimeout.<UsersBWApplicationName> [.<UsersBWApplicationVersion>][.<UsersBWComponentName>]=0
bw.engine.checkpoint.expired.dupkey.purge.interval	You can also configure the periodic interval in which expired duplicate keys should be purged from the database by configuring the bw.engine.checkpoint.expired.dupkey.purge.interval property. It specifies the default interval for the background thread to poll for expired duplicate keys. The numerical value can be preceded either by "P" or "D". A value "P60" indicates the background thread polls after every 60 minutes whereas a value "D2" indicates the background thread polls after every 2 days. The default value is "P30".

Subtopics

Did you find this helpful?

Yes No