Performance Benchmarking Criteria

The first step when measuring performance is to identify the Service Level Agreement. Performance targets are determined by user response time and message processing requirements.

Examples of performance requirements include:
  • Engine throughput or number of messages processed per second
  • Processing speed or average process instance duration and latency
  • Web response time. The response and request time
  • Resource utilization
  • Concurrent request, sleep time, registered and if applicable, the concurrent users

Defining the minimum, desired, and peak targets for each requirement helps identifying the type of data to collect and to evaluate the test results.

In addition to these normal load expectations, abnormal error-recovery scenarios under unusually high loads should also be considered. For example, the TIBCO ActiveMatrix BusinessWorks™ process might be receiving or polling messages from a TIBCO Enterprise Message Service queue, and the above targets reflect the normal flow of messages through the queue.

However, if communication to the TIBCO Enterprise Message Service server has been disrupted for an extended period of time, or if the ActiveMatrix BusinessWorks™ Engine shuts down, a much higher load may be experienced when communication is re-established or when the engine restarts.

These scenarios must be addressed when considering the engine performance under load, and to ensure that the throughput does not deteriorate below the target in such situations.

Business requirements also control the decision to use reliable or certified messaging. Certified messaging has an impact on performance.