Copyright © TIBCO Software Inc. All Rights Reserved
Copyright © TIBCO Software Inc. All Rights Reserved


Chapter 5 Running the EMS Server : Performance Tuning

Performance Tuning
By default, the TIBCO Enterprise Message Service server has the following general thread architecture:
Setting Thread Affinity for Increased Throughput
If the default behavior of the EMS server cannot provide the required throughput and the EMS server machine has multiple cores, you can assign specific cores to the EMS threads that handle network traffic and stores.
For instance, with a 4-core machine, you can use the processor_ids parameter to assign core 0 and core 1 to handle network traffic. You can also use the store configuration processor_id parameter to assign core 2 to handle the $sys.failsafe store. This configuration causes the EMS server to create two threads that handle network traffic, and sets the affinity of them to core 0 and core 1 respectively. It also sets the affinity of the thread handling the store $sys.failsafe to core 2. No affinity is set for other threads.
Determining Core Allocation
The phrase "less is more" summarizes the best practices for EMS performance tuning.
1.
When the EMS server does not set thread affinity, the operating system can better schedule EMS server threads to react to changing workloads on the machine. Also examine if the application is making efficient use of the API before changing the default behavior. For example, when performing persistent messaging operations, consider using multiple threads in the applications (each with its own session) or consider using local transactions to batch sends and acknowledgements.
2.
Use the minimum number of cores to handle network traffic. Binding a single core may yield sufficient performance improvements over the default behavior, so start testing affinity there. Using excessive numbers of cores leads to greater thread contention for global data structures, which can reduce throughput and waste machine resources. Excessive numbers can also lead to more unbalanced connection assignments. TIBCO tests have shown that three (or four under some workloads) is the maximum useful number for network traffic.
3.
When setting core assignment for network traffic for persistent messaging, also set core assignment for stores in order to prevent contention between threads handling those tasks.
Network I/O Connections
When a client connects to the EMS server, the EMS server assigns it to one of the threads handling network traffic based on which of those threads have the fewest existing connections. This balances the total number of connections evenly across those threads.
Note that if all the connections to one thread are closed, the EMS server does not move existing connections from other threads in order to rebalance them.
Also note that the EMS server does not account for the traffic generated by those connections. For instance, the EMS server could assign ten connections to one thread and ten connections to another thread but still have an unbalanced state if the first ten connections account for 90% of all network traffic to the EMS server.
Other Considerations
Assign cores on the same die if possible. This reduces cache sharing between dies. High levels of cache sharing between dies reduces memory performance.
Hyper-threads are not real cores. Disable hyper-threading if possible. Do not assign cores to the EMS server such that it sets affinity for two "cores" that are actually sharing the same physical core by hyper-threading.
 

Copyright © TIBCO Software Inc. All Rights Reserved
Copyright © TIBCO Software Inc. All Rights Reserved