Understanding mstore Intervals

The mstore is designed to ensure a quick EMS server start-up time. To enable this functionality, the EMS server must continually monitor the store in the background. The server reads through the mstore incrementally and discards stale data, such as purged and expired messages.

In order to keep the background activity from degrading server performance, the examination is performed in increments. The length of these increments and the amount of data processed each increment are controlled by two parameter settings. These stores.conf parameters can be configured for each mstore.

The default parameter settings are optimized for best performance in most production environments (see mstore Parameters for information about the default values). However, if the amount of data in the mstore grows significantly, the read rates associated with the background activity may begin to affect message transmission rates in the EMS server. If the EMS server performance is negatively affected by the size of the mstore, you can tune the mstore parameter values to spread mstore background activity over a longer period of time, thereby decreasing the associated read rates.

  • scan_target_interval — the maximum amount of time allowed before each message in the store is examined.

    For example, if the scan_target_interval is 24 hours, each section of the mstore will be examined at least once every day. Because purged and expired messages are not removed from the mstore until they are examined by this background process, this means that it can take up to 24 hours before a message is removed from the queue following a purge command (making underlying storage space available for re-use).

  • scan_iter_interval — the length of time between each increment of background activity.

    For example, if the scan_iter_interval is 10 seconds, the EMS server begins examining a new section of the mstore every 10 seconds. The amount of data read in each increment is dependent on the total size of the store and the length of the scan_target_interval. The server must examine enough data in each interval to fully traverse the store within the target interval.

Example

For example, assume that scan_iter_interval is 10 seconds, scan_target_interval is 1 day (86,400 seconds), and the mstore contains 9 GBs of data. Every 10 seconds, the EMS server will examine about 1 MB of data. This produces an average read rate of about 100 KB/sec, which is unlikely to produce performance degradation with most modern storage mediums.

If EMS server performance does slow, you may need to increase the scan_target_interval value in order to spread the background activity over longer period of time. You can monitor the settings for problems using the show store command and checking the ratio of "Discard scan bytes" to "Discard scan interval". For best results, this ratio should be kept below 20% of the disk processing capacity for each mstore. Consider this ratio in relation to your storage medium's overall data transfer capacity, so as to make sure that the background activity does not occupy an excessive amount of the system's resources.

Implications for Statistics

The background monitoring and cleanup that occurs in the mstore also affect some key server statistics. Before the first scan has been completed, some message statistics may be underreported due to purged and expired messages that the server has not yet removed. Until the first background scan is complete for some or all mstores, the server may not have an accurate messages count.

For example, when the EMS server first starts, the "Pending Messages" and "Pending Message Size" counts reported by the info command in the administration tool can be understated, because the command only reports on messages it has scanned before the command is issued. Similarly, the "Message Count" and "Message Size" reported by the show store command may report a smaller number than actually exist in the store.

Once the first scan is complete, these counts can be considered accurate. To check the scan status on an mstore, use the show store command. The statistics returned now include a "First scan finished" field, which reports the scan status since the last EMS server start time. When the value of this field is true, the server statistics can be considered accurate.

If it is important to acquire the correct values for these statistics sooner, you will need to decrease the scan_target_interval.