Performance: Indexes on Disk

When persistent messages are stored on disk, message metadata (such as the message’s sequence number) and indexes over the messages (such as an index of messages in a particular queue) might be retained in memory or stored on disk. To store indexes and metadata on disk, enable the indexes on disk feature.

Enabling indexes on disk has the following advantages:

• Constant memory usage: The persistence service does not consume memory for each pending message. Storing additional messages does not increase memory usage.

• Constant recovery time: When the persistence service restarts, it does not need to read message data from disk. Recovery time is minimal, regardless of how many pending messages are stored.

In general, you must plan for scenarios where your consumers go offline for a time. Enabling indexes on disk is suitable for use cases that require large message backlogs to be stored by the server, because you do not need to provision extra memory for your servers, and you do not risk long recovery times if FTL server restarts.

As metadata and indexes must be stored on disk, some additional disk space is required. Enabling indexes on disk also comes at the cost of some performance, as the persistence service must read/write the indexes on disk in addition to the metadata.

The semantics of sync and async disk persistence are unchanged if the indexes on disk feature is enabled.

If excess memory is available on the server host, consider setting max.working.memory in the FTL server YAML configuration file. This allows the persistence service to allocate additional memory for write buffers or read caches, potentially improving performance. See Persistence Service Configuration Parameters.

For more information, see Disk-Based Persistence. For information about how to enable indexes on disk, see Cluster Details Panel.