Persistence Architecture

Storage, Replication, and Fault Tolerance

Persistence services manage stores and durable subscriptions (durables). A store holds messages until they are consumed. A persistent store is a store that is replicated to further ensure against loss of messages.

Stores hold message data typically in process memory to avoid the latency associated with disk I/O. However, with optional message swapping, if storage requirements exceed configured memory limits, excess messages are temporarily written to disk as needed. The use of message swapping can hedge against bursts. Memory threshold limits can be set on both a per-store and per-durable basis.

Replication of stores across a cluster of persistence services protects against hardware or network failures on a small scale. (However, this replication scheme cannot guarantee delivery after catastrophic failures.)

Latency

Using persistence for delivery assurance is consistent with high-speed message delivery with low latency. Delivery assurance operates alongside regular direct-path delivery. Transports carry messages directly from publishers to subscribers without an intermediary hop through a persistence service, which would add message latency. Separate transports carry messages from publishers to standard durables in a store in the persistence services, which retain them for as long as subscribers might need to recover them.

However, using persistence to apportion message streams or for last-value availability emphasizes throughput rather than the lowest latency. Delivery through durables replaces direct-path delivery. The persistence service is an intermediary hop, which adds message latency.

Meanwhile, a message broker emphasizes the convenience of a well-known pattern and minimal configuration, at the cost of added latency.

Wide-area stores involve the inherent latency of a WAN.

Publisher Quality of Service

For each store, administrators can balance appropriately between performance requirements and the need to confirm message replication.

Subscriber Acknowledgment

Within application programs, subscribers can acknowledge message delivery automatically or explicitly.

Administrators can configure durables to receive individual acknowledgments synchronously, or in asynchronous batches.

Durable Creation

Administrators can arrange for dynamic durables, which applications create as needed. Dynamic durables require minimal administrative configuration. Programmers take responsibility for the number of durables and their names.

Administrators can define static durables in the realm. Static durables require more administrative configuration and greater coordination between programmers and administrators. Administrators control the number of durables and their names.

Durables can be configured for Total Time to Live (TTL). Durables with a low TTL value are considered ephemeral durables.

Persistence Effectiveness

The flexibility of FTL persistence allows for various levels of persistence effectiveness, depending on factors such as the number of replicated stores, data limits store and persistence service hosting, and durable TTL. Persistence is generally considered adequately effective when stores are replicated and durables are non-ephemeral.