Compact Disk Persistence Files with Persistence Service Online

With disk persistence enabled, a persistence service can compact its disk persistence files while running. There is no interruption to active publishers or subscribers.

Once started, compaction proceeds in the background until complete. In general, all persistence services in a given cluster will compact their disks at about the same time. A restart of the persistence service cancels any ongoing compaction.

Compaction Methods

You can compact disk persistence files with the following methods. Backup and online compaction cannot run at the same time.

Automatically Start Online Compaction

When automatic disk persistence compaction is enabled, the persistence services in the cluster automatically start a compaction when certain conditions are met. Compaction only runs when there is space to be reclaimed. If there is not enough disk space, the persistence service will not start compaction and will log a warning.

Automatic disk persistence compaction is set at the persistence cluster level in the realm configuration. Automatic compaction is enabled by default for newly created persistence clusters. The defaults follow. See Cluster Details Panel for more information.

  • disk_compact=true (Disk compaction is enabled.)

  • min_disk_inuse_ratio=0.05. (If the ratio of disk inuse size to disk allocated size falls below this value, the persistence service can start a compaction.)

Manually Start Online Compaction via the REST API

When the REST API compaction is manually invoked, all members of the persistence cluster start compaction. See POST persistence/clusters/<clus_name>, cmd: compactdisk.

You can invoke compaction for a cluster from the GUI, Persistence Clusters Status Table.

Compaction runs even if there is no space to be reclaimed. If disk persistence is not enabled, an error results. If there is not enough disk space, the persistence service will not start compaction and will log an error message.

The log reflects when compaction is complete. Compaction can also be monitored via the field compaction_in_progress from the REST API, GET persistence/clusters/<clus_name>/servers.

Other Considerations

The amount of disk that can be reclaimed can be estimated by comparing disk space in use and allocated disk space. The larger the difference between inuse size and allocated size, the more disk can be reclaimed. See Catalog of Persistence Metrics, disk size (3013) and disk in use size (3019).

For details on disk space during compaction, see Persistence Service Disk Capacity, section Disk Space for Backup or Compaction.

Compaction requires additional disk bandwidth, so some performance degradation is possible while compaction is running. A higher rate of live data, or a larger amount of disk space inuse, will increase the disk activity associated with compaction.

If the performance degradation due to compaction is not tolerable, you may want to disable automatic compaction in the realm configuration. If disabled, you can do one of the following: