Storage Volume Watermarks

Pertaining to NAS or SAN storage and LogLogic LMI local disk storage, three watermark thresholds in the /loglogic/conf/archive_config file determine how log storage is treated on these devices.

These three watermark thresholds are described in the following sections:

localVolHighWatermark

The localVolHighWatermark watermark pertains to the appliance's local volume. If the percentage of used local disk space (on the /loglogic partition) exceeds the specified watermark threshold, the appliance starts incrementally deleting the files with the earliest expiration date from the DAS to accommodate new data.

If everything is set to the same expiration policy (for example, 1 year), then the oldest file is purged. However, when an appliance is configured with multiple different retention policies, files with the earliest expiration date are purged first. For example, files that are set to expire in one week are deleted under emergency purging circumstances before files that expire after one year and 10 months.

Typically, this threshold is not reached when connected to a NAS/SAN, because another watermark, localVolHighArchiveWatermark, triggers the engine_archive to begin archiving files off the DAS and over to the NAS/SAN server, thus keeping the local DAS disk space relatively free.

However, if the NAS/SAN is not configured properly or if the network is not available, it is possible that the local logs are not archived (moved) and the local /loglogic partition can become full. Therefore this localVolHighWatermark watermark prevents an undesirable full partition by beginning the file deletion process once the watermark is reached.

This watermark value can be adjusted from 0% to 99%.

localVolHighArchiveWatermark

The localVolHighArchiveWatermark watermark is the other watermark threshold pertaining to the appliance local volume DAS. If the percentage of used local disk space (on the /loglogic partition) exceeds this specified threshold, the appliance starts incrementally archiving the oldest files to the specified NAS/SAN server. Although files are copied temporarily, they ultimately only exist on archive storage after all operations are performed for a given file, thus freeing up space on the DAS. This is considered an effective move operation because a copy operation is used to transfer data to the archive storage, followed by a delete operation to remove the local copies of the file. This ensures a safer data transfer than using a formal move command.

This watermark value can be adjusted from 2% to 94% with a default level of 75%.

Note: Immediate Archiving: Setting this watermark to a very low value (localVolHighArchiveWatermark=2) triggers near-immediate archiving when the NAS/SAN volume is successfully mounted. Although the lowest allowed value is 2%, the recommendation is minimum 5%, which can still provide near-immediate archiving capability after files are initially created on the local disk.

Setting the archiving threshold to a value below 5% might increase the chance of interference between the indexer and the archiver processes, and FileNotFoundException messages in indexer.log might be displayed. This is because, as soon as files are collected in the LogLogic LMI appliance, the archiving process starts moving files while the indexer is trying to access them. The lower the archiving threshold, the greater is the chance of interference.

nasVolLowWatermark

The nasVolLowWatermark watermark pertains to the NAS/SAN volume storage. If the available disk space on the NAS/SAN device goes under this specified threshold, the appliance starts incrementally deleting the files with the earliest expiration date on the NAS/SAN volume to accommodate new data.

This NAS/SAN Volume Low Watermark value can be adjusted from 10GB to 1024GB with a default level of 20 GB x the number of mounting points set (up to 32). It is specified in gigabytes instead of a percentage, because a percentage on a NAS/SAN can vary greatly in actual disk space. For example, if the user has 2 mounting points for NAS, the NAS Volume Low Watermark value is changed to 2x20 GB=40GB instead of only 20GB.

Warning: Up to 256TB of aggregate archive space is supported across up to 32 mount points. If you have already connected the appliance to a SAN or a NAS server and then want to change the threshold or the mount directory, you will no longer have access to your stored data.

By reverting the configuration to the old archive storage information, you can access the data again.

Warning: On the NAS server, you must create a directory as a file share for the appliance to mount, with rw, no_root_squash access permission.