Checkpoints of In-Memory Data

You can save in-memory data tables, and other in-memory objects (thesauri, custom character maps, and Learn Models) to a permanent storage device. The saved copies, or snapshots, can be easily restored at a later time. This feature is called checkpointing. Its primary purpose is to permit efficient recovery of the data in tables when the TIBCO Patterns server is restarted after a planned shutdown.

A typical usage scenario is to checkpoint all in memory objects after the initial load is complete. If you perform batch updates of a table, the table should be checkpointed after each batch update. A checkpoint is a snapshot of the object as it exists at the time of the checkpoint. No history of updates is kept. Thus, if updates on a table are performed after a checkpoint of the table, or if a thesaurus or Learn Model is replaced after it was last checkpointed, those updates are not reflected when the table, model, or thesaurus is restored from the checkpoint.

If a controlled shutdown of the TIBCO Patterns server is planned, all in-memory objects can be checkpointed before the planned shutdown. On restart, direct the server to restore all checkpointed objects. This method is far more efficient than reloading the content of a large table from the original source.

Here are some additional points to keep in mind about the checkpoint feature:

•

Checkpointing and restoring operations are not automatic, but are controlled through explicit commands.

•

Checkpointing does not provide a history of versions of a table. There is at most one checkpoint for a table – the latest performed. Again, even with the checkpointing feature, the TIBCO Patterns in-memory tables are not a substitute for a full-fledged DBMS or other persistent data store.

•

Checkpointing operations are safe. This means that if the system crashes during a checkpointing operation on a table, the previous checkpoint of that table remains intact (assuming, of course, the physical device on which the checkpoint is stored remains intact). It also means that if an operation such as a deletion or a renaming of a checkpointed table fails, the checkpoint of that table remains intact.

•

Checkpoint operations are very fast. They generally operate at or close to the maximum write speed of the device they are writing to. Checkpoints on small or mid-sized tables are generally non-intrusive. However, frequent checkpoints on large tables that have a combination of record update and queries in progress, can cause a significant drop in performance.

Checkpoints of very large tables (approximately 100 million records or more) can impact performance. Checkpoints of such large tables should only be done when the system is inactive.

•

Restoring a checkpointed table after a server restart is significantly slower than the checkpointing operation, and thus potentially more intrusive. However, the restore operation generally has no effect on operations performed on other tables.

•

Renaming a table renames any associated checkpoint file. For example, if a table named "table_A" is loaded, a checkpoint is performed and then the table is renamed to "table_B". A restore of "table_A" fails because there is no longer any such checkpoint, but a restore of "table_B" succeeds as the checkpoint is now named "table_B".

•

Deleting a table deletes all associated checkpoint files. Thus, if you delete a table that was checkpointed, you cannot recover it by doing a restore. Checkpointing is not intended as a backup mechanism, it is designed for reloading tables on a restart.

•

If a table uses a custom character map the character map must be loaded before the table is restored. If the character map has been checkpointed, instructing the server to restore all objects loads the checkpointed custom character maps before loading any table that uses it. Therefore, it is a good practice to checkpoint the custom character maps if tables are checkpointed and a restore all operation is used. If the application chooses to restore objects individually, it is up to the application to ensure that any custom character maps are restored before any table or thesaurus that uses it.

•

Checkpointed data is NOT encrypted. The checkpoint directory, and any files in the directory, must be protected from unauthorized access.