Removing Copysets

The Remove Copyset feature enhances the ActiveSpaces data grid scalability by enabling scaling down of resources.

Procedure
1. Initiate deactivation of a copyset by running the following command:

   tibdg copyset deactivate <copyset_name>

   This command marks the copyset as inactive, but does not begin redistribution of data to other copysets. It enables more efficient data redistribution when you are removing multiple copysets.

2. Redistribute data by running the following command:

   tibdg grid redistribute

   When redistribution starts, any copysets marked as deactivated transfer data to the remaining active copysets. To determine when redistribution is complete, check the tibdg status.

3. When the status indicates there are no redistributions in progress, remove the copyset from the grid configuration by running the following command:

   tibdg copyset remove <copyset_name>

   When a copyset is removed, all tibdgnode processes associated with that copyset exit. When the processes have stopped, their data directories can be removed. If the data redistribution fails, this command fails. Removing a copyset that still owns data would result in immediate data loss, so the copyset remove command returns failure until the copyset can be safely removed.

   Note: If the grid is currently using checkpoints (such as, Live Backup and Restore or DR/Mirroring), take a new checkpoint immediately.

Checkpoints and Copyset Removal

When data is being redistributed, checkpoint data is not moved with the live data. This means that removing copysets might remove access to some checkpoints. Checkpoints that are not available due to copyset removal are displayed with an exclamation mark (!) in the output of the tibdg checkpoint list command. If using checkpoints, it is necessary to take a new checkpoint after redistribution completes to provide a target for the latest checkpoint name. After removing a copyset, all recent (since the last redistribution) checkpoints are marked as unavailable and cannot be used for checkpoint reads, checkpoint queries, or DR mirroring.

$tibdg -r http://myhost:8080 checkpoint list
                                    ID                 NAME                                                   DIRECTORY                       TIMESTAMP   STATUS  NOTES
  607F019E-56C1-2400-9845-7647EC31E14D  only_original_nodes  20250115T223824.452Z_00000000_00000001_only_original_nodes  2025-01-15 22:38:24.452383000Z  Success       
  93B69141-F773-3100-6D0C-F1B7A43BB817     with_set3_inst_0     20250115T223842.124Z_00000000_00000002_with_set3_inst_0  2025-01-15 22:38:42.124752000Z  Success      !
  AC1CA7B1-4C9C-EC00-C6D3-DAC3AC364F29    after_set3_inst_0    20250115T223902.937Z_00000000_00000003_after_set3_inst_0  2025-01-15 22:39:02.937931000Z  Success       
  98C53D7E-907D-7F00-79C0-386DD8E790EE     with_set3_inst_1     20250115T223950.214Z_00000000_00000004_with_set3_inst_1  2025-01-15 22:39:50.214332000Z  Success      !
  DC4E8BF6-A499-5600-58C7-34FC7BC5CA95    after_set3_inst_1    20250115T224013.527Z_00000000_00000005_after_set3_inst_1  2025-01-15 22:40:13.527030000Z  Success       
  93A3B9E9-F2E4-0C00-28D6-160EFE2BF2EC     with_set3_inst_2     20250115T224102.214Z_00000000_00000006_with_set3_inst_2  2025-01-15 22:41:02.214492000Z  Success      !
  B75E0AB0-E80E-3000-BE50-CD69907ADDBE    after_set3_inst_2    20250115T224124.836Z_00000000_00000007_after_set3_inst_2  2025-01-15 22:41:24.836348000Z  Success       

! - No Longer Available because a required copyset is not present in the grid config.

To use this information programmatically, specifying the -j flag provides the same information in JSON format.

Similarly, attempting to verify a checkpoint that requires removed copysets fails:

$tibdg checkpoint validate with_set3_inst_2
Checkpoint with_set3_inst_2 (epoch: 0, sqn: 6) failed validation on the following copysets:
        <removed> (E9F1FA52-F50D-4F7E-BC67-5BFF791F3AAE)
*** ERROR OCCURRED OR CMD TIMED OUT RC = 1***