Upgrading in a High Availability Environment

The process of upgrading your appliances in a High Availability (HA) appliance environment is different.

For details about HA, see the TIBCO LogLogic® Log Management Intelligence Administration Guide.

Prerequisites

The following prerequisites must be met before starting the upgrade process on your HA appliances.
  • Both HA appliances must have the same hardware model and software version.
  • Validate the disk requirements, as described in File Update.
  • Load and extract the upgrade files, as described in Updating the Appliance
  • The Active and Standby Appliances must be in sync. Ensure that there are no warning messages that appear on the Dashboards > System Status page.
Note: If the HA cluster was configured with the archive feature, disable it before the upgrade process.

The log data collected between Step 2 and Step 9 on appliance A is ignored at Step 17 during initialization and syncing.

Note: The terms "Active" and "Standby" might suggest that these appliances are not equivalent in every respect. In fact they must have identical capabilities in order for High Availability pairing to work. Therefore, for clarity during the upgrade process below, the original active appliance is referred to as “Appliance A” and the original standby appliance is referred to as “Appliance B” (even though their roles reverse and then revert). At step 10 in the upgrade process, Appliance B becomes the active appliance temporarily, and Appliance A becomes the standby appliance temporarily. At the conclusion of the upgrade process, the roles of the two appliances revert to their original assignments, although that is not necessary for successful HA operation.

This HA upgrade procedure uses the following example appliance information:

  • Appliance A: HA Active, IP address 10.20.0.44
  • Appliance B: HA Standby, IP address 10.20.0.45
  • Public: HA Public, IP address 10.20.0.46
  • Network Mask, IP address 255.255.255.0
  • Network Broadcast, IP address 10.20.0.255

Procedure

  1. Ensure that your HA environment is synchronized.
  2. On the Appliance B, disable the HA configuration.
    1. Open a command prompt and log in by using user name: root and type the password.
    2. Run the following command: 
      set failover disable
      The prompt returns CHANGES HAVE NOT BEEN SAVED!.
    3. Type save and then press enter.
  3. Install the correct LogLogic LSP version based on LogLogic LSP - LogLogic LMI support matrix for the target LogLogic LMI software.
  4. Make sure that the system is ready for upgrade by running the health check command healthcheck.sh with the correct path. For example: 
    /loglogic/update/healthcheck/healthcheck.sh
    Note:
    1. During this process, if system finds any inconsistencies, the console displays the message: Ignore the inconsistency?[yes/no]:Type no to stop the process and go back and fix the issue before you proceed. You can run the health check command multiple times until you fix all inconsistencies. If you type yes, all inconsistencies are ignored and you can proceed.
    2. During this process, the console might prompt the following message to warn you about unconverted report data from previous upgrade process: There is report data on the system that was not converted after the last upgrade. Do you want to convert this data now? [yes/no]:Type yes to exit the health check command at this point and run the rundbm command to complete the post-upgrade process from the previous upgrade. Otherwise, type no to ignore this warning and proceed. 

    3. During this process, the health check might fail with the following error, Schema consistency checking failed. 
If this occurs use the following workaround. In mysql: 
      use logappconfig
alter table oddsfieldtags drop FOREIGN KEY oddsfieldtags_ibfk_1;
alter table oddsfieldtags drop index msgPatternId;
alter table oddsfieldtags add index oddsfieldtags_ibfk_1(msgPatternId);
alter table oddsfieldtags add constraint oddsfieldtags_ibfk_1 FOREIGN KEY (msgPatternId) REFERENCES oddsmessagepattern(uuid) ON DELETE CASCADE;
    4. During this process, the health check might fail and the following error might be displayed: Data Vault is turned on but auto-unlock is not enabled.. If this occurs, use the system command system data_vault enable_auto_unlock to enable the automatic unlocking of the data vault.
    5. During this process, the health check might fail and the following warning might be displayed: WARNING: Advanced Features may not perform correctly with the current memory configuration on LMI 6.2.0 and above! Do you want to continue upgrade? [yes/no] This is because Advanced Features is enabled on the appliance but the configured memory is less than 32 GB. If this occurs, type yes to continue, or disable the Advanced Features, or configure more memory on the appliance.
    


  5. Make sure that the update files (bz2 and bz2.sig) are downloaded and are available in the /loglogic/update directory on the Appliance B. For details of the upgrade matrix, see TIBCO LogLogic® Log Management Intelligence Release Notes.

    The following files are examples of the update files you need to copy:

    • update-<timestamp>-full.tar.bz2
    • update-<timestamp>-full.tar.bz2.sig
  6. Log in to Appliance B and complete the update.
    1. Open a web browser and log in to the Appliance B.
    2. In the navigation menu, click Administration > File Update.
    3. The update-<timestamp>-full.tar.bz2 is displayed in the Select File box.
    4. Click the Update button. You might need to wait about 30 - 40 minutes for the update to complete.
  7. After you upgrade Appliance B and the appliance reboots, you must run the Post Upgrade Script. For post upgrade steps, follow the steps as described in Running the Post Upgrade Script.
  8. (Optional) If you want to update the SSL signature algorithm to sha2, run the script: /loglogic/tomcat/conf/cert_utils/certgen.sh.
  9. On the Appliance A, disable the HA configuration.
    1. Open a command prompt and log in by using user name: root and type the password.
    2. Run the following command: 
      set failover disable

      The prompt returns CHANGES HAVE NOT BEEN SAVED!

    3. Type save and then press enter.
  10. On Appliance B, set up the HA configuration.
    1. Open a command prompt and log in by using user name: root and type the password.
    2. Run the following command: 
      set failover configure
    3. Follow the prompts. Make sure you type save when prompted. The following example IP addresses are as noted at the beginning of this section:
      Public IP address of the cluster: 
 
      10.20.0.46 255.255.255.0 10.20.0.255 bond0

      When prompted about this appliance being the destination of automatic migration, enter N.

      IP address of the peer appliance: 
10.20.0.44

      At this point Appliance B has become the active appliance. Appliance A is no longer part of the HA pair, and thus is reported “missing” by Appliance B.
      Note: If the HA cluster was configured with the archive feature, disable it before the upgrade process.
  11. Install the correct LogLogic LSP version based on LogLogic LSP - LogLogic LMI support matrix for the target LogLogic LMI software.
    Note: Both Appliance A and Appliance B should now be on the same LogLogic LSP version.
  12. On Appliance A, make sure that the system is ready for upgrade by running the healthcheck command healthcheck.sh with the correct path. For example: /loglogic/update/healthcheck/healthcheck.sh
    Note:
    1. During this process, if system finds any inconsistencies, the console displays the message: Ignore the inconsistency?[yes/no]:Type no to stop the process and go back and fix the issue before you proceed. You can run the health check command multiple times until you fix all inconsistencies. If you type yes, all inconsistencies are ignored and you can proceed.
    2. During this process, the console might prompt the following message to warn you about unconverted report data from previous upgrade process: There is report data on the system that was not converted after the last upgrade. Do you want to convert this data now? [yes/no]:Type yes to exit the health check command at this point and run the rundbm command to complete the post-upgrade process from the previous upgrade. Otherwise, type no to ignore this warning and proceed. 

    3. During this process, the health check might fail with the following error, Schema consistency checking failed. 
If this occurs use the following workaround. In mysql: 
      use logappconfig
alter table oddsfieldtags drop FOREIGN KEY oddsfieldtags_ibfk_1;
alter table oddsfieldtags drop index msgPatternId;
alter table oddsfieldtags add index oddsfieldtags_ibfk_1(msgPatternId);
alter table oddsfieldtags add constraint oddsfieldtags_ibfk_1 FOREIGN KEY (msgPatternId) REFERENCES oddsmessagepattern(uuid) ON DELETE CASCADE;
    4. During this process, the health check might fail and the following error might be displayed: Data Vault is turned on but auto-unlock is not enabled.. If this occurs, use the system command system data_vault enable_auto_unlock to enable the automatic unlocking of the data vault.
    5. During this process, the health check might fail and the following warning might be displayed: WARNING: Advanced Features may not perform correctly with the current memory configuration on LMI 6.2.0 and above! Do you want to continue upgrade? [yes/no] This is because Advanced Features is enabled on the appliance but the configured memory is less than 32 GB. If this occurs, type yes to continue, or disable the Advanced Features, or configure more memory on the appliance.
    


  13. Make sure that the update files (bz2 and bz2.sig) are downloaded and are available in the /loglogic/update directory on the Appliance A. For details of the upgrade matrix, see TIBCO LogLogic® Log Management Intelligence Release Notes. The following files are examples of the update files that you need to copy:
    • update-<timestamp>-full.tar.bz2
    • update-<timestamp>-full.tar.bz2.sig
  14. Log in to Appliance A and complete the update.
    1. Open a web browser and log in to Appliance A.
    2. In the navigation menu, click Administration > File Update.
      The update-<timestamp>-full.tar.bz2 is displayed in the Select File box.
    3. Click Update.

      You might need to wait 30 - 40 minutes for the update to complete.

  15. After you upgrade Appliance A and the appliance reboots, you must run the Post Upgrade Script. For post upgrade steps, follow the steps as described in Running the Post Upgrade Script.
    Warning: You must let Appliance A complete the upgrade process before syncing up with Appliance B.
  16. (Optional) If you want to update the SSL signature algorithm to sha2, run the script: /loglogic/tomcat/conf/cert_utils/certgen.sh.
  17. On the Appliance A, set up the HA configuration.
    1. Open a command prompt and log in by using user name: root and type the password.
    2. Run the following command: 
      set failover configure
    3. Follow the prompts. Make sure you type save when prompted.
    The following example IP addresses are as noted at the beginning of this section:

    Public IP address of the cluster: 
 

    10.20.0.46 255.255.255.0 10.20.0.255 bond0
    When prompted about this appliance being the destination of automatic migration, enter Y.

    IP address of the peer appliance:
10.20.0.45

  18. If the HA cluster was configured with the archive feature and you disabled it before the upgrade process (in Step 9), re-enable it now.

Result

The upgrade is complete. Appliance B is active, and Appliance A is standby.
Warning:
  • Forcing failovers for the purpose of keeping one particular appliance “active” is not recommended, because it has no particular benefit and any failover event has the potential to cause some loss of data.
  • After the upgrade is successful, in HA mode, you can enable Advanced Features and Monthly Index only through the CLI on the master node.