Backing Up and Restoring Kubernetes Cluster on AWS S3 using Velero

Velero is an open-source tool. It can be run with a Cloud Provider or on premises software. You can use Velero to back-up and restore your Kubernetes cluster resources and also replicate your production cluster to development and testing clusters.

To take a backup and restore your Kubernetes cluster, perform the following steps:

    Procedure
  1. Install Velero

  2. Create a WebFOCUS backup

  3. Restore a WebFOCUS backup

  4. Add multiple (S3 bucket) back-up locations

  5. Deploy Velero in EKS using IAM role

  6. Deploy Velero using the helm chart

Install Velero

    Procedure
  1. To install the Velero command-line interface on your system, use the following commands:

    1. Set the latest version of velero in VELERO_VERSION.

      VELERO_VERSION=v1.10.0
      Note: To check the latest version, see Velero Version
    2. Download the .tar file using the wget command.

      Caution: Code snippets in the PDF could have undesired line breaks due to space constraints and should be verified before directly copying and running them in your program
      wget 
      https://github.com/vmware-tanzu/velero/releases/download/$VELERO_VERSION/velero-$VELERO_VERSION-linux-amd64.tar.gz
    3. Extract the downloaded file using the tar command.

      tar -xvf velero-$VELERO_VERSION-linux-amd64.tar.gz
    4. Set the file permissions using the chmod command.

      chmod +x velero-$VELERO_VERSION-linux-amd64/velero
    5. Use the cp command to copy the file to your given location.

      Caution: Code snippets in the PDF could have undesired line breaks due to space constraints and should be verified before directly copying and running them in your program
      sudo cp velero-$VELERO_VERSION-linux-amd64/velero /usr/local/bin
  2. Create the "s3-credentials" file and update aws_access_key_id, aws_secret_access_key, aws_session_token with valid parameter, refer the below s3-credentials sample file.

    [default]
    aws_access_key_id=ASIA2TQTLBEVZQHU2WV2
    aws_secret_access_key=+K9EHu+y5XXmlPBVoBsvPTfVjwiMUHdDKjjY+/cZ
    aws_session_token=FwoGZXIvYXdzEGgaDAd3IszIO1V7X2yenCK0ASb9s0ysMBeJO6
    xbgg3LojCr5Xmmk9yvrZioyh0RvPM4BFwSMqjZItIDpTGlcAJK357TkbhI1CCG3IZtuC
    UqKLaZhfNQ7NQOzHWKtBU5CGx75uI1NQLwLenGwK6evehlK5igMUmYdaLxVS28sMVDYt
    qhS09BeLQB6TAqmxUf2K2ByUMjlG2h+DQ+ZjzFwsp3+pWFob+iNC62IlnyFC6K+PP5AW
    9dL0Zld64+wLWvAFNty5SpuCiKtbOpBjItvDF/AHBzRg4ci3tZiytoG4p5qCWJMXD3jO
    CPbpWWzSrGMSmjvkVVrXOBUI4u
  3. To deploy Velero with Kopia, use the following command:

    • Create the S3 bucket on AWS console with the appropriate region. Set the BUCKET name and REGION using the following command.

      BUCKET=<bucket_name>
      REGION=<aws_region_name>
    • Use the created "s3-credential" file and follow the following command:

      velero install \
      --provider aws \
      --plugins velero/velero-plugin-for-aws \
      --bucket $BUCKET \
      --backup-location-config region=$REGION \
      --snapshot-location-config region=$REGION \
      --secret-file ./s3-credentials \
      --default-volumes-to-fs-backup \
      --uploader-type kopia \
      --use-node-agent
      Tip: Kopia is a quick and safe open-source backup and restore application that enables you to make encrypted copies of your data, and save the images to local disk, network-attached storage, or a server of your choosing, as well as remote or cloud storage of your choice.

      Kopia works well with the efs-sc storage class on AWS, but it does not support the socket and pipe file types. To do this, we create a .kopiaignore file in the clm pod "/opt/ibi/srv/temp". Use the following command:

      kubectl -n webfocus exec -it clm-0 -- /bin/bash
      echo ".tscom300">/opt/ibi/srv/temp/.kopiaignore
      exit

Create a WebFOCUS backup

  • Use the backup command to create the webfocus backup.

    velero backup create <backup_name> --include-namespaces <namespace> --wait
  • Use the describe command to check the details for specific webfocus backup.

    velero describe backup <backup_name> --details

Restore a WebFOCUS backup

  • Use the restore command, to get back the webfocus backup.

    velero restore create --from-backup <backup_name>
  • Velero can restore resources into different namespaces using the "--namespace-mappings" flag. Use the following command.

    velero restore create <restore_name> \
    --from-backup <backup_name> \
    --namespace-mappings <old-namespace>:<new-namespace>
    • After namespace mapping, run the restore command.

    • After restoring the wfce resources, update the namespace name from Appserver UI.

      Then go to Management Center → Administration Console → Reporting Servers → Cluster Manager → EDASERVE and update the namespace for Remote CLM Host location. For example: clm.new-namespace.

    • Lastly, redeploy the failing "prom-adapter-prometheus-adapter" pod.

Note: Velero backup and restore does not work as expected, if you are deploying WebFOCUS - Container Edition using infra Postgresql database. It is must use the Velero backup and restore with Amazon RDS or external databses only.

Add multiple (S3 bucket) back-up locations

    Procedure
  1. You can change the default back-up storage location at any time by setting the "--default" flag using the velero backup-location set command and configure a different location to be the default. For example, refer to the below command.

    velero backup-location create backups-primary \
    --provider aws \
    --bucket <bucket_1> \
    --config region=$REGION

    velero backup-location create backups-secondary \
    --provider aws \
    --bucket <bucket_2> \
    --config region=$REGION
  2. Use the below command to set the S3 bucket back-up location.

    velero backup-location set backups-secondary --default
  3. Use the below command to check the default back-up location.

    velero get backup-location

Deploy Velero in EKS using IAM role

To deploy velero in EKS using the IAM role, refer to the following link: Backup and restore your Amazon EKS cluster resources using Velero.

Deploy Velero using the helm chart

To deploy velero using a helm chart, refer to the following link: Velero using helm chart.

Useful flags for Velero

If no filtering options are used, Velero includes everything in the backup or restore objects. For more information, see Resource Filtering.

Troubleshooting

This section provides notes for troubleshooting purposes as needed.

Issue Workaround
Getting an error "The provided token has expired", when running the backup or restore velero command.