Deploying TIBCO BusinessEvents® Cluster for Shared Nothing Storage on AWS

By using the Kubernetes elements such as the StatefulSets object and dynamic volume provisioning features, you can create TIBCO ActiveSpaces and Shared Nothing deployments.

StatefulSets gives deterministic names to the pods. Along with dynamic volume provisioning, StatefulSets also give deterministic names to PersistentVolumeClaims (PVC). This ensures that when a particular member of a StatefulSet goes down and comes up again, it attaches itself to the same PVC. For more information about the Kubernetes concepts of StatefulSets, dynamic volume provisioning, and PersistentVolumeClaims, refer to the Kubernetes documentation at https://kubernetes.io/docs/concepts/.

Prerequisites

Ensure that CDD of your application is configured to use Shared Nothing. For deploying BusinessEvents cluster for Shared Nothing storage on AWS, you must first set up Kubernetes cluster on AWS and then upload your docker image to an AWS docker registry. For more information, see Running TIBCO BusinessEvents® on AWS Based Kubernetes Cluster.

Procedure

  1. In AWS, create an EFS file system.
    For more information on the steps to create an EFS file system, see Amazon EFS documentation at https://docs.aws.amazon.com/efs/latest/ug/gs-step-two-create-efs-resources.html. Specify the Kubernetes cluster Virtual Private Cloud (VPC) and Security Group while creating a mount target for the file system. On the File Systems page, verify that the mount target shows the Life Cycle State as Available. Under File system access, you see the file system DNS name. Make a note of this DNS name.
    After successful creation of the EFS file system, note its File System ID, which can be used for creating EFS provisioner.
  2. Create the EFS provisioner and other associated resources. Specify all the connection setup values for the EFS file system in a manifest.yaml file and run the kubectl command to create the EFS provisioner.
    1. Download the sample manifest.yaml file from https://raw.githubusercontent.com/kubernetes-incubator/external-storage/master/aws/efs/deploy/manifest.yaml and edit it according to your setup.
    2. In the configmap section specify File System ID of the newly created EFS as the value of the file.system.id: variable and Availability Zone of the newly created EFS as the value of the aws.region: variables.
    3. In the Deployment section, specify DNS name of the newly created EFS as the value of the server: variable.
    4. Run the kubectl command to apply the settings in manifest.yaml.
      kubectl apply -f manifest.yaml
    5. Ensure that the EFS provisioner pod is in the running state using the kubectl command.
      kubectl get pods
  3. Now, create Kubernetes resources, required for deploying TIBCO BusinessEvents cluster for the shared nothing storage, using the YAML files.
    These resources include StatefulSets and services for the cluster. Thus, to deploy a TIBCO BusinessEvents cluster, create:
    • A service to connect to discovery node
    • A cache agent node which connects to the discovery node service
    • An inference agent node that connects to the discovery node service
    • An external service to connect to the inference agent
    You can find sample YAML files for shared nothing at BE_HOME\cloud\kubernetes\sn. The following are the sample files that are provided (configured for FraudDetectionStore example):
    Sample Kubernetes Resource YAML Files for Shared Nothing Storage
    File Name Resource Resource Type Description
    bediscovery-service-sn.yaml Discovery node service Service (Internal) Setup the service to connect to the discovery node. Setup the label of the cache agent as the value of the selector variable for connection. Other nodes in the cluster use this service to connect to the discovery node. Specify the protocol and port to connect to this service.
    becacheagent-sn.yaml Cache agent node StatefulSet Setup the container (containers) with the docker image (image) of the application. Specify replicas value and start as many cache agent as specified in the value. Connect to the discovery node service using the discovery protocol and port specified in the discovery node service.
    beinferenceagent-sn.yaml Inference agent node StatefulSet Setup the container (containers) with the docker image (image) of the application. Provide a label (labels) to the StatefulSet which the inference agent service can use as selector. Specify at least one replica (replicas) of the inference agent node. Connect to the discovery node service using the discovery protocol and port specified in the discovery node service.
    befdservice-sn.yaml Inference agent service Service (LoadBalancer/External) Setup the service to externally connect to the inference agent. Setup the label of the inference agent as the value of the selector variable for connection. Specify the protocol and port to connect to this service externally.
  4. Run the create command of kubectl utility using the YAML files to deploy the BusinessEvents cluster.
    For example, deploy the cluster by using the following sample files:
    kubectl create -f bediscovery-service-sn.yaml
    
    kubectl create -f becacheagent-sn.yaml
    
    kubectl create -f beinferenceagent-sn.yaml
    
    kubectl create -f befdservice-sn.yaml
    You can also get the external IP to the external service of the cluster by using the get services command. You can then use that IP to connect to the cluster.
    kubectl get services befdservice
    You can check the logs of individual BusinessEvents container pods using the following command:
    kubectl logs <pod>