Deploying a Swarm Cluster on On-premises VMWare

To deploy a Swarm cluster:
  1. Navigate to the folder tmgc-deploy/onprem/swarm/manifest-onprem-swarm.

    Put all the TIBCO Mashery Local images in this folder that you downloaded earlier and execute the following command.

    If you built the TML images using tml-installer, copy those images from that build machine to this location on deployment host. If your build and deployment host are the same, you can copy those images from tml-installer container into this location using the following example docker cp command (assuming build no 1): 
    docker cp tml-installer:/var/jenkins_home/jobs/tml-build-docker/builds/1/archive/tml-docker/tml-nosql.tar.gz .
docker cp tml-installer:/var/jenkins_home/jobs/tml-build-docker/builds/1/archive/tml-docker/tml-sql.tar.gz .
docker cp tml-installer:/var/jenkins_home/jobs/tml-build-docker/builds/1/archive/tml-docker/tml-log.tar.gz .
docker cp tml-installer:/var/jenkins_home/jobs/tml-build-docker/builds/1/archive/tml-docker/tml-cm.tar.gz .
docker cp tml-installer:/var/jenkins_home/jobs/tml-build-docker/builds/1/archive/tml-docker/tml-cache.tar.gz .
docker cp tml-installer:/var/jenkins_home/jobs/tml-build-docker/builds/1/archive/tml-docker/tml-tm.tar.gz .

  2. Prepare Virtual Boxes on ESX host. VMs can be provisioned on ESX host in multiple ways. Clients can prepare VMs as per their practice or use docker-machine utility.

    The following example command can be used to prepare VM. It provisions VM with Rancher OS.

    Docker Machine Command to Create VM on ESX Host 

    docker-machine create --driver vmwarevsphere --vmwarevsphere-datacenter ha-datacenter --vmwarevsphere-vcenter <Vcenter IP> --vmwarevsphere-username <User Name> --vmwarevsphere-password <Password> --vmwarevsphere-datastore datastore1 --vmwarevsphere-network 'VM Network' --vmwarevsphere-cfgparam "guestinfo.hostname=<Host Name>" --vmwarevsphere-cfgparam "guestinfo.interface.0.name=eth0" --vmwarevsphere-cfgparam "guestinfo.interface.0.ip.0.address=<Host IP to be Assigned>" --vmwarevsphere-cfgparam "guestinfo.interface.0.route.0.gateway=<ESX Server Gateway IP>" --vmwarevsphere-boot2docker-url https://github.com/rancher/os/releases/download/v1.4.2/rancheros-vmware.iso <Host Name>
    Sample command: 
    docker-machine create --driver vmwarevsphere --vmwarevsphere-datacenter ha-datacenter --vmwarevsphere-vcenter x.x.x.x --vmwarevsphere-username root --vmwarevsphere-password XYZ --vmwarevsphere-datastore datastore1 --vmwarevsphere-network 'VM Network' --vmwarevsphere-cfgparam "guestinfo.hostname=manager" --vmwarevsphere-cfgparam "guestinfo.interface.0.name=eth0" --vmwarevsphere-cfgparam "guestinfo.interface.0.ip.0.address=x.x.x.x/23" --vmwarevsphere-cfgparam "guestinfo.interface.0.route.0.gateway=x.x.x.1" --vmwarevsphere-boot2docker-url
https://github.com/rancher/os/releases/download/v1.4.2/rancheros-vmware.iso manager

    You need to execute the above command as many times as the number of nodes you want to create in the swarm cluster including manager node.

  3. Once all the VMs are provisioned, SSH into the VM that you want to run as swarm manager. If you used docker-machine to create VMs, you can run the following command. 
    docker-machine ssh <master vm name>
  4. Start swarm manager on the selected node. Execute the following. 
    docker swarm init --advertise-addr <Manager IP>
    This will produce an output command (starting with docker swarm join) that needs to be executed on all other worker nodes.
    Note:

    Sample

    Sample command: 
     docker swarm init --advertise-addr
10.127.135.61
    Output: 
    Swarm initialized: current node (1qqfygb5po6r0y8vqjjceu6pj) is now a manager.

To add a worker to this swarm, run the following command:

    docker swarm join --token SWMTKN-1-5ef0r0rlhqrgw5olehorwrsmn0cz83ygnkfex8lzeixndv4av3-at5f983kuwyyiyko6hls412cu 10.127.135.61:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
    You need to ssh into other nodes and run the docker swarm join command (output of step 4) and advertising node's IP. Use the following command as a reference to be executed on other nodes. 
    docker swarm join --token <TOKEN> <Manger IP>:2377
--advertise-addr <Node IP>
    Sample command based on above command output. 
    Node Command

docker swarm join --token SWMTKN-1-0xuxeo975vwq2dprcy77loazmyqqqr9muzw29yh0xvopq8398z-3zo8r2hb6o3xw51cyogz3z0k6 10.107.138.60:2377 --advertise-addr 10.107.138.62

This node joined a swarm as a worker.

  5. Verify that the cluster is created successfully by running the following command on the manager. Status should be ready for all the nodes.

    Do not proceed until all the nodes are in active status. 
    docker node ls
    Sample output:

  6. If the swarm cluster is created successfully, create an Overlay network "ml5" for the containers networking. Execute the following command on the manager node. 
    docker network create -d overlay --attachable ml5
  7. Run the following Docker load image commands to load TML images on each machine in the cluster. 
    docker load -i tml-nosql.tar.gz
docker load -i tml-cm.tar.gz
docker load -i tml-log.tar.gz
docker load -i tml-sql.tar.gz
docker load -i tml-cache.tar.gz
docker load -i tml-tm.tar.gz