Deploying a Swarm Cluster on On-premises VMWare

To deploy a Swarm cluster:
  1. Navigate to the folder tmgc-deploy/onprem/swarm/manifest-onprem-swarm.

  2. Prepare Virtual Boxes on ESX host. VMs can be provisioned on ESX host in multiple ways. You can prepare VMs as per your practice or use docker-machine utility.

    The following example command can be used to prepare VM. It provisions VM with Rancher OS.

    Docker Machine Command to Create VM on ESX Host 

    docker-machine create --driver vmwarevsphere --vmwarevsphere-datacenter ha-datacenter --vmwarevsphere-vcenter <Vcenter IP> --vmwarevsphere-username <User Name> --vmwarevsphere-password <Password> --vmwarevsphere-datastore datastore1 --vmwarevsphere-network 'VM Network' --vmwarevsphere-cfgparam "guestinfo.hostname=<Host Name>" --vmwarevsphere-cfgparam "guestinfo.interface.0.name=eth0" --vmwarevsphere-cfgparam "guestinfo.interface.0.ip.0.address=<Host IP to be Assigned>" --vmwarevsphere-cfgparam "guestinfo.interface.0.route.0.gateway=<ESX Server Gateway IP>" --vmwarevsphere-boot2docker-url https://github.com/rancher/os/releases/download/v1.4.2/rancheros-vmware.iso <Host Name>
    Sample command: 
    docker-machine create --driver vmwarevsphere --vmwarevsphere-datacenter ha-datacenter --vmwarevsphere-vcenter 10.127.138.47 --vmwarevsphere-username root --vmwarevsphere-password XYZ --vmwarevsphere-datastore datastore1 --vmwarevsphere-network 'VM Network' --vmwarevsphere-cfgparam "guestinfo.hostname=manager" --vmwarevsphere-cfgparam "guestinfo.interface.0.name=eth0" --vmwarevsphere-cfgparam "guestinfo.interface.0.ip.0.address=10.127.138.50/23" --vmwarevsphere-cfgparam "guestinfo.interface.0.route.0.gateway=10.127.138.1" --vmwarevsphere-boot2docker-url
https://github.com/rancher/os/releases/download/v1.4.2/rancheros-vmware.iso manager

    You need to execute the above command as many times as the number of nodes you want to create in the swarm cluster including manager node.

  3. Once all the VMs are provisioned, SSH into the VM that you want to run as swarm manager. If you used docker-machine to create VMs, you can run the following command. 
    docker-machine ssh <master vm name>
  4. Start swarm manager on the selected node. Execute the following. 
    docker swarm init --advertise-addr <Manager IP>
    This will produce an output command (starting with docker swarm join) that needs to be executed on all other worker nodes.
    Note:

    Sample

    Sample command: 
     docker swarm init --advertise-addr 10.127.135.61
    Output: 
    Swarm initialized: current node (1qqfygb5po6r0y8vqjjceu6pj) is now a manager.
    To add a worker to this swarm, run the following command: 
    docker swarm join --token SWMTKN-1-5ef0r0rlhqrgw5olehorwrsmn0cz83ygnkfex8lzeixndv4av3-at5f983kuwyyiyko6hls412cu 10.127.135.61:2377
    To add a manager to this swarm, run the following command: 
    docker swarm join-token manager
    You need to ssh into other nodes and run the docker swarm join command (output of step 4) and advertising node's IP. Use the following command as a reference to be executed on other nodes. 
    docker swarm join --token <TOKEN> <Manger IP>:2377
--advertise-addr <Node IP>
    Sample command based on above command output. 
    Node Command

docker swarm join --token SWMTKN-1-0xuxeo975vwq2dprcy77loazmyqqqr9muzw29yh0xvopq8398z-3zo8r2hb6o3xw51cyogz3z0k6 10.107.138.60:2377 --advertise-addr 10.107.138.62

This node joined a swarm as a worker.

  5. Verify that the cluster is created successfully by running the following command on the manager. Status should be ready for all the nodes.

    Do not proceed until all the nodes are in active status. 
    docker node ls
    Sample output:

  6. Login to the docker registry. 
    docker login <registry host> -u <user> -p <password>
  7. On successfull creation of swarm cluster, create an overlay network ml5 for the containers networking. Execute below on manager node. 
    docker network create -d overlay --attachable ml5