• Get application security done the right way! Detect, Protect, Monitor, Accelerate, and more…
  • All about the Docker Swarm and how it manages and orchestrates all the containers in a cluster.

    What is Docker Swarm?

    Docker swarm is a mode of handling a cluster of Docker Engines, hence the name Swarm. The cluster of Docker hosts run in swarm mode consisting of managers and workers. The docker-engine instances which participate in the swarm are called nodes.

    A production level swarm deployment consists of docker nodes spread across multiple servers.

    Why use it? – Container Orchestration

    When you are working in a production environment, 100s of docker containers will be running multiple applications in it. Managing all these containers can be a big pain for all the DevOps engineers; this is where Docker Swarm helps you out. It manages and orchestrates the cluster running multiple docker containers with ease.

    Below are some of its features:

    • High-Availability – aims to offer no downtime or outage.
    • Load Balancing – allocate the resources and requests on other nodes in the cluster automatically if any node fails.
    • De-centralized – multiple manager nodes run in a production environment; hence the cluster is never dependent on a single manager node.
    • Scalability – using a single docker swarm command, you can easily scale-up or scale-down containers in the cluster.

    Orchestrate Docker Containers

    Now that you know the basics of Docker Swarm let us look at an example of its implementation.

    In this example, I have three machines running in a cluster with the below details:

    manager1: 192.168.56.104
    
    worker1: 192.168.56.105
    
    worker2: 192.168.56.102

    To initialize the swarm mode in docker, run the below command on the manager node. The flag --advertise-addr is used for advertising itself to the nodes which can join the cluster.

    [email protected]:~$ docker swarm init --advertise-addr 192.168.56.104
    
    Swarm initialized: current node (lssbyfzuiuh3sye1on63eyixf) is now a manager.
    
    To add a worker to this swarm, run the following command:
    
    docker swarm join --token SWMTKN-1-3h3d8qgvdlxi8tl1oqpfho9khx7i1t5nq7562s9gzojbcm9kr6-azy4rffrzou0nem9hxq4ro5am 192.168.56.104:2377
    
    To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

    The above command will generate a token which will be used by other nodes to join this cluster. Copy the command with the token generated and run it on the worker nodes.

    Running the token on worker1 node.

    [email protected]:~$ docker swarm join --token SWMTKN-1-3h3d8qgvdlxi8tl1oqpfho9khx7i1t5nq7562s9gzojbcm9kr6-azy4rffrzou0nem9hxq4ro5am 192.168.56.104:2377
    
    This node joined a swarm as a worker.

    Running the token on worker2 node.

    [email protected]:~$ docker swarm join --token SWMTKN-1-3h3d8qgvdlxi8tl1oqpfho9khx7i1t5nq7562s9gzojbcm9kr6-azy4rffrzou0nem9hxq4ro5am 192.168.56.104:2377
    
    This node joined a swarm as a worker.

    Now, on the manager node, you can check which nodes are running in the cluster.

    [email protected]:~$ docker node ls
    
    ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
    
    lssbyfzuiuh3sye1on63eyixf *   manager1            Ready               Active              Leader              18.09.6
    
    utdr3dnngqf1oy1spupy1qlhu     worker1             Ready               Active                                  18.09.6
    
    xs6jqp95lw4cml1i1npygt3cg     worker2             Ready               Active                                  18.09.6

    Let’s build the geekflare_mongodb docker image which we used in Dockerfile Tutorial.

    docker build -t geekflare_mongodb .

    Run a container of the MongoDB docker image by creating a swarm service. 27017 is the port number on which MongoDB is exposed.

    [email protected]:~$ docker service create --name "Mongo-Container" -p 27017:27017 geekflare_mongodb
    
    image geekflare_mongodb:latest could not be accessed on a registry to record its digest. Each node will access geekflare_mongodb:latest independently, possibly leading to different nodes running different versions of the image.
    
    kok58xa4zi05psh3uy6s5x9e6
    
    overall progress: 1 out of 1 tasks
    
    1/1: running
    
    verify: Service converged

    Check if the docker swarm service has started.  MODE replicated means the container has been replicated on all the nodes in the cluster and REPLICAS 1/1 means only one swarm service is currently running.

    [email protected]:~$ docker service ls
    
    ID                  NAME                MODE                REPLICAS            IMAGE                      PORTS
    
    kok58xa4zi05        Mongo-Container     replicated          1/1                 geekflare_mongodb:latest   *:27017->27017/tcp

    Let us check on which node in the cluster this single service is running. It is running on manager1 node.

    [email protected]:~$ docker service ps Mongo-Container
    
    ID                  NAME                IMAGE                      NODE                DESIRED STATE       CURRENT STATE                ERROR               PORTS
    
    jgqjo92rbq23        Mongo-Container.1   geekflare_mongodb:latest   manager1            Running             Running about a minute ago

    Run docker ps command to get more details about the container which is running this swarm service.

    [email protected]:~$ docker ps
    
    CONTAINER ID        IMAGE                      COMMAND                  CREATED             STATUS              PORTS               NAMES
    
    05d77e7b4850        geekflare_mongodb:latest   "/bin/sh -c usr/bin/…"   2 minutes ago       Up 2 minutes        27017/tcp           Mongo-Container.1.jgqjo92rbq23sv01hrufdigtx

    You can run the swarm service in “global” mode also instead of default “replicated” mode. Global mode runs one task of the swarm service on all the nodes in the cluster.

    Before I run the service in global mode, let me remove the existing running container.

    [email protected]:~$ docker service rm Mongo-Container
    
    Mongo-Container

    Start the swarm service inside a docker container in global mode using –mode flag.

    [email protected]:~$ docker service create --name "Mongo-Container" -p 27017:27017 --mode global geekflare_mongodb
    
    image geekflare_mongodb:latest could not be accessed on a registry to record its digest. Each node will access geekflare_mongodb:latest independently, possibly leading to different nodes running different versions of the image.
    
    mfw8dp0zylffppkllkcjl8391
    
    overall progress: 3 out of 3 tasks
    
    utdr3dnngqf1: running
    
    lssbyfzuiuh3: running
    
    xs6jqp95lw4c: running
    
    verify: Service converged

    Check if the swarm service started in global mode. Since, three nodes (1 manager, 2 workers) are running in the cluster, that’s why the number of replicas is 3.

    [email protected]:~$ docker service ls
    
    ID                  NAME                MODE                REPLICAS            IMAGE                      PORTS
    
    mfw8dp0zylff        Mongo-Container     global              3/3                 geekflare_mongodb:latest   *:27017->27017/tcp

    3 services are running now across 3 nodes, check it by running the below command.

    [email protected]:~$ docker service ps Mongo-Container
    
    ID                  NAME                                        IMAGE                      NODE                DESIRED STATE       CURRENT STATE                ERROR               PORTS
    
    zj2blvptkvj6        Mongo-Container.xs6jqp95lw4cml1i1npygt3cg   geekflare_mongodb:latest   worker2             Running             Running about a minute ago
    
    3eaweijbbutf        Mongo-Container.utdr3dnngqf1oy1spupy1qlhu   geekflare_mongodb:latest   worker1             Running             Running about a minute ago
    
    yejg1o2oyab7        Mongo-Container.lssbyfzuiuh3sye1on63eyixf   geekflare_mongodb:latest   manager1            Running             Running about a minute ago

    Next, let me show how you can define the number of replicas. Before that, I will remove the current container, which is running.

    [email protected]:~$ docker service rm Mongo-Container
    
    Mongo-Container

    Use –replicas flag in the command and mention the number of replicas you want of the swarm service. For example, I want to have two replicas of the swarm service:

    [email protected]:~$ docker service create --name "Mongo-Container" -p 27017:27017 --replicas=2 geekflare_mongodb
    
    image geekflare_mongodb:latest could not be accessed on a registry to record its digest. Each node will access geekflare_mongodb:latest independently, possibly leading to different nodes running different versions of the image.
    
    4yfl41n7sfak65p6zqwwjq82c
    
    overall progress: 2 out of 2 tasks
    
    1/2: running
    
    2/2: running
    
    verify: Service converged

    Check the swarm services currently running. You can see one replica is running on manager1 node and the other one on worker1 node.

    [email protected]:~$ docker service ps Mongo-Container
    
    ID                  NAME                IMAGE                      NODE                DESIRED STATE       CURRENT STATE           ERROR               PORTS
    
    xukodj69h79q        Mongo-Container.1   geekflare_mongodb:latest   worker1             Running             Running 9 seconds ago
    
    e66zllm0foc8        Mongo-Container.2   geekflare_mongodb:latest   manager1            Running             Running 9 seconds ago

    Go to worker1 node and check if the docker container is running the swarm service.

    [email protected]:~$ docker ps
    
    CONTAINER ID        IMAGE                      COMMAND                  CREATED              STATUS              PORTS               NAMES
    
    5042b7f161cb        geekflare_mongodb:latest   "/bin/sh -c usr/bin/…"   About a minute ago   Up About a minute   27017/tcp           Mongo-Container.1.xukodj69h79q3xf0pouwm7bwv

    To stop this container, run the command below.

    [email protected]:~$ docker stop 5042b7f161cb
    
    5042b7f161cb

    Now from manager1 node if you check which all nodes are running the service, you will see its running on manager1 node and worker2 node. The CURRENT STATE of worker1 node is Shutdown (as we stopped the container running the service). But since two replicas must run of this service, another service was started on worker 2.

    This is how you achieve high availability using docker swarm.

    [email protected]:~$ docker service ps Mongo-Container
    
    ID                  NAME                    IMAGE                      NODE                DESIRED STATE       CURRENT STATE            ERROR                         PORTS
    
    cd2rlv90umej        Mongo-Container.1       geekflare_mongodb:latest   worker2             Running             Running 30 seconds ago
    
    xukodj69h79q         \_ Mongo-Container.1   geekflare_mongodb:latest   worker1             Shutdown            Failed 38 seconds ago    "task: non-zero exit (137)"
    
    e66zllm0foc8        Mongo-Container.2       geekflare_mongodb:latest   manager1            Running             Running 3 minutes ago

    It’s very easy to scale up or down docker containers. The command below will scale up the mongo container to 5.

    [email protected]:~$ docker service scale Mongo-Container=5
    
    Mongo-Container scaled to 5
    
    overall progress: 5 out of 5 tasks
    
    1/5: running
    
    2/5: running
    
    3/5: running
    
    4/5: running
    
    5/5: running
    
    verify: Service converged

    Check how many replicas of mongo container is running now, it must be 5.

    [email protected]:~$ docker service ls
    
    ID                  NAME                MODE                REPLICAS            IMAGE                      PORTS
    
    4yfl41n7sfak        Mongo-Container     replicated          5/5                 geekflare_mongodb:latest   *:27017->27017/tcp

    Check where these 5 replicas are running in the cluster. 1 replica is running on manager1 node and 2 replicas on both the worker nodes each.

    [email protected]:~$ docker service ps Mongo-Container
    
    ID                  NAME                    IMAGE                      NODE                DESIRED STATE       CURRENT STATE            ERROR                         PORTS
    
    cd2rlv90umej        Mongo-Container.1       geekflare_mongodb:latest   worker2             Running             Running 2 minutes ago
    
    xukodj69h79q         \_ Mongo-Container.1   geekflare_mongodb:latest   worker1             Shutdown            Failed 2 minutes ago     "task: non-zero exit (137)"
    
    e66zllm0foc8        Mongo-Container.2       geekflare_mongodb:latest   manager1            Running             Running 5 minutes ago
    
    qmp0gqr6ilxi        Mongo-Container.3       geekflare_mongodb:latest   worker2             Running             Running 47 seconds ago
    
    9ddrf4tsvnu2        Mongo-Container.4       geekflare_mongodb:latest   worker1             Running             Running 46 seconds ago
    
    e9dhoud30nlk        Mongo-Container.5       geekflare_mongodb:latest   worker1             Running             Running 44 seconds ago

    In your cluster, if you don’t want your services to run on manager node(s), and want to keep it only for managing the nodes, you can drain the manager node out.

    [email protected]:~$ docker node update --availability drain manager1
    
    manager1

    Check the availability of the manager node.

    [email protected]:~$ docker node ls
    
    ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
    
    lssbyfzuiuh3sye1on63eyixf *   manager1            Ready               Drain               Leader              18.09.6
    
    utdr3dnngqf1oy1spupy1qlhu     worker1             Ready               Active                                  18.09.6
    
    xs6jqp95lw4cml1i1npygt3cg     worker2             Ready               Active                                  18.09.6

    You will see the services are not running on the manager node anymore; they are spread across the worker nodes in the cluster.

    [email protected]:~$ docker service ps Mongo-Container
    
    ID                  NAME                    IMAGE                      NODE                DESIRED STATE       CURRENT STATE             ERROR                         PORTS
    
    cd2rlv90umej        Mongo-Container.1       geekflare_mongodb:latest   worker2             Running             Running 5 minutes ago
    
    xukodj69h79q         \_ Mongo-Container.1   geekflare_mongodb:latest   worker1             Shutdown            Failed 5 minutes ago      "task: non-zero exit (137)"
    
    qo405dheuutj        Mongo-Container.2       geekflare_mongodb:latest   worker1             Running             Running 41 seconds ago
    
    e66zllm0foc8         \_ Mongo-Container.2   geekflare_mongodb:latest   manager1            Shutdown            Shutdown 44 seconds ago
    
    qmp0gqr6ilxi        Mongo-Container.3       geekflare_mongodb:latest   worker2             Running             Running 3 minutes ago
    
    9ddrf4tsvnu2        Mongo-Container.4       geekflare_mongodb:latest   worker1             Running             Running 3 minutes ago
    
    e9dhoud30nlk        Mongo-Container.5       geekflare_mongodb:latest   worker1             Running             Running 3 minutes ago

    That was all about Docker Swarm and how to orchestrate containers in docker swarm mode. Try out these on your non-production environment to get an idea of how it works.