How to Secure Docker for Production Environment?

Let’s look at how to harden and secure the Docker for the production environment.

Although Docker has made it possible for software developers and DevOps engineers to build and deploy applications rapidly, it also comes with a large attack surface for cyber hackers to leverage on.

We will look at how to secure a Docker on a Linux platform from the following.

• Configuration flaws
• Remote code execution
• Buffer Overflows
• Image forgery and so on.

We will make use of the following tools, such as Docker’s notary server to sign images and Docker bench security to check for the host, daemon configuration, and so on.

Before we proceed to secure, let’s touch-base the basics.

What is a Container Technology?

Container technology allows developers or DevOps engineers to package an application so it can run with dependencies isolated from other processes.

There is a number of container technologies in the market, such as Apache Mesos, lxc, and Docker. Although they fall within the category of container technology, they function differently.

Difference Between VM & VE

A virtual machine host is entirely different from a virtual environment host. On virtual machines, each containerized application comes with its own set of libraries and operating system whilst applications, by default, on a virtual environment host such as lxc, and docker share the Linux kernel.

What is Docker?

Docker is a container technology used by millions to create a web application and deploy it from a testing to a production environment.

Docker Engine

The Docker Engine is made up of three components.

• A Server: This component is a long-running process or daemon responsible for managing images and containers.
• REST API:  This interface makes it possible for the docker daemon and the docker client tool to communicate.
• Docker Client tool: The Docker client tool makes use of the REST API component to inform the docker daemon to operate a containerized application.

Docker Trusted Registry

Docker Trusted Registry is an image storage solution from Docker for the enterprise platform business. It is different from the docker hub.  Whereas the docker hub is hosted in the cloud, the docker trusted registry is an on-premise storage solution for Docker enterprise edition.

Docker Content Trust

Docker Content Trust provides the ability to use data signatures for images sent and received to and from remote docker registries such as docker hub.

Linux Namespaces

Linux namespaces are a Linux kernel feature which isolates a containerized application or process running on virtual environment host from other processes.

Linux Control Groups(Cgroups)

Linux Control Groups is a Linux kernel feature that allows you to allocate resources such as CPU time, network bandwidth, system memory, and so on to active processes on a host.

Capabilities

In Linux, there is a security feature in the kernel subsystem which can be set or enforced to limit the privileged process such a process executed by a user with UID 1. Although privileged processes or users can bypass discretionary access control permissions, they can not bypass capabilities rules.

Now let’s focus on security.

Securing Docker Host

In this section, we will look at how to secure the host where Docker resides.

Scanning Linux kernel

Before you host a docker on a Linux platform, you first need to inspect the kernel. There are several open-source tools such as Lynis and OpenVAS you can use to scan the Linux kernel.

Copy or clone the Lynis project from Github using the git clone command.

git clone https://github.com/CISOfy/lynis.git

Next, use the command below to navigate to the lynis directory and audit the Linux system.

cd lynis; ./lynis audit system

Harden Linux kernel

After you have scanned the Linux kernel for system-based vulnerabilities, you can add another extra layer of protection to the kernel via grsecurity. It provides security features such as the following.

• Buffer overflow exploitation prevention
• /tmp race vulnerability prevention
• /proc restrictions that don’t leak information about process owners.
• Prevention of arbitrary code execution in the kernel and so on.

Initially, you can download patches for free from grsecurity and apply it to your current kernel. But it does not allow free patches anymore.

Install Docker in a VM

Instead of installing Docker directly on a Linux host, you can add an extra layer of protection by installing it inside a virtual machine. By so doing, even if there is a vulnerability issue with the host kernel, it won’t affect docker containers.

Protecting Root Privileges

By default, Docker requires root privileges to create and manage containers. The malicious script can leverage this attack surface to escalate to a superuser on a Linux host and eventually access sensitive files/folders, images, certificates, etc.

To prevent it, we can make use of the following command. We can decide to drop capabilities such as setgid and setuid to prevent other programs or processes from changing their GID to another GID which can result in escalation privilege. You can also check here for a list of Linux capabilities definition.

The command below runs the apache webserver container and drops the setgid and setuid capabilities via --cap-drop to prevent the apache container from changing its GID and UID to another  UID and GID.

GID and UID in this context refers to  group ID and user ID respectively.

docker run -d --cap-drop SETGID --cap-drop SETUID apache

Docker User

Apart from preventing other programs or processes, you can also create a user to manage docker operations such as docker run  instead of managing it via a superuser.

You can add or create a docker user via the following:

sudo groupadd docker

The command above creates a group called docker

Next, create a user using the command below:

sudo useradd mike

Finally use the command below to add a user mike to the group docker to administer docker operations.

sudo usermod  -aG docker mike

Managing Container with Cgroups

In a production environment, you may have more than one container.

If you don’t have cgroups installed on your host, you can use the following command to install it and then check here (for Ubuntu) on how to configure it.

sudo apt-get install cgroup-bin cgroup-lite cgroup-tools cgroupfs-mount libcgroup1

We can allocate the containers to limited CPU resources via the --cpu-shares and --cpuset-cpus

The following command example shows prodnginx container process is executed only on the first core via --cpuset-cpus and is allocate 20 CPU via  --cpu-shares whilst the proxnginx container process is executed on the first two CPU cores and is also allocated 20 CPU.

docker run -d --name prodnginx --cpuset-cpus=0 --cpu-shares=20 nginx

docker run -d --name testnginx --cpuset-cpus=2 --cpu-shares=20 nginx

Then type the command docker stats to view the CPU usage by the prodnginx and testnginx containers

CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O
845bea7263fb        prodnginx            57.69%              1.258MiB / 985.2MiB   0.13%               578B / 0B           1.33MB / 0B 
189ba15e8258        testnginx            55.85%              1.25MiB / 985.2MiB    0.13%               578b / 0B           1.33MB / 0B 

It is a good idea to define CPU-shares for a docker host when you have more than one container running on it.

Managing Containers with Namespaces

A namespace can prevent containers from running as privileged users, which can help to avoid privilege escalation attacks.

We can enable namespace in docker by making use of /etc/subuid  and  /etc/subgid files as shown below.

• create a user using the adduser command

sudo adduser dockremap

• Setup a subuid for the user  dockremap

sudo sh -c 'echo dockremap:400000:65536 > /etc/subuid'

• Then set up subgid for the user dockremap

sudo sh -c 'echo dockremap:400000:65536 > /etc/subgid'

• Open the daemon.json file and fill it with the following content to associate the userns-remap attribute to the user dockremap

vi   /etc/docker/daemon.json

{ 

 "userns-remap": "dockremap"

}

• Press :wq to save and close daemon.json file and finally restart docker to enable namespaces on a docker host

sudo  /etc/init.d/docker  restart

Securing the Docker Daemon

It is also necessary to configure the Docker daemon to ensure secure communication between docker client and docker daemon via TLS.

Use the following command to open daemon.json file and copy and paste the following content (replace the IP with your actual) as shown below

vi  daemon.json

{
  "debug": false,
  "tls": true,
  "tlscert": "/var/docker/server.pem",
  "tlskey": "/var/docker/serverkey.pem",
  "hosts": ["tcp://192.168.16.5:2376"]
}

Securing Docker Components

Let’s look at how to make use of tools such as CodeNotary and notary server to sign images in order to avoid image forgery. In addition, it is also necessary to scan images just to be sure images are not packed with vulnerabilities

We will make use of Docker’s notary server to sign and verify images and use Anchor Engine to scan images for vulnerabilities.

Verify Images with Notary Server

Before we can make use of the Notary server to sign images, we need to download and install docker-compose. We will use Docker Compose to set up a notary server.

• Run the command below to download the latest version of Docker Compose

sudo curl -L "https://github.com/docker/compose/releases/download/1.25.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

• Apply executable permissions to the docker-compose as shown below

sudo chmod 700  /usr/local/bin/docker-compose

• You can test if you have successfully installed docker-compose  via the following command

docker-compose  --version

• Now we can install the notary server via docker-compose

git clone https://github.com/theupdateframework/notary.git

• The command above clones or copies the notary server from the notary repository
• Start the notary server and signer via the commands below:

docker-compose build

docker-compose up -d

• Then copy the configuration and tests certificates to your local notary directory using the command below

mkdir -p ~/.notary && cp cmd/notary/config.json cmd/notary/root-ca.crt ~/.notary

• Now run the following command to connect the notary server to the docker client

export   DOCKER_CONTENT_TRUST=1

export  DOCKER_CONTENT_TRUST_SERVER=https://notaryserver:4443

• Generate a delegation key pair via the command below

docker trust key generate mike --dir ~./docker/trust          

• Now let’s create a target new keys in case the repository does not exist

docker trust signer add --key ~/.docker/trust/mike.pub mike mikedem0/whalesay

• Then you can sign your docker image using the command docker trust sign. You need to pull the docker image from docker hub and re-tag using the command docker pull and docker tag respectively.

docker trust sign mikedem0/nginx:latest

You can also scan docker images for vulnerabilities and configuration flaws. You can check here to find out how you to use Anchor Engine to scan for vulnerabilities and Docker Bench Security to check for configuration flaws.

I hope the above gives you an idea about the security Docker for the production environment. You may also want to check out this Udemy course about hacking and securing Docker containers.