Lightweight Virtual Machines Made Simple with Docker or How to Run 100 Virtual Machines

Running virtual machines has many benefits. They utilize your hardware much better, are easy to backup and exchange, and isolate services from each other. But running virtual machines also has downsides. Virtual machine images are clunky. Also and more importantly, virtual machines require a fair amount of resources as they emulate hardware and run a full stack operating system. With Linux Containers there exists a lightweight alternative to full blown virtual machines while retaining their benefits. In this article, I present Docker, a tool that eases the practical application of Linux Containers with a fully working example available on Github to get you started with Docker yourself.

What is Docker?

Linux Container (LXC) has been part of Linux since version 2.6.24 and provides system-level virtualization. It uses Linux cgroups and name spaces to isolate processes from each other so they appear to run on their own system. Docker builds upon LXC and consists of three parts: Docker Daemon, Docker Images, the Docker Repositories which together make Linux Container easy and fun to use.

Docker Daemon runs as root and orchestrates all running containers. Just as virtual machines are based on images, Docker Containers are based on Docker images. These images are tiny compared to virtual machine images and are stackable thanks to AUFS storing only changes — see below. Docker Images can be exchanged with others and versioned like source code in private or public Docker Repositories.

Let me give you an example: 100 Virtual Machines on your System

As an introduction on how to use Docker yourself, I walk you through an example to show you how Docker works. In our example, 100 Docker Containers run each with an individual IP address serving web requests by a simple Python web server on port 8080. The full code for this example is available from Github so you can make your first steps with Docker right with me.

From Dockerfile to Docker Image

A Dockerfile describes how to build a Docker Image. For this example, I use two Dockerfiles, one for the Python runtime and another one for the Python script implementing the web server. Listings 1 and 2 show the corresponding files.

FROM ubuntu:quantal
MAINTAINER Lukas Pustina <lukas.pustina@centerdevice.com>
RUN apt-get install -y python
Listing 1

The FROM command defines the base image from which we start. In this example, I use a public Ubuntu image. The images are looked up locally as well as in the publicly available Docker repository. The RUN command specifies which commands to run during the build process. Here, only Python is added to the base image.

FROM docker-demo/python
MAINTAINER Lukas Pustina <lukas.pustina@centerdevice.com>
ADD webserver.py /opt/webserver/webserver.py
ADD run.sh /opt/webserver/run.sh
EXPOSE 8080
VOLUME ["/logs”]
Listing 2

The second Docker Image derives from the Python runtime, and adds the two files webserver.py and run.sh for running the web server. Generally, all Docker Containers run isolated from the world with no communication allowed — deny all policy. If there should be communication to the outside world, this must be explicitly defined through the EXPOSE command. In this example, port 8080 is exposed. The VOLUME commands specifies a mount point to which we can bind filesystems from the host operating system or other containers. This allows us to attach globally reusable and shareable mount point.

Building Images: docker build

From these Dockerfiles, Docker Images are build with each Dockerfile command generating a new Docker Image which can be individually accessed by its id — a git commit-like fingerprint. For our example we build the python runtime container

> docker build -rm -t docker-demo/python python

in directory “python”, remove all intermediate images (-rm) and tag the final results with “docker-demo/python”.

> docker images

shows all currently existing images. In our examples this is (The size denotes the virtual size — see below):

docker-demo/python      latest   ef489f0186e1        Less than a second ago   210.7 MB
docker-demo/webserver   latest   b898a85622e4        Less than a second ago   210.7 MB
ubuntu                  quantal  b750fe79269d        9 months ago             175.3 MB

> docker images -tree

shows the hierarchal inheritance and the physical as well as virtual size of the images:

└─27cf78414709 Size: 175.3 MB (virtual 175.3 MB)
  └─b750fe79269d Size: 77 B (virtual 175.3 MB) Tags: ubuntu:quantal
    └─7c1926658b21 Size: 0 B (virtual 175.3 MB)
      └─ef489f0186e1 Size: 35.43 MB (virtual 210.7 MB) Tags: docker-demo/python:latest
        └─90807ce05b64 Size: 0 B (virtual 210.7 MB)
          └─d3822b811f4c Size: 0 B (virtual 210.7 MB)
            └─380871b22bde Size: 1.454 kB (virtual 210.7 MB)
              └─8f5a9cde5ae2 Size: 161 B (virtual 210.7 MB)
                └─b898a85622e4 Size: 0 B (virtual 210.7 MB) Tags: docker-demo/webserver:latest

As you can see, the physical size changed only by the size of files added. Tagging does not increase the size. Interestingly, a Docker Image running a python web server is only around 180 MB in size.

Making Changes

If a change to one image and thus a rebuild is necessary, only the corresponding containers change. In this example, the web server container has been changed and docker images -tree rerun:

└─27cf78414709 Size: 175.3 MB (virtual 175.3 MB)
  └─b750fe79269d Size: 77 B (virtual 175.3 MB) Tags: ubuntu:quantal
    └─7c1926658b21 Size: 0 B (virtual 175.3 MB)
      └─ef489f0186e1 Size: 35.43 MB (virtual 210.7 MB) Tags: docker-demo/python:latest
        └─90807ce05b64 Size: 0 B (virtual 210.7 MB)
          |─0e153dd3c547 Size: 0 B (virtual 210.7 MB)
          | └─922d6e34d2d5 Size: 1.454 kB (virtual 210.7 MB)
          |   └─4bfc3d0fb5b2 Size: 161 B (virtual 210.7 MB)
          |     └─abac38e20f27 Size: 0 B (virtual 210.7 MB) Tags: docker-demo/webserver:latest
          └─d3822b811f4c Size: 0 B (virtual 210.7 MB)
            └─380871b22bde Size: 1.454 kB (virtual 210.7 MB)
              └─8f5a9cde5ae2 Size: 161 B (virtual 210.7 MB)
                └─b898a85622e4 Size: 0 B (virtual 210.7 MB)

As you can see, the base images stay the same and only the changes from the web server Dockerfile are reapplied changing only a few Bytes. Not bad for virtual machine images.

Running Containers: docker run

To start a Docker Container and run it based on a Docker Image, you use docker run:

> docker run -d -cidfile=webserver.cid -name webserver -v `pwd`/logs:/logs docker-demo/webserver:latest /opt/webserver/run.sh

In our example, the Docker’s most recently image tagged “docker-demo/webserver” is run. This means image abac38e20f27 is used instead of b898a85622e4. The container id is written to the file webserver.cid (-cidfile), named “webserver” (-name), the host local directory logs is mapped to the container directory /logs (-v; see Dockerfile above). The process to execute inside the container is /opt/webserver/run.sh which was copied into the image by the second Dockerfile.

You can also run several containers:

> for i in `seq 1 10`; do
docker run -d -cidfile=webserver-$i.cid -name webserver-$i -v `pwd`/logs:/logs docker-demo/webserver:latest /opt/webserver/run.sh /logs;
done

The full examples on Github runs 100 machines with ease in a Vagrant Box.

Showing Running Containers: docker ps

Similar to ps, docker ps shows the running containers:

CONTAINER ID  IMAGE                          COMMAND                PORTS     NAMES
63120c069662  docker-demo/webserver:latest   /opt/webserver/run.s   8080/tcp  webserver-10
5af293d56a0b  docker-demo/webserver:latest   /opt/webserver/run.s   8080/tcp  webserver-9
a4a44f159658  docker-demo/webserver:latest   /opt/webserver/run.s   8080/tcp  webserver-8
40b35ff10aa2  docker-demo/webserver:latest   /opt/webserver/run.s   8080/tcp  webserver-7
6e9127f8a113  docker-demo/webserver:latest   /opt/webserver/run.s   8080/tcp  webserver-6
4623905191d2  docker-demo/webserver:latest   /opt/webserver/run.s   8080/tcp  webserver-5
e57def12fe25  docker-demo/webserver:latest   /opt/webserver/run.s   8080/tcp  webserver-4
c352eccb21de  docker-demo/webserver:latest   /opt/webserver/run.s   8080/tcp  webserver-3
f65e43115864  docker-demo/webserver:latest   /opt/webserver/run.s   8080/tcp  webserver-2
111c936c2763  docker-demo/webserver:latest   /opt/webserver/run.s   8080/tcp  webserver-1

As you can see, there are 10 containers running with the names webserver-* and TCP port 8080 exposed.

Getting additional Information: docker inspect

docker inspect gives additional information as a JONS document for a container id. This includes the hostname, host directory the images resides, ip addresses and more. To show the IP addresses of the above started containers, we grep for IPAddress:

> docker ps -q | xargs docker inspect | grep IPAddress | cut -d ‘”‘ -f 4

172.17.0.12
172.17.0.11
172.17.0.10
172.17.0.9
172.17.0.8
172.17.0.7
172.17.0.6
172.17.0.5
172.17.0.4
172.17.0.3

As you can see, each container runs with a different network interface and IP address forming a network of isolated system-level virtual machines.

Resource Usage

Running “ps aux | grep ‘lxc-start|python’” to inspect the resources used by these 10 Linux Containers, you see that for each machine the container process takes around 1.3 MB; the Python instance takes around 7 MB. That is less than 8.5 MB in total per virtual machine – quite an impressive figure as compared to real virtual machines.

  PID %CPU %MEM   RSS COMMAND
12197  0.0  0.2  1316 lxc-start
12249  0.0  0.2  1320 lxc-start
12291  0.0  1.3  6968 /usr/bin/python /opt/webserver/webserver.py
12296  0.0  0.2  1320 lxc-start
12346  0.0  1.3  6960 /usr/bin/python /opt/webserver/webserver.py
12348  0.0  0.2  1316 lxc-start
12399  0.0  1.3  6964 /usr/bin/python /opt/webserver/webserver.py
12400  0.0  0.2  1316 lxc-start
12450  0.0  0.2  1316 lxc-start
12496  0.0  1.3  6964 /usr/bin/python /opt/webserver/webserver.py
12497  0.0  0.2  1316 lxc-start
12546  0.0  0.2  1316 lxc-start
12594  0.0  0.2  1320 lxc-start
12648  0.0  1.3  6968 /usr/bin/python /opt/webserver/webserver.py
12649  0.0  0.2  1316 lxc-start
12700  0.0  1.3  6968 /usr/bin/python /opt/webserver/webserver.py
12701  0.0  1.3  6968 /usr/bin/python /opt/webserver/webserver.py
12707  0.0  1.3  6960 /usr/bin/python /opt/webserver/webserver.py
12711  0.0  1.3  6968 /usr/bin/python /opt/webserver/webserver.py
12712  0.0  1.3  6964 /usr/bin/python /opt/webserver/webserver.py

docker stop, kill, remove

In order to stop a container, it is sufficient to stop the process that the containers executes by sending SIGTERM or SIGKILL. docker stop tries to send SIGTERM first and after a timeout SIGKILL. docker kill sends SIGKILL immediately.

Like in stopped virtual machines, stopped Linux Containers still exist. To remove them, docker rm is used.

There’s more

Linux Containers offer much more functionality through Docker than I described. It is possible to share volumes between containers. Containers may be multi-homed using multiple network interfaces and networks freely configured. Private repositories can be easily set up and allow you to maintain your own set of Docker Images. Further, many command line arguments to “docker” can be moved statically to Dockerfiles.

Docker at CenterDevice

At CenterDevice we migrated our development environment on a Vagrant based Virtual Box that uses Docker Images for all backend services like MongoDB, RabbitMQ or Tomcat. In this way, each developer can easily set up a working environment, run services isolated and distributed like on production, but without exceeding resources of their development machine.

In the future, we plan to move the creation of Docker Images into our Continuous Integration pipeline. That way, a successful build generates Docker Images for development, testing, and even production.

I will blog about these topics in the future. So stay tuned and feel free to contact me, if you have questions or comments.