Testing S2I Images

Source-to-Image (S2I) is part of the OpenShift tool chain. It is built on Docker and abstracts how applications are built and run in OpenShift. For this, S2I images contain some logic in the form of shell scripts. To make sure that an S2I image works as expected after changing it, it is important to create some kind of test infrastructure. This can be achieved by writing test code which uses the S2I image and asserts its behavior. In this blog post I’m going to show how to set up a test environment for S2I images and how to automate test execution on Travis CI . So the following is of interest to everybody who creates custom S2I images.

What to test?

The first thing to ask ourselves is: what do we want to test anyway? There are usually a number of things which should be tested:

1. Is the image in a valid state? Does it contain all tools necessary to build and run an application? In other words: is the Dockerfile correct?
2. Do all the S2I scripts work as expected?
3. Is the image capable of assembling and running an application?

The question is: how can we test these things? We certainly cannot write a unit test in Ruby or Scala, because we’re not testing application code. What we have to do is use the S2I command line tool and Docker to build and run a test application inside the builder image. So the language for testing the S2I image will be shell.

In the following sections I will show how I’ve implemented the test environment for the codecentric/springboot-maven3-centos S2I builder image. I’ve adopted the test environment from openshift/sti-ruby . It is a good source of inspiration for setting up an S2I project. Note that the openshift/sti-ruby test setup is a bit more complicated because it tests a RHEL and a CentOS based version of the Ruby S2I builder image.

Anatomy of an S2I project

Before we dig into the details of how to test an S2I image, let’s first take a moment and think about how we can structure the necessary code. At the point of this writing there is no official guide on how to structure an S2I repository. I’ve drawn my inspiration from the official OpenShift S2I image repositories. But even between those there are inconsistencies. Nevertheless, S2I image repositories usually look something like this:

1image-dir
2 ├── s2i
3 |    └── bin
4 |         ├── assemble
5 |         ├── run
6 |         ├── usage
7 |         └── save-artifacts
8 ├── test
9 |    ├── test-app
10 |    └── run
11 ├── Dockerfile
12 └── Makefile

The s2i/bin folder contains the S2I scripts which will be part of the builder image. The contents have to be copied into the Docker image during the Docker build. The test folder contains one or more simple sample projects. In the case of the codecentric/springboot-maven3-centos image, it contains a basic Spring Boot application. The test/run script implements the logic for testing the S2I image (more on that in the next section). The Dockerfile defines the S2I docker image and contains the statements for installing for example a Java SDK and some build tools. Last but not least a Makefile can be used for starting the docker build and the tests.

The test script explained

Let’s take a more detailed look at the test/run script . Lines 135 through 160 implement the steps for testing the image:

1prepare
2 
3info "Testing STI incremental build"
4run_s2i_build
5check_result $?
6 
7# Verify the 'usage' script is working properly
8test_s2i_usage
9check_result $?
10 
11# Verify the 'usage' script is working properly when running the base image with 'docker run ...'
12test_docker_run_usage
13check_result $?
14 
15# Verify that the HTTP connection can be established to test application container
16run_test_application &
17 
18# Wait for the container to write it's CID file
19wait_for_cid
20 
21test_connection
22check_result $?
23 
24cleanup
25 
26info "All tests finished successfully."

The first step is the prepare step. It checks whether the builder image exists and then initializes a new git repository inside the test application directory. This is necessary because S2I currently requires the application sources to be inside a valid git repository. Next the run_s2i_build step will use S2I to build an application image from the builder image being tested and the test application sources. After we have built the application image, the test_s2i_usage and test_docker_run_usage make sure the usage script does work, both when calling s2i usage and when running the builder image using docker run. The last thing we have to verify is that the application can be started when running the application image. This is done by running the application image (run_test_application) and then accessing it via cURL (test_connection). The check_result function is used throughout the test script to make sure none of other the command exits with a non-zero return code.

I won’t go into the details of how the steps are implemented here. The test script is pretty straightforward and you can have a look at it yourself. If you have Docker and S2I installed, you can even try it out on your local machine. Unfortunately you will see the following error when executing the make test on Mac OS:

1readlink: illegal option -- z
2usage: readlink [-n] [file ...]

This is because readlink among other core utilities does not work the same way on Mac OS as it does on Linux. For this reason we have created a linux VM using Vagrant , which has all necessary tools already installed. It is available as open source in the codecentric GitHub account: codecentric/s2i-test-vm .

Test automation on Travis CI

To automate testing of an S2I builder image on travis, a number of things have to be set up correctly:

1. Docker has to be available
2. Go has to be available
3. S2I has to be installe

To make Docker available in a Travis build, the following two lines have to be added to .travis.yml:

1sudo: required
2
3services:
4  - docker

The first line tells Travis not to run the test in the containerized infrastructure. This is important because if the build is run inside a Docker container, it cannot access Docker. The service directive tells Travis to make the Docker service available to the build.
Setting up Go is as easy as adding the following lines to .travis.yml:

1language: go
2go: 1.4

Since S2I requires at least Go 1.4, it is important to specify the version explicitly. Next, we have to install S2I inside the Travis build. Luckily Travis provides the `install` hook, which can be used to set up the build environment. Translating the installation instructions for S2I to .travis.yml yields:

1install:
2  - go get github.com/openshift/source-to-image
3  - pushd ${GOPATH}/src/github.com/openshift/source-to-image
4  - export PATH=$PATH:${GOPATH}/src/github.com/openshift/source-to-image/_output/local/bin/linux/amd64/
5  - hack/build-go.sh
6  - popd

The last thing to do is to tell Travis to call the make file via script: make test. The full working example can be reviewed in the codecentric/springboot-maven3-centos GitHub repository .

Conclusion

In this post I’ve demonstrated how to test an S2I image using a test project and a test shell script. For testing an S2I image locally, I have provided a virtual machine that has everything needed for running tests. To take testing of S2I images to the next level, I’ve shown how to set up an S2I image test build on Travis CI. This way you can be confident when making changes to your S2I image.