Deploying Pull Requests with Docker

The Git repositories in my current project are hosted on Bitbucket Cloud. Any code changes have to go through pull requests. Jenkins builds the pull requests and gives its approval if the build is green. Additionally, at least one team member carries out a code review. Though a code review does not generally mandate that a reviewer build and test the app locally, one sometimes does want to see the result of the code changes, especially when, e. g., CSS/HTML are affected. In such cases, a visual test can help to understand and verify the changes.

In order to make things as easy as possible for the reviewer, we wanted to find a way to make a pull request testable with just the click of a button. We’ve just recently migrated our application to Spring Boot. Currently, we are still building a war file that is deployed on Tomcat. However, the executable war file can be run directly with the embedded Tomcat and, thus, can quite easily be packaged into a Docker image.

MongoDB is used as database. For the local development we dockerize the database. What’s special about that, is that the Docker image already contains data. The nature of our application requires that the database always have current data. For that purpose we have a nightly job that generates test data for the integration instance of our MongoDB. Jenkins in turn builds a Docker image for the MongoDB every night. The job creates a dump from the integration instance and imports it into the Docker image. The image is pushed to the Docker registry. Every morning, the developers can then pull a fresh image.

Our Jenkins also runs on Docker, one container for the master and another one for the build slave. The host’s Docker socket is mounted into the slave container. Now, when Jenkins builds a pull request, it also builds a Docker image thereof, which is tagged with the pull request number. Via Build Pipeline Plugin  it is now possible to deploy a pull request together with a fresh MongoDB instance on the Jenkins host.

The job for building the pull request has a manual downstream job (part of the Build Pipeline Plugin ) which deploys app and database.

A Python script creates a Docker network and starts the containers.

1#!/usr/bin/env python
2# -*- coding: utf-8 -*-
3 
4import socket
5from argparse import ArgumentParser, ArgumentDefaultsHelpFormatter
6from subprocess import Popen, STDOUT, PIPE
7 
8 
9def docker(*arguments):
10    cmdline = ['docker']
11    cmdline.extend(arguments)
12 
13    print()
14    print(' '.join(arg for arg in cmdline))
15 
16    proc = Popen(cmdline, stdout=PIPE, stderr=STDOUT)
17    output = []
18    while True:
19        line = proc.stdout.readline()
20        if not line:
21            break
22 
23        line = line.rstrip()
24        print('docker> %s' % line)
25        output.append(line)
26 
27    proc.wait()
28    return output
29 
30 
31def start_myproject():
32    print('Starting MyProject: %s' % version)
33 
34    docker('network', 'create', '-d', 'bridge', network_name)
35 
36    docker('run', '-d', '--name', mongo, '--network', network_name, '--network-alias', mongo, '-P', 'myrepo/mongo')
37 
38    docker('run', '-d', '--name', myproject, '--network', network_name, '-P', 'myrepo/myproject:%s' % version,
39           '--spring.profiles.active=dev', '--database.hosts=%s' % mongo)
40 
41    docker('port', mongo)
42    docker('port', myproject)
43 
44 
45def destroy_myproject():
46    print('Destroying MyProject: %s' % version)
47 
48    docker('stop', myproject)
49    docker('rm', myproject)
50 
51    docker('stop', mongo)
52    docker('rm', mongo)
53 
54    docker('network', 'rm', network_name)
55 
56 
57args_parser = ArgumentParser(description='Manages MyProject/Docker dev deployments')
58sub_parsers = args_parser.add_subparsers()
59 
60start_parser = sub_parsers.add_parser('start', help='Start MyProject and Mongo')
61start_parser.add_argument('--version', '-v', required=True, help='The MyProject version to start')
62start_parser.set_defaults(func=start_myproject)
63 
64destroy_parser = sub_parsers.add_parser('destroy', help='Destroy MyProject and Mongo')
65destroy_parser.add_argument('--version', '-v', required=True, help='The MyProject version to destroy')
66destroy_parser.set_defaults(func=destroy_myproject)
67 
68args = args_parser.parse_args()
69 
70version = args.version
71network_name = version
72mongo = 'mongo_%s' % version
73myproject = 'myproject_%s' % version
74 
75args.func()

The port for the application is automatically assigned by Docker. The URL is printed to the log of the job and can be quickly accessed from the build pipeline view. The job gets the pull request number as parameter and thus knows which image to start.

The deployment job has yet another manual downstream job for destroying the deployment.

In order to get quick access to application logs or to open a shell on a container without having to ssh into the Jenkins host, we installed Shipyard. This gives a good overview over running containers and allows you to destroy obsolete forgotten ones.

Why no Pipeline Job?

Unfortunately, the Bitbucket Pull Request Builder Plugin  is not compatible with the new Jenkins pipelines. There is a JIRA ticket for this which refers to the Bitbucket Branch Source Plugin . Unfortunately, that’s out of question for us because it uses webhooks. For the master build, however, we do already use a pipeline job.