Provisioning of Java web applications using Chef, VirtualBox and Vagrant

Question: I want to use virtualization and configuration management to automate deployment of a Java web application and infrastructure provisioning. How can I try it real quick and set up someting I can reuse later?

Answer: try it locally. You need Chef for configuration management, VirtualBox for virtualization and Vagrant to wire them together in a seamless way.

Short description: the following describes how to set up all these components in order to deploy a very simple web application to a Tomcat in a dynamic topology of virtualized servers. All necessary infrastructure components including the JDK, Tomcat, network configuration etc. will be automaticly provisioned to every new server. The server itself will be brought up from a minimal baseline image.

Components:

Ubuntu 11.10 64bit: I use it on my notebook. Feel free to try it in another environment

VirtualBox: I didn’t use the one you can find in Ubuntu’s repositories. Instead, I used the original Oracle version from https://www.virtualbox.org/

Chef: http://wiki.opscode.com/display/chef/Installing+Chef+Server+on+Debian+or+Ubuntu+using+Packages describes how to set up a private Chef. I didn’t go for Chef solo since I wanted to reuse my expetiment later and thus to see how I can set up the whole thing.

chef-server-webui is likely to fail starting, so you can work around this issue using instructions from https://github.com/CoreMedia/chef-bootstrap

Apache2: you will need one so Chef can download you WAR file somewhere. You can use whatever HTTP server you like though

Init scripts: I didn’t want Chef components as well as Apache to come up each time I boot my machine which would be their default installation options. So I just disabled all relevant startup scripts on boot:

1sudo update-rc.d -f couchdb remove
2sudo update-rc.d -f rabbitmq-server remove
3sudo update-rc.d -f chef-server remove
4...

and  wrote two simple start/stop-all scripts that I can call on demand:

1#!/bin/sh
2 
3#start
4 
5/etc/init.d/couchdb start
6/etc/init.d/rabbitmq-server start
7/etc/init.d/chef-server start
8/etc/init.d/chef-server-webui start
9/etc/init.d/chef-expander start
10/etc/init.d/chef-solr start
11/etc/init.d/chef-client start
12/etc/init.d/apache2 start

1#!/bin/sh
2 
3#stop
4 
5/etc/init.d/apache2 stop
6/etc/init.d/chef-client stop
7/etc/init.d/chef-solr stop
8/etc/init.d/chef-expander stop
9/etc/init.d/chef-server-webui stop
10/etc/init.d/chef-server stop
11/etc/init.d/rabbitmq-server stop
12/etc/init.d/couchdb stop

Vagrant: here it’s a little bit more tricky. Most examples I’ve found describe how to set up Vagrant upon Chef solo. So I had to tinker around a little to teach it work with my private Chef.

A good starting point is Vagrant’s web site, where they describe first steps needed to set up this thing: http://vagrantup.com/docs/getting-started/index.html

Then you would need a Vagrantfile where you would configure your whole virtual environment and how to provision components to its virtual servers using your private Chef. In order to create such a file from scratch, you would have to Google a lot and maybe stumble upon these pages:

http://vagrantup.com/docs/getting-started/provisioning.html  (Vagrant and Chef howto)

http://vagrantup.com/docs/provisioners/chef_server.html (Vagrant and Chef howto – a deeper one)

http://wiki.opscode.com/display/chef/Vagrant (Opscode’s howto for Vagrant and hosted Chef)

http://basho.com/blog/technical/2011/02/04/creating-a-local-riak-cluster-with-vagrant-and-chef/  (Basho’s Vagrant + Chef Riak cluster howto)

https://gist.github.com/812241  (Susan Potter’s corresponding gist)

http://vagrantup.com/docs/vagrantfile.html (Vagrantfile documentation)

http://rubydoc.info/github/mitchellh/vagrant/master  (Vagrant’s Ruby documentation)

or you just keep it simple and start off with mine:

1# -*- mode: ruby -*-
2# vi: set ft=ruby :
3 
4Vagrant::Config.run do |config|
5  config.vm.box = "lucid32"
6 
7  (1..2).each do |index|
8    config.vm.define "n#{index}".to_sym do |cfg|
9      cfg.vm.host_name = "n#{index}"
10      cfg.vm.network("192.168.100.1#{index}")
11      cfg.vm.forward_port "ssh", 22, "220#{index}".to_i
12      cfg.vm.customize do |vm|
13        vm.name = "Node #{index} VM"
14        vm.memory_size = 512
15      end
16 
17      cfg.vm.provision :chef_client do |chef|
18        chef.chef_server_url = "http://192.168.100.1:4000"
19        chef.validation_key_path = "/etc/chef/validation.pem"
20        chef.node_name = "n#{index}"
21        chef.add_role :webcontainer
22        chef.add_role :test
23      end
24    end
25  end
26end

What happens here? Basically, I bring up two VMs based on the Vagrant baseline and dynamically configure some parameters for them: network, ports, name, memory etc. The host of your VMs will also have a virtual network with the corresponding netmask, so you need to provide the Chef server url based on this virtual network. The clients running on the VMs will be able to contact the server then. Some security stuff, and that’s almost it.

In the end, I assign every new VM two Chef roles: webcontainer and test. The first one describes all recipes for the middleware components, the last one is the role for the applications itself.

Putting it all together: Now we need to configure Chef to know about all that. First, you need a bunch of cookbooks to be able to run a java program on a VM. You get the cookbooks from the Opscode page: http://community.opscode.com/ . You can get the basic cookbooks list from this page: https://github.com/opscode/java-quick-start  (though for this example you won’t need database and haproxy). Furthermore, you need to get some additional ones. Search for a cookbook, download and unpack it into your Chef repository (you first need to get yourself one: http://wiki.opscode.com/display/chef/Chef+Repository). Here is my list:

1$ ls -la
2total 124
3drwxr-xr-x 21 pb root 4096 2011-12-12 16:01 .
4drwxr-xr-x  9 pb root 4096 2011-12-11 16:30 ..
5drwxr-xr-x  7 pb pb   4096 2011-12-10 01:25 apache2
6drwxr-xr-x  4 pb pb   4096 2011-07-12 22:28 application
7drwxr-xr-x  6 pb pb   4096 2011-12-11 17:18 apt
8drwxr-xr-x  3 pb pb   4096 2011-04-20 04:39 build-essential
9-rw-r--r--  1 pb pb     82 2010-07-31 17:14 ._.DS_Store
10-rw-r--r--  1 pb pb   6148 2010-07-31 17:14 .DS_Store
11drwxr-xr-x  7 pb pb   4096 2011-02-18 04:21 gunicorn
12drwxr-xr-x  5 pb pb   4096 2011-07-06 17:09 hosts-awareness
13drwxr-xr-x  5 pb pb   4096 2011-03-29 01:10 java
14drwxr-xr-x  5 pb pb   4096 2010-11-18 21:01 jpackage
15drwxr-xr-x  6 pb pb   4096 2011-11-29 02:50 mysql
16drwxr-xr-x  4 pb pb   4096 2011-06-04 04:50 openssl
17drwxr-xr-x  5 pb pb   4096 2011-12-10 01:26 passenger_apache2
18drwxr-xr-x  7 pb pb   4096 2011-04-23 05:02 php
19drwxr-xr-x  6 pb pb   4096 2011-03-17 18:55 python
20-rw-r--r--  1 pb root 3064 2011-12-09 17:10 README.md
21drwxr-xr-x  7 pb pb   4096 2011-01-16 06:50 runit
22drwxr-xr-x  5 pb pb   4096 2010-12-10 18:36 tomcat
23drwxr-xr-x  8 pb pb   4096 2011-01-16 06:47 tomcat6
24drwxr-xr-x  5 pb pb   4096 2011-05-31 08:36 unicorn
25drwxr-xr-x  4 pb pb   4096 2010-07-31 17:16 vagrant_main
26drwxr-xr-x  3 pb pb   4096 2011-06-04 06:01 xml

Most of them come as dependencies. The original “application” cookbook will force you to set up all the database stuff. In my case, you don’t need it, and I’ve provided a modified application cookbook in my clone on github: https://github.com/pavlobaron/cookbooks/tree/pb_experiments .

Now to roles. They are pretty simple. I keep the test role in an external file:

1name "test"
2description "test app role"
3run_list(
4  "recipe[application]"
5)

The webcontainer role is being configured using knife:

1{
2  "name": "webcontainer",
3  "default_attributes": {
4  },
5  "json_class": "Chef::Role",
6  "env_run_lists": {
7  },
8  "run_list": [
9    "recipe[apt]",
10    "recipe[java]",
11    "recipe[tomcat]",
12    "recipe[application]"
13  ],
14  "description": "Virtualized Web Container",
15  "chef_type": "role",
16  "override_attributes": {
17    "java": {
18      "install_flavor": "sun"
19    }
20  }
21}

Here, you see the relevant recipes in the correct and logical order. Since the virtualized Ubuntu is real basic, we even need to provision apt there. After that, we put the Sun JDK there, followed by tomcat.

The “magic” behind the scenes is that the application cookbook assumes that there are data bags in the app folder. Mine looks like that:

1{
2  "id": "test",
3    "server_roles": [
4      "test"
5      ],
6    "type": {
7      "test": [
8        "java_webapp",
9        "tomcat"
10        ]
11    },
12    "war": {
13      "_default": {
14        "source": "http://192.168.100.1/files/test.war",
15        "checksum": "test"
16      }
17    },
18    "deploy_to": "/srv/test",
19    "owner": "nobody",
20    "group": "nogroup"
21}

This data bag wires application artifacts to our test role. I have only one artifact: a WAR file (you can create an own one or use the one attached to this post: test ). And here comes Apache into play: mine delivers files contained in “/files”. Whatever you use therefor, it must speak HTTP. “checksum” has been hacked in my clone since I just didn’t want to compute it. There sure is a clean solution for my hack, and maybe I will do it one day.

That’s it. This blog post doesn’t describe how to work with Chef – you definitely should learn its concepts and how to manage it. Btw. Chef: there is a little bug (or a feature): once the virtual clients/nodes have been registered with Chef, you will not be able to register them again. You explicitly need to delete them from the Chef configuration before restarting Vagrant. For this purpose, I’ve created a little shell script:

1#!/bin/sh
2 
3cd /home/pb/vagrant
4sudo vagrant destroy
5 
6for i in 1 2
7do
8  knife node delete "n$i" -y
9  knife client delete "n$i" -y
10done
11sudo vagrant up

Chances are that my setup isn’t optimal and could be done in a more elegant way. I appreciate every feedback.