“The Foreman” – Automatic installation of “The Foreman” with Puppet

This is the second blog post in a series about “The Foreman” , a complete lifecycle management tool for physical and virtual servers. In the first post we have learned what “The Foreman” is made for, how it works and what we can use it for.
Now with the second post we will start to get more hands on. The goal of this post is to show how to install “The Foreman” and supporting services like for example DNS, DHCP automatically through puppet on a bare-metal server.

The network setup used for this post is shown in figure 1. First of all, the network contains a VPN-Gateway connected to the internet. The backend network is secured through a DMZ and consists of two VLANs. On the VLAN 1, “The Foreman” will provide the DHCP-, DNS- and TFTP-Services. Through the network VLAN 2 it is ensured, that all machines can be accessed through the VPN.

Figure 1: Network setup

To provision the Host A and B we need to install “The Foreman” on the one machine called “The Foreman” in figure 1. Therefore we have to do the following preparation work on this machine:

• Installation of operating system.
• Installation of prerequisites.
• Get and run a provided Puppet Manifest to install “The Foreman”.

Now that we know the setup, lets prepare the machine we want to install “The Foreman” on. First of all, we have to install the Ubuntu 12.04 Server Edition. We decided to use Ubuntu 12.04 Server because it comes with long term support and is supported by “The Foreman”.

For this post we assume that you have already installed the OS and know some basics about Puppet. After we have finished the OS installation, we need to run a post installation script which you can see in listing 1. The script will install an openssh server for remote access as well as git. Further, the script creates a folder named git and clones a provided git repository. Thereafter, the script is going to copy a prepared network interface file, through which both network interfaces gets configured. Finally the script installs puppet and provides a prepared puppet configuration file, that will then be used to install the needed services like DHCP and DNS and “The Foreman”.

Listing 1:

1#!/bin/bash
2 
3# Install an openssh server and git
4cd $HOME
5sudo apt-get install openssh-server git
6mkdir -p git
7cd git
8 
9# Checkout the git repository
10if [ ! -d "$HOME/git/foreman-poc" ]; then
11git clone https://github.com/codecentric/foreman-poc.git
12fi
13 
14# Change the branch to bare-metal
15cd foreman-poc
16git checkout bare_metal
17 
18# Prepare the network interfaces
19# You have to change this file depending on your network setup
20sudo cp $HOME/git/foreman-poc/files/System/interfaces /etc/network/
21 
22# Get Puppet debian packages and install Puppet
23wget https://apt.puppetlabs.com/puppetlabs-release-precise.deb
24sudo dpkg -i puppetlabs-release-precise.deb
25sudo apt-get update
26sudo apt-get install --yes puppet
27 
28# Provide a new Puppet configuration, restart the service and get the standard libs
29sudo cp $HOME/git/foreman-poc/files/System/puppet.conf /etc/puppet/
30sudo service puppet restart
31sudo puppet module install --force puppetlabs-stdlib
32 
33rm puppetlabs-release-precise.deb
34 
35# Reboot the system
36sudo reboot

The following commands are needed to download and execute the post installation script:
> wget https://github.com/codecentric/foreman-poc/blob/bare_metal/files/System/post-install.sh > chmod +x post-install.sh > sudo ./post-install.sh
Now we we are ready to install “The Foreman”. However, before we start a local puppet run, lets have a look on the the resources described inside the Puppet Manifest called ‘server.pp’.
We will have a look on some of the important parts of the preparation and installation itself. To get started with the “The Foreman” installation, we first of all need some sources and packages. Therefore we have to provide the apt key and apt sources to be ready to find and install “The Foreman” specific packages, see listing 2.

Listing 2:

1# Function to create a apt key. If called with ensure => present the apt key will be added 
2define aptkey($ensure, $apt_key_url = 'http://deb.theforeman.org') {
3  case $ensure {
4    'present': {
5      exec { "apt-key present $name":
6    command => "/usr/bin/wget -q $apt_key_url/$name -O -|/usr/bin/apt-key add -",
7    unless  => "/usr/bin/apt-key list|/bin/grep -c $name",
8      }
9    }
10    'absent': {
11      exec { "apt-key absent $name":
12    command => "/usr/bin/apt-key del $name",
13    onlyif  => "/usr/bin/apt-key list|/bin/grep -c $name",
14      }
15    }
16    default: {
17      fail "Invalid 'ensure' value '$ensure' for apt::key"
18    }
19  }
20}
21 
22# Creates a file for the apt source
23file {'foremanlist':
24    path	=> '/etc/apt/sources.list.d/foreman.list',
25    ensure	=> present,
26    mode	=> 0644,
27    content	=> 'deb http://deb.theforeman.org/ precise 1.4'
28}
29 
30# Calls the aptkey function above with name => ‘foreman.asc’ and ensure => present 
31aptkey { 'foreman.asc':
32    ensure	=> present
33}
34# Call apt-get update which requires the apt key and source file
35exec { "apt-update":
36    command	=> "/usr/bin/apt-get update",
37    require	=> [
38        Aptkey['foreman.asc'],
39        File['foremanlist'],
40    ]
41}

In a next step, we have to ensure that all needed packages are present – including the foreman-installer, bind9 for dns, isc-dhcp-server and gem, see listing 3.

Listing 3:

1# Ensures that “The Foreman” installer is present
2package { "foreman-installer":
3    ensure	=> "installed",
4    require	=> Exec['apt-update'],
5}
6# Ensures DNS-Server is present
7package { "bind9":
8    ensure	=> "installed",
9    require	=> Exec['apt-update'],
10}
11# Ensures DHCP-Server is present
12package { "isc-dhcp-server":
13    ensure	=> "installed",
14    require	=> Exec['apt-update'],
15}
16# Ensures Gem is present
17package { "gem":
18    ensure => "installed",
19    require => Exec['apt-update'],
20}

Now that we have all packages at hand, lets have a detailed look at the DHCP and DNS configuration.
In listing 4, with the first resource we are placing a ‘rndc.key’ file into the config folder of the DNS-Server. This allows us secured server to server communication by providing the key also to the installation file used by “The Foreman”. The second resource creates a user ‘dhcpd’ and adds it to the group ‘bind’. Now we have to provide a new apparmor config file due to a missing write permission on the folder “etc/bind”, see the resource 4. Next, we have to add a line to the dhclient.conf, to ensure that the correct DNS-Server will be called by the server, resource 5. Finally we have to provide a modified proxydhcp.pp Manifest in which we added the correct path to the ‘rndc.key’ file.

As I am not an expert for linux administration, there are probably better solutions for setting up DNS and DHCP and I would like to see them. So feel free to add a comment below. However the described solution worked for us and we have a stable DHCP-Server which updates the DNS entries.

Listing 4:

1# Placing the keyfile
2file { "/etc/bind/rndc.key":
3    ensure	=> present,
4    source	=> "/home/server/git/foreman-poc/files/BIND/rndc.key",
5    owner	=> root,
6    group	=> bind,
7    mode	=> 640,
8    require	=> Package["bind9"],
9}
10# Adding user 'dhcpd' to group 'bind', as this users needs to read the keyfile
11user { "dhcpd":
12    ensure	=> present,
13    groups	=> ['bind'],
14    require => [
15        Package["isc-dhcp-server"],
16        Package["bind9"],
17    ],
18}
19# Workaround that DHCP can read the keyfile
20# Replace existing DHCPd-apparmor configuration
21service { "apparmor":
22    ensure  => "running",
23    enable  => "true",
24}
25file { "/etc/apparmor.d/usr.sbin.dhcpd":
26    notify  => Service["apparmor"],
27    ensure	=> present,
28    owner	=> root,
29    group	=> root,
30    mode	=> 644,
31    source	=> "/home/server/git/foreman-poc/files/DHCP/apparmor_usr.sbin.dhcpd",
32    require => Package["isc-dhcp-server"],
33}
34# Dhclient: prepend DNS-server
35file_line { 'dhclient':
36    path	=> '/etc/dhcp/dhclient.conf',
37    line	=> 'prepend domain-name-servers 172.16.0.2;',
38    match	=> "prepend domain-name-servers",
39}
40# Modifying foreman-installer to support DDNS
41file { "/usr/share/foreman-installer/modules/foreman_proxy/manifests/proxydhcp.pp":
42    ensure	=> present,
43    source	=> "/vagrant/files/DHCP/proxydhcp.pp",
44    owner	=> root,
45    group	=> root,
46    mode	=> 644,
47    require	=> Package["foreman-installer"],
48}

Finally we need two more things for the installation of “The Foreman”. First, the installation itself should be unattended. Therefore we need a file called ‘answers.yaml’ that includes the answers to question that are normally asked during the installation process. You can see the first resource in listing 5 how to provide the file. Second, the last resource in listing 5 does nothing else than just start the installation process itself.

Listing 5:

1# Options for foreman-installer
2file { "/usr/share/foreman-installer/config/answers.yaml":
3    ensure	=> present,
4    source	=> "/vagrant/files/Foreman/answers.yaml",
5    owner	=> root,
6    group	=> root,
7    mode	=> 600,
8    require	=> Package["foreman-installer"],
9}
10# Installation of foreman
11exec { 'foreman-installer':
12    command	=> "/usr/bin/foreman-installer",
13    timeout => 0,
14    require => [
15        Package["bind9"],
16        File['/usr/share/foreman-installer/modules/foreman_proxy/manifests/proxydhcp.pp'],
17        File['/usr/share/foreman-installer/config/answers.yaml'],
18        File["/etc/bind/rndc.key"],
19    ],
20}

As I mentioned before, for the installation to run unattended, a file called ‘answers.yaml’ is needed. In listing 6 you can see the full ‘answers.yaml’ file used by our installation. The file tells the installation process to install “The Foreman” with a custom environment called ‘cloudbox’, line 3. Environments are used to group Puppet Modules for different kind of hosts. Further environments could be for example production, test or development.
Foreman-Proxies will be installed for TFTP, DHCP and DNS. These Foreman-Proxies will be installed on top of our before installed TFTP-, DNS-, and DHCP-Servers. While installing the Foreman-Proxies we can configure many details like the rndc-key and secret. Finally the listing 6 shows, that we are installing the Puppet-Master with the same environment ‘cloudbox’ as we did before.

Listing 6:

1---
2foreman:
3  environment: cloudbox
4  custom_repo: true
5  oauth_consumer_key: Ls6P7vd3sfv8QZviRTNnPUX2k5RPhTnn
6  oauth_consumer_secret: Uq2Hwyp7kHuSB7YGU3beMXcTKuyA9VaD
7foreman_proxy:
8  custom_repo: true
9  puppetrun: true
10  tftp: true
11  tftp_servername: 172.16.0.2
12  dhcp: true
13  dhcp_managed: true
14  dhcp_interface: eth2
15  dhcp_gateway: 172.16.0.2
16  dhcp_range: 172.16.0.16 172.16.0.255
17  dhcp_nameservers: 172.16.0.2
18  dhcp_key_name: rndc-key
19  dhcp_key_secret: bQR3x3fquV+YjZ+aChpfJQ==
20  dns: true
21  dns_interface: eth2
22  dns_zone: local.cloud
23  dns_reverse: 0.16.172.in-addr.arpa
24  dns_server: 172.16.0.2
25  dns_forwarders: 8.8.8.8
26  foreman_base_url: http://server.local.cloud
27  oauth_consumer_key: Ls6P7vd3sfv8QZviRTNnPUX2k5RPhTnn
28  oauth_consumer_secret: Uq2Hwyp7kHuSB7YGU3beMXcTKuyA9VaD
29puppet:
30  server: true
31  server_environments:
32      - cloudbox

If you like to have a look on the final Puppet Manifest, with the full installation for example the TFTP-Server and net boot images which will be provided for the provisioning, you can check it out using following Link .

After the puppet run succeeded, we have to find out our local IP-address (ifconfig) and open a web browser on our local machine. Now enter the IP-address and don’t forget to put https:// in front. The final address should look like this (https://yourlocalipaddress) and if everything worked out correct you should be able to see the login screen. Enter admin as username and changeme as password and you should be able to see the figure 2.

Figure 2: “The Foreman”: Start screen

Now we have a Foreman-Server, fully installed and configured with DHCP-, DNS-, TFTP- and PXE-Services, ready to use.

So, thats it for this blog post. I hope you enjoyed the second post. Now lets recap what we have done up to now. On a bare-metal machine we have installed an OS. Afterwards, we downloaded a post installation script which prepared our machine to run puppet and git. The preparation script has cloned a repository from github and started a local puppet run. This puppet run has automatically installed a DNS-, DHCP- and TFTP-Server as well as “The Foreman” itself. We had a closer look on the Puppet Manifest for the installation of “The Foreman” itself and the installation was again unattended through the use of a file called ‘answers.yaml’. Finally we checked if the installation was successful.
If you enjoyed what you have seen up to now. Get a feeling for “The Foremans” web user interface and remember about the chapters “Provisioning setup” and “Provision a host” from the first blog post and try to provision a host on your own. You should also check out the following blog post, which describes the Automatic Provisioning of a Hadoop Cluster on Bare Metal with The Foreman and Puppet . I will continue the blog series with a third post in which I will show how to use “The Foreman” API to configure “The Foreman” automatically.

Authors

Felix Massem and Jan-Frederic Markert