Continuous Delivery in the Cloud – Part 2: Getting started with Amazon EC2

In this article I will give you an overview of the Amazon WebServices (AWS) that I am using for the demo. All you need to get started is a credit card & phone number for the AWS registration. If you are already familiar with Amazon EC2, go ahead and jump to part 3.

• Part 1: Continuous Delivery Overview
• Part 2: Getting started with Amazon EC2
• Part 3: Configure your Continuous Integration Server
• Part 4: Provisioning your Test, Staging and Production environments
• Part 5: Configure your Continuous Delivery Pipeline
• Part 6: Create a Dashboard of your System

Introduction

Amazon offers a wide range of cloud services starting with elastic computing, content delivery, storage services, networking and many more. Most services are offered on pay-as-you-go basis. To get you started Amazon offers several services for free for the first year as long as you stay below a monthly threshold. For this demo I will concentrate on the EC2 elastic compute service. It offers everything we need. For a full list of the AWS product portfolio have a look at http://aws.amazon.com/en/products .

Common questions

I will cover the most common questions concerning AWS to get you quickly up and running with your first AWS instance.

Is it safe?

I am not going to talk about security policies, privacy laws, the patriot act or any of that stuff in this article. You need to verify yourself if your company allows you to store software and data in the cloud. Nevertheless, I will talk about security aspects of the server instances that are created, so you know what can be done in order to protect your cloud instances.

When can I use the cloud?

We use cloud services for training courses, i.e. when we need a lot of servers for a short period of time. They are useful if you need computing power and your hardware vendor has put you on a long waiting list because server or networking parts are somewhere on a ship between you and the manufacturer. You can set up complex testing environments in a breeze without having to buy additional hardware. You can use the AWS cloud services to run all of your test and production servers (including the continuous delivery pipeline). And the best part is you know exactly what running your system costs.

Who uses Amazon EC2 services?

What is better than the typical marketing buzzwords? A real world example.

Ask yourself:

• Can I build and host a web application without any hardware?
• … with a couple of engineers?
• … have more than a million users?
• … and scale on demand to grow even bigger?

If you think this is impossible, you might have heard of a little company called Instagram 🙂

Here are their specs:

• 3 engineers
• over 14 million users
• no hardware
• +100 instances on Amazon EC2

Another nice fact: Instagram was just bought for 1 billion USD by Facebook in march 2012 [INS] . Nice.

How much does it cost?

As part of the AWS free usage tier , you can get started and try out the services for free.
Here is an excerpt of the regular prices once you are starting to get serious.

What do I get for my money?

The following table lists hardware details for the typical instance types. There is a lot more you can get from Amazon. This is just an excerpt.

For the demo I am using the Medium instance type for the central node. Since we will run Jenkins, Artifactory and Sonar on a single machine, we need a little more memory. I prefer good I/O performance so that Jenkins jobs perform nicely.
All test, user acceptance and production environments use the micro instance type. They are just used to run a Tomcat webserver with the web application and a MySQL database. That is where the AWS free usage tier comes in handy for the demo.

Let’s get it started

Registration

To register for the Amazon WebServices (AWS) go to http://aws.amazon.com/ , create an account and follow the registration steps. You need to provide your credit card information and a phone number during the registration process. An automatic service will call the provided number and require you to enter a PIN via the dial pad. The credit card is used for charging you once you start using services that cost money. As long as you do not have any server instances running, you do not pay anything.

Once you are registered you are ready to go. On the start page select the “AWS Management Console” from the top right and sign in with your user credentials.

Dashboard

The central point of information is the AWS dashboard. If you log in for the first time, I suggest you set the Start Page from the dropdown list to EC2. In the center of the dashboard you will find a list of Amazon Web Services, on the right side Latest Announcements and below that you can configure which system health checks you want to monitor.

AWS is split into four main regions: North America, South America, Europe and Asia Pacific. This is where your instances will be hosted. There are minor price differences that you can ignore for now. What is more important for you to decide is which region you should use to host your cloud service. For a production system I suggest a region that is closest to your customers (or multiple regions). For our demo I selected “US East (N. Virginia)” since its the cheapest region and latency is not a factor in our use case.

In the Service Health section, you can configure which AWS service and region you want to have displayed on the dashboard. Below that section is a fully-featured Service Health Dashboard.

Create your first EC2 Instance

Lets create your first instance. I will explain the different options and settings of the Dashboard along the way. In the middle of the EC2 dashboard select Launch Instance. In the next window give your instance a meaningful name, i.e. appserver. After that create a new key pair and click the download button. Keep this key (appserver.pem) in a safe place. You will need it later on to login to your new instance. In the Launch Configuration section, you have to select an Amazon Machine Image (AMI). Amazon provides a lot of different Linux and Windows images. For our demo you are good with the 32bit Amazon Linux AMI 2012.03. When you are done click continue.

On the next step you can edit the configuration details in case you want to make any changes, i.e. choose a different instance type size. In our case you can select the launch button. This will trigger AWS to create a new instance for you with your configuration settings. Once the instance is starting it will be added to the EC2 dashboard. At first the status will be pending until it is fully started.

If you right-click on the new instance you get a context menu with actions for Instance Management, Instance Lifecycle and CloudWatch Monitoring. Selecting the Connect link will give you the login information. Every time you stop and restart an instance it will be assigned a new IP address and login url.

Now that you know the login URL you can use a ssh client to login to your running AWS instance using the key pair you have downloaded before. Depending on the operating system you selected, the login user tends to be differert. Ubuntu images mostly use ubuntu, CentOS and the Amazon Linux AMI images use ec2-user. Make sure you check the correct login details in the connect section, if you run into problems.

# login to AWS instance ssh -i appserver.pem ec2-user@ec2-184-72-81-216.compute-1.amazonaws.com

# update system through the package manager
sudo yum update

Done. Now you are now free to do whatever you want with your EC2 instance.

Security

When we started the EC2 instance it was assigned to a default security group that only allows port 22 (ssh) to be opened to the outside world. If you select Security Groups from the navigation menu you can select which port range should be open on the instance. You can create a placement group that specifies which groups can communicate with each other. This is a common way to control access to your instances. You can configure Network Interfaces and Loadbalancers for your instances.

Elastic IP

If you want a static IP address for your instance, you can assign an elastic IP to your instance through the dashboard. Elastic IP addresses belong to the account and can be assigned to any instance. Once you have registered an elastic ip you need to assign it to an instance, otherwise it will cost 1 cent/hour while it is unassigned.

EC2 API tools

Another important topic I need to cover are the EC2 API tools. Everthing I have done through the web interface can be done using EC2 API tools. The scripts serve as the client interface to the EC2 web service. Use these tools to register and launch instances, manipulate security groups, tagging, terminate instances, and more.

You can either install them locally on your machine or use the EC2 tools on the new AWS instance, which comes preconfigured with the EC2 API tools. Have a look at the following blog on how to configure the EC2 API tools [EC2] .

Here are a couple of useful commands:

For a list of all EC2 API Tool commands please refer to the official documentation at [REF]

Screencast

I prepared a screencast that shows how to create an EC2 instance. Screencast: Create your first EC2 instance

Summary

I hope I have given you a good overview how AWS works and how to get your own AWS instance up and running. Now that I covered the basics you are ready for part 3, Configure your Continuous Integration Server. Stay tuned …

References

[INS] Facebook Instagram deal
[EC2] Configure EC2 API Tools
[REF] AWS Command Line Reference