Will AWS CDK replace Terraform and the Serverless Framework?

This is a post about infrastructure management with code for AWS serverless projects. However, much of the findings can be applied to more generic cloud management as well. Recently I got the opportunity to work with the Serverless Framework, Terraform and AWS’s CDK in the same month. It involved two new projects and one conversion. All projects are serverless in nature and had Lambda execution at its core, but also involved some additional non-serverless cloud management. It provided for a nice opportunity to compare them and share my experiences. In the past four years the Serverless Framework began as your average go-to tool for Lambda deployments. Terraform proved to be a more feature-rich tool, and since 2018 there is the AWS CDK which promises to be a developer-friendly cloud-management solution (for AWS). I will discuss my journey with the three technologies first, and provide a summarised overview in the final chapter. Even though two of these tools are cloud-independent, I have assessed their fitness in managing AWS resources.

First a bit of background on these projects. In three projects we are using Lambda as the main business logic execution processor. There is one data-processing project which uses S3, RDS, and Quicksight for reporting. One API project that uses API Gateway, Cognito, Dynamo and one web-project using Dynamo, API gateway, S3, SQS, SNS, Cognito. While all three are built primarily on serverless building blocks, there are also more infrastructural resources in each project (not related to a single Lambda), to build up the necessary groundwork (VPC, IAM, Cognito, S3 policies and buckets).

Serverless Framework experiences

In the past four years the Serverless Framework kickstarted Lambda deployment automation. In the early days of Lambda some people were experimenting in the online AWS Console IDE and after some first experimentation people were looking for tools that made Lambda development easy. The Serverless Framework was really ahead of the game in presenting an easy configuration file (the serverless.yml) in which you could declare your configuration. Using only this file and another file comprising the code you could spin up a Lambda and API gateway in seconds.

An example of declaring the infrastructure for Lambda with the Serverless Framework is depicted as follows:

1service: serverless-simple-http-endpoint
2provider:
3  name: aws
4  runtime: nodejs8.10
5
6functions:
7  currentTime:
8    handler: handler.endpoint
9    events:
10    - http:
11      path: ping
12      method: get

In time the Serverless Framework expanded its offering by providing some sort of control plane / dashboard with additional services like security, compliance checking, logs, metrics and others. The merits of these added services are not evaluated in this article.

The Serverless Framework will coordinate the building, and packaging of your source code on deployment. It will generate a CloudFormation template and call the CloudFormation executor to kick-off a fresh deploy. Afterwards CloudFormation will take care of the actual deployment.

What’s great about the Serverless Framework?

I had a great time using the Serverless Framework. If you are starting a new project, you can easily spin up some Lambdas. It has support for some well known AWS resources Lamba interact with (like: S3, SQS). Also, it has an active plugin ecosystem where third parties contribute to expand the tooling support. If you run into problems, you are usually not alone, and in practice most problems and solutions are well described on Stack Overflow.

What are the limitations of the Serverless Framework?

The Serverless Framework is clearly built to support the deployment of Lambdas, which limits its usefulness in managing your complete cloud setup. You can add CloudFormation scripts to expand its reach, but in that case you are just running CloudFormation instead of Serverless Framework. In each project I like to script my entire cloud account, which in practice forces me to use a different tool for general cloud sculpting and Serverless for my Lambdas. Switching between both is a hassle and it messes up the attribution of resources (like getting the proper ARNs to connect stuff).

Also for resources which are part of generic Lambda use-cases, like attaching a bucket to a Lambda, the Serverless Framework already falls short. In many cases you don’t want a lifecycle-dependency from a Lambda to a bucket. In enterprise grade setups you want a bucket to have complex life cycle policies, access controls, logging, etc. Afterwards you might need some Lambas to read it, some to write to it, and one to trigger on write (buckets only allow one!). I would say these requirements are beyond a sensible scope of using Serverless Framework.

Terraform experiences

This quote on the Terraform website says it all: “Use Infrastructure as Code to provision and manage any cloud, infrastructure, or service”.
Terraform is a declarative syntax and command line executor that performs cloud management through a native CLI bridge.

An example of declaring the infrastructure for Lambda with Terraform is depicted as follows:

1provider "aws" {
2  region = "us-east-1"
3}
4 
5resource "aws_lambda_function" "example" {
6  function_name = "ServerlessExample"
7 
8  # The bucket name as created earlier with "aws s3api create-bucket"
9  s3_bucket = "terraform-serverless-example"
10  s3_key = "v1.0.0/example.zip"
11 
12  # "main" is the filename within the zip file (main.js) and "handler"
13  # is the name of the property under which the handler function was
14  # exported in that file.
15  handler = "main.handler"
16  runtime = "nodejs10.x"
17 
18  role = "${aws_iam_role.lambda_exec.arn}"
19}
20 
21# IAM role which dictates what other AWS services the Lambda function
22# may access.
23resource "aws_iam_role" "lambda_exec" {
24  name = "serverless_example_lambda"
25 
26  assume_role_policy = 
27  <<EOF
28  {
29    "Version": "2012-10-17",
30    "Statement": [
31       {
32        "Action": "sts:AssumeRole",
33        "Principal": {
34        "Service": "lambda.amazonaws.com"
35       },
36       "Effect": "Allow",
37       "Sid": ""
38     }
39    ]
40  }
41  EOF
42}
43 
44resource "aws_api_gateway_rest_api" "example" {
45  name = "ServerlessExample"
46  description = "Terraform Serverless Application Example"
47}
48 
49resource "aws_api_gateway_resource" "proxy" {
50  rest_api_id = "${aws_api_gateway_rest_api.example.id}"
51  parent_id = "${aws_api_gateway_rest_api.example.root_resource_id}"
52  path_part = "{proxy+}"
53}
54 
55resource "aws_api_gateway_method" "proxy" {
56  rest_api_id = "${aws_api_gateway_rest_api.example.id}"
57  resource_id = "${aws_api_gateway_resource.proxy.id}"
58  http_method = "ANY"
59  authorization = "NONE"
60}
61 
62resource "aws_api_gateway_integration" "lambda" {
63  rest_api_id = "${aws_api_gateway_rest_api.example.id}"
64  resource_id = "${aws_api_gateway_method.proxy.resource_id}"
65  http_method = "${aws_api_gateway_method.proxy.http_method}"
66 
67  integration_http_method = "POST"
68  type = "AWS_PROXY"
69  uri = "${aws_lambda_function.example.invoke_arn}"
70}

It offers a cloud-vendor-independent syntax, which, through the use of cloud-specific providers, offers complete administrative access to many cloud vendors and services like Heroku or Mailgun.

On deployment Terraform queries the existing resources in the cloud, creates a plan which describes the transformation to the desired end-state, and sequentially executes the necessary changes in the proper order.

What’s great about Terraform?

In terms of AWS resource support it’s great! I have yet to find resources or specific parameters that I could not control properly with Terraform. I have done Terraform setups comprising of complete IAM, billing, account, and security settings.

The framework has an active development team and community, is stable, and extendable. People are writing their own providers when they feel like the API is uncomfortable, and with modules people make specific use-case modules which provide nice abstractions for bundles of resources (like a Lambda + API gateway for a Web API).

The speed of migrations in Terraform is really fast. Migrations in Terraform are a sequence of CLI instructions which are not performed transactionally. If some error happens during a migration it will abort the execution of subsequent steps. Rerunning the same script will create a new delta for the changes which are remaining, and executes these.

What are the limitations of Terraform?

I love Terraform, but I have some beef with the security model. In a simple straightforward setup you can have your Terraform scripts in your favorite containerised CI being executed against your favorite cloud. However you need to supply the Terraform executor loads of IAM permissions, which is a potential security risk. Alternatively in a more complex setup you could execute these Terraform scripts from within your cloud account by running it on a VM (or for example a Gitlab runner), but this requires quite some work and adds fixed costs.

Terraform with its JSON-like structure is really bloated. Most modules now support defaults, which keeps you from writing out all parameters, but it’s still heavy on the details. You can immediately see this if you compare the examples.

Furthermore migrations are not transactional, which can lead to weird infrastructure states which you need to manually solve by either importing or deleting resources and rerunning a deploy.

AWS CDK experiences

CDK is the new kid on the block, introduced at Re:Invent 2018. It promises to be a developer-friendly abstraction to manage cloud infrastructure as code in your everyday programming language. Meanwhile it supports an impressive set of languages: Typescript, Javascript, Python, Java, and C#.

An example of declaring the infrastructure for Lambda with CDK is depicted as follows:

1const getOneLambda = new lambda.Function(this, 'getOneItemFunction', {
2  code: new lambda.AssetCode('src'),
3  handler: 'get-one.handler',
4  runtime: lambda.Runtime.NODEJS_8_10,
5  environment: {
6    TABLE_NAME: dynamoTable.tableName,
7    PRIMARY_KEY: 'itemId'
8  }
9});
10 
11const api = new apigateway.RestApi(this, 'itemsApi', {
12   restApiName: 'Items Service'
13});
14 
15const singleItem = items.addResource('{id}');
16const getOneIntegration = new apigateway.LambdaIntegration(getOneLambda);
17singleItem.addMethod('GET', getOneIntegration);

On deployment the code where you defined your infrastructure gets compiled and ‘synthesized’ to CloudFormation. In turn CloudFormation will transactionally perform the actual deployment.

What’s great about CDK?

AWS shows dedication by already releasing the CDK in a lot of languages. The fact that you can write your deployment in the same language as your Lambda is a nice idea (less switching). The relative ease for DevOps teams to create their own packages and libraries for controlling infrastructure is great though. Without doubt this technology has the ability to foster a great ecosystem of community built CDK extensions (called constructs).

You will get all benefits from IDE support like highlighting, syntax checking, inspections, refactoring support (big plus!!!), and testability. This makes for a compelling argument to use the CDK.

What are the limitations of CDK?

CDK programming still felt much like using Typescript (in my case) as a DSL for declaring resources instead of actual programming. While all the fancy programming possibilities are there, but the fact that you are declaring your infrastructure state pushes code to look like a declarative specification.

At this point, my biggest complaint is with the documentation. Managing common resources like RDS or even VPC management took days to get right. API’s are still fluent, closed Stack Overflow tickets are often outdated, and I often find myself staring at an open pull request. CDK is by no means a steady project at the time of writing.

You can clearly see that AWS’s vision on CDK is one to satisfy developers. They create a new abstraction through CDK constructs that capture one or more resources. At compile time they translate these constructs to Cloudformation scripts. Because of my previous experience with AWS console, AWS CLI, Terraform and Cloudformation I often felt lost in interpreting these constructs. The CDK developers really wanted to make a properly typed library which is easy to use, but in the process they added lots of constructs that do not one-on-one map on thing I know from my previous experiences. The good thing is that – using your IDE – you can just click through the underlying library to see what’s happening.

Comparing CDK, Terraform, and the Serverless Framework

Based on the findings I tried to capture my experience in the table below. Let’s make one thing clear: all three tools are perfect for deploying Lambdas in AWS! But in reality deploying a single Lambda is never good enough in actual software projects. For each framework I assessed my experience and provided a + whenever I was comparatively positive or negative relative to the other candidates.

Based on this table, there is not a single winner here. It’s really contextual to your project and organisation what tool you might select. If you want me to consider an additional factor, please drop me a comment or message. I am more than happy to add it.

Should you use AWS CDK on your next serverless project?

All three tools are perfect for deploying Lambdas in AWS! It’s managing all the other resources that will make your life either miserable or productive. I would consider using Serverless Framework for smaller projects, PoC’s and side projects only. It’s an ideal tool to quickly bootstrap a project. After some idle months you (and your colleagues) are still able to read and modify your cloud setup.

From a developer’s perspective I love Terraform because it has really solid documentation and non-transactional migrations are quick to perform and easy to understand. From an operations perspective CloudFormation’s transactional stack changes are – while slow – more reliable compared to Terraform. However, you will be able to fix a mistake in Terraform way more rapidly compared to a CloudFormation migration. The downside is that it requires a really complicated CI/CD setup if you want to make Terraform migrations secure.

AWS CDK has the promise to be in a sweet spot of using CloudFormation’s mature and secure migration coordination, while being developer friendly because you can write deployments with code. The concept of CDK constructs will make you pull your hair out in the first week, but this will grow on you. While CDK looks promising I would not recommend using it in enterprise projects just yet. This is because of one reason: the maturity of the API & documentation. I am not being conservative. There are plenty of alpha-/beta-level projects out there that I am more than willing to use, and I am not looking for the world’s most steady API, all I am asking for is more accurate documentation and proper examples. I suggest to check up on this Github sample listing regularly and make sure your use-case is covered.

CDK has a promising future

Once you actually get the proper configuration (or should I say code) right it’s a really nice experience. But at this point, every new AWS service takes us at least four times as much effort compared to Terraform. Also, the CDK deploys we rigged now will likely break in the weeks or months to come. So be prepared to update your CDK deploys in the near future!

Ultimately, using CDK is still very promising. A proper develop experience is one of the most foundational requirements for setting up new projects these days. I will keep using CDK in the few projects where I used it now and see what happens in the months to come. I hope to write a ‘after six months experience blog’ in some time.

If you want to get started with AWS CDK, I can recommend this CDK workshop .

Want to read more about generic serverless develop experience in this article.