AWS CDK Part 6: Lessons learned

In this blog post we will focus on reflecting on our AWS CDK experience during one of our projects where we had to set up a new infrastructure for one of our customers. We will address the issues of version iterations within the library, what we deemed a bad developer experience and why that is and, of course, also what we loved about CDK. Those are our lessons learned.

In Part 1 to 5 we created a custom VPC, an S3 bucket, an RDS instance, Lambdas, and coordinated these with step functions. If you haven’t read them yet, here is what we have dealt with in the past blog posts:

• How to create a custom VPC
• How to create S3 buckets
• How to create an RDS instance
• How to create Lambdas
• How to create a step function
• CDK lessons learned

At this point we have successfully completed our application and architecture setup depicted in the graphic below.

Lessons learned – The good

A real developer experience

We enjoyed the TypeScript API very much. TypeScript seems to be in the sweet spot of having a nice typing system, but still offers the possibility to opt out when needed. We really felt this hybrid typing was handy in a lot of cases. TypeScript is a joy to work with. Your CDK scripts do need to be compiled using ‘tsc’, a step that’s easy to forget and sometimes leads to unexpected CDK behaviour. However, the tsc watch feature ‘tsc -w’, will save you some trouble by automatically recompiling on change. We also like the overview of our infrastructure provided by CDK scripts.

Easy CDK bootstrapping

After having completed a few CDK projects, you will have a lot of reusable snippets for your day-to-day infrastructure coding (networking, virtual machines, Lambdas, etc.). With a bit of experience you can set up a new project in minutes and spin up elaborate environments in hours. Your platform team will be able to provide nice libraries to support development teams.

Continuous improvement of the CDK library

We equally listed this aspect under the ugly, but where there is shadow, there must be light. The rapid iterations in versions and immense changes in CDKs API over the last three months is impressive and promising for the future. AWS is serious in developing a contender to Terraform and its siblings. CDK is a powerful extension to Cloudformation and we loved the experience of coding our infrastructure and application logic in one breath.

Lessons learned – The Bad

Lack of proper documentation

In the course of three months we came across lots of documentation inconsistencies, which are mostly caused by rapid succession of new versions with incompatible APIs. These rapid API successions are also reflected in the many outdated examples on StackOverflow. Most elements are properly documented, but in practice you often need to combine very deeply nested structures (like passing the IP CIDR ranges of a security group to an RDS’s connection structure). You will not find many practical examples relating to that topic.

Freedom of “overengineering”

TypeScript as a language might offer too much freedom, which can lead to very complicated infrastructure development. While code might look understandable in the moment of writing it, it is highly probable that infrastructure code is not touched by a team on a weekly basis. In general we recommend to stick to simple procedural building blocks as much as possible.

Hard to find proper IAM policies

During our project we set out to create a least-privileged user that would be assigned as the CloudFormation executor. From no attached policies to all the necessary ones, which allow for synthesis free of errors, we went through some cumbersome back and forth of deploying to Cloudformation, getting Unauthorised error messages and converting them into policies. In the end, setting up a least privileged user just with Cloudformation and CDK is quite a bit of work. To our knowledge, there are no infrastructure libraries that currently support a generation of needed policies. In order to facilitate a secure infrastructure platform setup, such a library is definitely desirable and will imply a huge time saving factor.

Defaults are not obvious

During the course of our project we ran into quite a few cases where certain resources within CDK come with default values for parameters. And if they are not explicitly overwritten, they will reach your AWS environment, possibly leading to unwanted behaviour in the infrastructure. While it is definitely a nice tweak to ensure fundamental functionality for the unexperienced CDK users that simply want to experiment with some infrastructure, we would definitely prefer the removal of default values and much rather configure the complete resource ourselves to avoid any accidental misconfiguration of the AWS environment. Especially, because most of the default parameters can only be discovered by reading through the source code itself.

Lessons learned – The Ugly

Rapid CDK dependency version iterations

During the time window of our blog series we have experienced eight minor version changes, today we are already at ten. We started out with 1.7.0 and ended up using 1.15.0 over the course of three months. One would assume that with minor version upgrades, backwards API compatibility is provided. However, we ran into issues more than once after upgrading to the newest version. The code simply did not compile any longer, previously existing library features were completely missing and types changed.

For CDK to become a serious contender to other libraries such as Terraform, we definitely would like to see bigger release cycles and for sure backwards compatibility in minor version upgrades (semantic versioning would be nice). In the current state CDK is definitely a nice toy to play with but in our opinion it is not a library to be recommended for production use YET. We are definitely looking forward to a more stable state of the CDK library.

Cloudformation ‘legacy’

CDK has been built on top of Cloudformation. Cloudformation is a proven product to support your infrastructure migrations. We found the CDK API heavily affected by Cloudformation intricacies. This came as a bit of a disappointment to us. After all, especially when getting to more difficult infrastructure, that constitute more complex resource wiring, you may find it hard to understand CDK stack constructs and also wire them correctly to each other.

Conclusion

To put everything into a nutshell, the non compatible version iterations have cost us minor inconveniences, but usually the code is updated and back up and running within the hour. Documentation and code examples are not always up-to-date, so do expect to have to figure out some issues by yourself every once in a while. And although the Cloudformation legacy is not properly abstracted away, we would still prefer the CDK library over simple Cloudformation template coding or Terraform for that matter, which lacks the typing mechanism of TypeScript. All in all, AWS has introduced a promising library to the market that allows us to migrate infrastructure within AWS on a code basis, and we are looking forward to future versions. We will definitely come back with a follow-up blog post on how CDK evolves in the next few months and years.