Development Containers & GitHub Codespaces kill the “works on my machine” problem

We love them, and hate them at the same time: local development environments. But what if we could use remote development techniques like Development Containers or GitHub Codespaces to finally overcome the “works on my machine” problem? And also end the cloud vs. local IDE battle?!

Currently, we’re running the so called Focus Months at codecentric. And among many others participating this spring our very own CEO Rainer Vehns finally wrote a blog post about cloud based IDEs . The topic had been there for a while, but I didn’t really dig into it until Rainer wrote his post. One of the tools he wrote about was GitHub Codespaces, which GitHub itself started to use as their default IDE setup for a while now – leaving their MacOS based one behind. I also remember my great colleague Thomas Darimont mentioning Visual Studio Code Dev Containers at a codecentric internal Dev Friday. Now my head started to put the pieces together. Aren’t GitHub Codespaces based on VS Code Dev Containers? So I thought it might be the right time to dig into the topic …

How to get developer environment setups right?

But it wasn’t only the post and my colleague that made me look into Dev Containers. My professional career somehow always focussed on finding the best ways to develop software. Focussing on CI/CD and DevOps topics I was thrown into the situation to automate the setup of developer environments many times. We tried to solve it using several tools and approaches. Some of them even involved using infrastructure automation tools like Ansible , which could easily configure our shiny MacOS or Linux machines. But most of the time we ended up with putting lots of effort into the automation – never really solving the issue completely. Over time the setup often became brittle and somehow the approach never felt like a perfect match for the problem.

But wait! Why don’t we use the benefits of containers to solve the issue? Didn’t they make pretty much every dependency our software needed to run finally describable in code, simply by using a Dockerfile?

Treat dev environments like any other infrastructure

So containers revolutionized the industry and made running applications using their exact depenencies simple. But what about the development side of things? Sure, we also already fully containerized our CI/CD pipelines. Check! However if we really go down that rabbit hole: Don’t we as developers shy away from checking out new projects? Doesn’t that expose us to the possibility of ruining our great local development environments?

Ok you may say – simply do a git clone! But then? The story might go like this:

“Hmm, the project seems to use Node.js. Which version? Oh, there are some native extensions needed. Let’s compile them quickly. Oh, I upgraded my MacOS recently and the native extension seems to refuse to work with this version of Node. Damn … “ That day, no commit addressing the business problem will reach the Git repository I guess.

And it’s even worse. Every change to our application may also introduce a change to some kind of dependency. We only need to remind ourselves about the “How to contribute in this project” paragraphs in the vast majority of open source projects. And the same is true (or even more hair raising) in enterprise projects behind big firewalls. Just think about collaborating on multiple branches across multiple projects – as Microservice-style architectures may force us to.

We seem to be stuck in the good old “works on my machine” hell. So what about using the benefits of containers at development time as well? Or to quote it from the GitHub engineering team :

We saw an opportunity to treat our dev environments much like we do infrastructure – a commodity we can churn – but still maintain the ability to curate our workbench.

Run the developer tools locally & connect to Development Containers (remotely)

If you dig into the topic you finally find out about the Visual Studio Code Remote – Containers extension . This VS Code extension originates from the VS Code Remote Development project at Microsoft and is basically one of three extensions provided there. The other two extensions are also really interesting: one is the VS Code Remote – SSH extension which let’s you use any remote machine with a SSH server as your development environment. The other extension VS Code Remote – WSL let’s you code inside a Visual Studio installed on Windows while the development environment runs inside a Linux container on WSL (v2) .

At least for me the interesting thing about the VS Code Remote Development project is that it has been already announced and released as a preview in May 2019 ! But shame on me, it needed three years and the invitation to GitHub Codespaces until I realised it’s full potential. I can only encourage you to read the blog post, especially the “A Different Approach” paragraph reveals what the original idea behind the Remote Development extensions was (and I guess having Windows as the default client-side OS in your company is a great driver to think in that direction):

We convinced ourselves that what we needed was a way to run VS Code in two places at once, to run the developer tools locally and connect to a set of development services running remotely in the context of a physical or virtual machine (for example, a container or VM). This gives you a rich local development experience in the context of what is in the remote environment.

And that needs some time to really sink in. Because that means, we can have any possible environment encapsulated and directly connected with possibly any IDE we want. In this article we focus on VS Code, but the concept is universal! And the essence behind it is: We can reliably define these development environments with code! So the everything-as-code train rolls again and the term Development Environment-as-Code is born 🙂

Configure your app to use Dev Containers in VS Code

Let’s check out the VS Code Remote Containers extension locally. Be sure to have VS Code installed (e.g. via a brew install visual-studio-code --cask) and then install the Remote Containers extension . If this went well, you should see a new icon in the bottom left of your VS Code screen:

Now to start with Dev Containers, choose a project you want to clone locally. As an example for this post I used a Vue.js / Nuxt.js based project that is available on GitHub . Open your project in Visual Studio Code and click on the new green icon in the bottom left (or press F1 and type Remote-Containers) and hit Open Folder in Container. Be sure to have Docker Desktop running locally (or even Rancher Desktop should work).

You will now be able to choose the root directory of the Git repository you’ve just cloned locally and then choose from a variety of Development Container configuration files:

If you cloned the example project from GitHub , simply choose Node.js & TypeScript and select a Node version like 16-bullseye. You can leave the additional features section unchecked and simply click OK. You can even hit Try a Development Container Sample which will do all the steps using an example project listed here .

Now a new VS Code window will open up and tell you that it’s Starting Dev Container where you can view the logs as well:

The devcontainer.json

Now you should notice two files have been generated for you, both inside the new .devcontainer directory: a devcontainer.json:

and a Dockerfile:

The contents of this file define a Dev Container that can include frameworks, tools, extensions, and port forwarding. The devcontainer.json file usually contains a reference to a Dockerfile, which is typically located alongside the devcontainer.json file.

You can also customize these files or even create your own custom dev container configurations . The full devcontainer.json reference docs can be found here.

You can define a single Dev Container configuration for a repository, different configurations for different branches, or multiple configurations.

What’s also really cool: There’s an open specification for the concept of Development Containers available (also on GitHub ). This will hopefully lead to a broad adoption of it in many other IDEs. Additionally we can reuse existing container images or Docker Compose configurations . There’s a image configuration key for example for a container image from DockerHub , GitHub Container Registry etc.

Running & developing your app inside a Dev Container

Don’t get confused here: We don’t want to run our app in production mode inside a Dev Container. Remember: we only want to use it to develop our app! Depending on your language and framework, you need to fire up your application in development mode.

Inside VS Code you can now check whether the correct Node.js version (in my case 16) has been installed. Simply fire up a new terminal window inside VS Code and run node --version and npm --version. Both might be different from your local workstation (or you don’t even have Node.js installed at all):

This already shows the basic concept in practice. The Dev Container is transparently used inside our IDE and provides our full development environment without changing our host’s environment. You can even debug your app running inside the Dev Container .

To install all the needed dependencies for the example app, we need to run the usual npm install inside our root directory. The first run may take a while and will download the needed npm packages into the node_modules directory.

As we’re using Nuxt.js we should now be able to start our app in development mode with webpack’s hot module replacement activated. Since this happens inside our Development Container, we need to forward the port 3000 of our container to our host. This can be easily done inside the .devcontainer/devcontainer.json using the "forwardPorts" key:

Having this configuration in place we can run npm run dev inside our VS Code terminal, which will fire up the Nuxt.js dev server inside our Dev Container. Inception! 🙂

If you click on Open in Browser, your app should show up as you’re already used to when starting it locally without a Dev Container:

Really cool! Checking things under the hood, you’ll notice VS Code Remote Containers creates an image for your application locally. Simply run docker images | grep vsc to see all the related container images:

But this image only contains everything needed to provide us with a development environment. Our source code isn’t located there, nor are the dependencies cached there. But if we rebuild the Dev Container (e.g. by hitting Rebuild Container inside VS Code) all the downloaded dependencies are preserved for us. And that works pretty simply. Our project residing in our local file system just gets mounted into the container by the VS Code Remote Container extension .

GitHub Codespaces in Action

So we learned about VS Code Dev Containers and they are pretty cool! And as they are based upon an open spec , the adoption of the concept by other IDEs seems just a question of time. But what about these Codespaces thing? GitHub even seems to develop GitHub using Codespaces now?!

To find out more I lately registered to GitHub Codespaces, which is currently in beta/early access. I opted for the beta using my personal mail address which is configured in my GitHub account. If you’re part of a GitHub organization that uses a paid plan, chances are that your org administrator can activate Codespaces for your repositories .

If you have Codespaces activated, you only need to head over to any of your GitHub repositories. The usual < > Code button now has tabs called Local and Codespaces. Clicking on the latter, the GitHub UI welcomes me to cloud editing. So let’s start this by creating a codespace on main:

And there we are: A Codespace already spins up and a new Browser tab is opened up with an inline Visual Studio Code editor waiting:

So at first glance Codespaces seem to spin up a Visual Studio Code for us in the cloud – no need to use a local one? You may also already noticed that you can hit . on any GitHub repository (if you’re logged in) and a webified VS Code will start in your Browser. The difference to Codespaces is: you don’t have an development environment ready where you can run your application. That’s reserved to Codespaces! And here the connection to Dev Containers become clear. Because GitHub Codespaces use VS Code Dev Containers to setup predefined development environments and connect the webified VS Code to them.

GitHub Codespaces is build on top of Dev Containers

As we’re already learned VS Code has multiple predefined dev container configurations available for us. In a new project without the .devcontainer directory they can be autogenerated by heading to the Codespaces / VS Code command palette (via F1), typing dev and selecting Codespaces: Add Development Container Configuration Files….:

You may also select additional packages to be available inside your Dev Container. These range from tools like AWS or Azure CLI, Docker-in-Docker support, other languages like Java, Go, Python, Ruby or even Terraform or Homebrew support. If there are already devcontainer.json definitions available inside the repository the github.com Codespaces tab will also let you choose from these configurations upfront .

And to complete the confusion it’s also possible to use a Codespace from your locally installed Visual Studio Code ! In this case the webified VS Code isn’t used – but the Dev Container based development environment is prepared for you in a Codespace. You can even set your local VS Code as the default editor for Codespaces in your GitHub account’s configuration.

Alternative Dev Containers implementations

Many companies experienced the same problem as GitHub and the VS Code team. So the idea of Dev Containers developed over years already. For example, engineers at LinkedIn used remote development environments called RDev . The RDev configuration is done by using the Development Container specification and VS Code as the (local) IDE of choice. Stripe also uses multiple remote devboxes in the cloud and only leaves the VS Code editor UI on the laptops.

In June 2021 Docker announced their new Docker Desktop Development Environments feature. These dev environments are managed by Docker Desktop and run locally on the developers laptop. It’s even possible to run Dev Containers completely without VS Code: The project devc provides a small CLI that runs almost the same commands as VS Code behind the scenes.

What about IntelliJ support you might ask? While there is some support for e.g. using Docker containers as Pycharm remote interpreters , JetBrains seems to follow their own way in terms of remote (containerized) development environments. A huge all-in-one software development tooling called JetBrains Space that includes hosting Git repository projects, CI/CD pipelines and even project and team management also comes with Space dev environments which can easily be compared to GitHub Codespaces. Both IntelliJ Idea and the next-gen JetBrains IDE fleet keep running on your local machine (like VS Code with Codespaces) and the development environment is spun up for you in JetBrains Space dev environments.

But JetBrains also supports another development environment alternative as a first class citizen: In April 2022 they announced their partnership with GitPod , which seems to be a quite neat solution at first glance. GitPod introduces it’s own Development-Environment-as-Code format based on a .gitpod.yml and looks really promising (enough for another blog post I guess).

Development Containers are here to stay

The basic concept of Development Containers (or Development Environment-as-Code) will make it’s breaktrough in the software industry for sure (if it hasn’t already). After containerizing the production deployments of our apps followed by our CI/CD pipelines the third container revolution takes place: the development environments will never be the same again. In some years from now, junior developers will ask the seniors what these old stories about problems on localhost that they found on the internet are all about.

As the concept is also not fixing the choice whether to use a local or a cloud based IDE, it will definitely be a gamechanger (and maybe end some discussions). And Development Containers can also be hosted locally or in the cloud. They can even be pre-initalized nightly based on Git branches, if you have a complex bootstrapping setup for your application (as GitHub itself does ). Now we are able to reach instant-on development environments for every developer in our company just by implementing the concept.

GitHub Codespaces deliver these promise already out-of-the-box based on all GitHub repositories. And they simply use the flexibility of the Development Container concept and host both IDE and dev environments in the cloud. With the option to switch to your local VS Code, if you’d like to. A similar approach is implemented by JetBrains, where the containerized dev environments are hosted via JetBrains Space or GitPod . And first-class support of GitPod for JetBrains IDEs looks promising! I hope to find the time to look into GitPod soon.