Stop re-writing pipelines! Why GitHub Actions drive the future of CI/CD

The Pipeline-as-Code pattern is implemented by most CI/CD platforms today. So what could be the next evolutionary step? Based on GitHub Actions, the article outlines why open-source Pipeline-as-Code Building Blocks will take your pipelines to the next level.

GitHub Actions – blog series

Part 1: GitHub Actions CI pipeline: GitHub Packages, Codecov, release to Maven Central & GitHub
Part 2: Publishing Docker images to GitHub Container Registry with GitHub Actions
Part 3: Stop re-writing pipelines! Why GitHub Actions drive the future of CI/CD

Isn’t it just another CI/CD tool?

While migrating many of my open-source projects to GitHub Actions, one thought gradually came to my mind over the last few months: I realized I was working with a kind of next-generation CI/CD platform. Which I didn’t get at first since GitHub Actions is just another CI/CD tool right? Only a different syntax for our pipeline, which shouldn’t be considered a really big step …

But then it slowly became clear to me: it’s all about this “Actions” thing! And I started to think about what I experienced in my professional career using CI/CD tooling as a key piece of my toolbelt. But to really explain what my thoughts are all about, I need to go back in time more than 10 years. 🙂 Please excuse me if I leave out a lot of great tools on this journey. I only wanted to sketch out an idea rather than provide a comprehensive overview of all CI/CD platforms available . So I will focus on the ones that have been most popular in my life as a software engineer.

Going back over a decade in time: the first wave of CI/CD platforms

Let’s start with some time in 2007 when I was in the middle of my higher-education studies. Since I was doing a work-study course, I was thrown into practical application really early. And I’m still honored that I was able take part in the so-called “Frameworks Team” as a student already. On this team, I was able to watch the introduction of the first CI server quite closely: back then, my colleague Xavier reported about the first installation of a Hudson server, his eyes shining with delight.

Logo sources: Jenkins logo , TravisCI logo , GitLab logo , GitHub Actions logo

This thing was so popular, it even received the Duke’s Choice Award at JavaOne 2008 . Forced to fork the project because of Oracle’s purchase of Sun Microsystems, the project was renamed to “Jenkins ” in January 2011. And I guess we all know Jenkins in some form. It is by far the most widely used CI/CD platform, I guess. And there are good reasons for that! I was in the role of a software architect in a project back in 2013, where my soon-to-be colleague Dennis Schulte showed us how to use the Jenkins Job DSL plugin to circumvent the maintainance hell brought about by hundreds of manually created Jenkins jobs.

Pipeline-as-Code: the second wave of CI/CD platforms

When referring to the time before the release of the Job DSL plugin, I think it’s fair to speak of the “first wave of CI/CD platforms”. The Job DSL plugin – and later the Jenkins 2.0 pipelines project in 2016 – are part of the second wave of CI/CD platforms. But Jenkins wasn’t the driver behind this next wave, really. Released as early as 2013, the TravisCI project aimed to provide CI/CD services free for open source projects. This was a huge step! Bringing together TravisCI with GitHub, the resulting setup could be seen as a forerunner of modern software development processes. And they didn’t stop there. I first realized how elegant and simple CI/CD pipelines could be when I joined codecentric in 2015. Encouraged to use our off-project time to build and play with open source projects, I started to create my first projects incorporating this setup. And using Travis felt like a revelation to me! I only had to write a few lines of code and had my pipelines test and build my application.

Back then I always considered the software development process inside enterprise companies to be separate from the one used in open-source projects. My “reasoning” was: Well, Travis is so simple and easy to use because they don’t need to deal with “all that enterprisey stuff”. But soon something began to work in the back of my head. And finally in 2018 I realized that GitLab CI brought the power & elegance of CI/CD pipelines to the enterprise that I only knew from TravisCI. I was really disillusioned by the implementation of the Pipeline-as-Code pattern by Jenkins 2.0 in 2017. Back then we tried to implemend it at a customer project (Marco , I hear you 🙂 ). But GitLab made me realize what this might look like. I also assume that it’s not an easy task to introduce a big application like Jenkins to a new pattern like Pipeline-as-Code. Especially when you’re striving to be completely downward-compatible, as Jenkins 2.0 was aiming to be. I was so convinced by the solution GitLab brought into the game, I wrote lots of articles about it (e.g. this one and this one ). And I recommended this great platform to my customers.

GitLab CI brought Pipeline-as-Code to the enterprise

GitLab CI also solved an issue I had with most Pipeline-as-Code platforms. Released in version 11.7 at the beginning of 2019 , GitLab CI made a huge step forward again. They introduced the include keyword into their pipeline syntax. With this great addition, it was possible to write re-usable pipeline templates (I know by the time also Jenkins pipelines made this possible, but I didn’t find their solution as elegant).

I remember being inside a customer project in 2019 where we faced a migration of several hundred projects to GitLab. Having a set of tested and high-quality pipeline templates at hand was a key success factor in the project. And I guess it is in many companies, because we all love DevOps. And we all want to do Microservices. But many customers aren’t able to fill their projects with Fullstack DevOps superstars only . So having some people focussing on writing really good CI/CD templates is not a bad idea.

To sum up, I would say that roughly the time from 2013 to 2019 could be seen as the second wave of CI/CD platforms. All platforms began to focus on the Pipeline-as-Code pattern and some implemented it better than others (as we’re accustomed to in IT history 🙂 ).

GitHub Actions kick off the third wave of CI/CD platforms

At the end of 2019 GitHub announced the availability of GitHub Actions I didn’t really have the time back then to dig into it deeper. One thing that also kept me from doing so was the thought: “Oh, just another CI/CD platform. So who cares?” I already wrote about the fact I was forced to look into it at the end of 2020. This was when TravisCI stopped broad support for open-source projects . I needed (and still need) to move all my open-source repositories to GitHub Actions. See the “The problem with TravisCI” paragraph in one of my last articles . This migration also forced me to really take a closer look at the platform. I realized that GitHub Actions easily adapted everything that was already great in other platforms. But it adds one piece that sets it apart from the others. And it’s right there in the name: Actions! But wait. What are those Actions?

To get a first understanding, you may want to have a look at the GitHub Actions Marketplace . The Actions listed there are either provided by GitHub itself, or they are developed by other companies and the community. In both cases they are available as open-source projects on GitHub. These Actions help you solve common problems that occur while writing CI/CD pipelines. And they are assemblable like the famous Lego blocks and finally form your CI/CD pipelines. Read that once again! As already said, for me it was the first thought when I heard about the Pipeline-as-Code pattern: How are we supposed to reuse stuff we write here? Do we need to write this release pipeline over and over again? How was this Sonar analysis supposed to be written again? And finally: how can we deploy our app to this shiny new cloud service using Infrastructure-as-Code? Many aspects of such questions are already covered by an Action available. And they ultimately reduce the lines of code needed inside our CI/CD pipelines drastically. Especially when compared to other tools.

It’s all about the Actions

Just insert these few lines describing the chosen Action and go for it. Let’s assume you need a changelog for your artifact releases. Simply use the metcalfc/changelog-generator Action in your GitHub Actions workflow (or another of the dozens of available changelog Actions):

1- name: Generate changelog
2        uses: metcalfc/changelog-generator@v0.4.4
3        with:
4          myToken: ${{ secrets.GITHUB_TOKEN }}

Maybe you need to set up a JDK to build and deploy to Maven Central (configuring a pre-defined settings.xml under the hood also). Just use GitHub’s setup-java Action :

1- name: Set up settings.xml for Maven Central Repository
2        uses: actions/setup-java@v1
3        with:
4          java-version: 11
5          server-id: oss.sonatype.org
6          server-username: MAVEN_USERNAME
7          server-password: MAVEN_PASSWORD
8          gpg-private-key: ${{ secrets.MAVEN_GPG_PRIVATE_KEY }}
9          gpg-passphrase: MAVEN_GPG_PASSPHRASE

And what about said shiny cloud provider and some Infrastructure-as-Code tooling? Let’s say we use Pulumi to provision our infrastructure in Azure . There’s a great Pulumi Action available:

1- uses: pulumi/actions@v1
2        with:
3          command: up
4        env:
5          ARM_SUBSCRIPTION_ID: ${{ secrets.ARM_SUBSCRIPTION_ID }}
6          ARM_CLIENT_ID: ${{ secrets.ARM_CLIENT_ID }}
7          ARM_CLIENT_SECRET: ${{ secrets.ARM_CLIENT_SECRET }}
8          ARM_TENANT_ID: ${{ secrets.ARM_TENANT_ID }}
9          PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}

You also don’t need to worry that your pipelines will spontaneously start to behave completely differently. As you may have noticed, every action has a specific version number. So upgrades are made explicit (and should be also automated with a tool like renovate 🙂 ). I guess this outlines the idea behind Actions aka Pipeline-as-Code Building Blocks 🙂 And in the rare event that there’s no Action for your use case: You can write and publish your own Actions, of course.

Open-source Pipeline-as-Code Building Blocks

OK, maybe I’m overly enthusiastic here. But for me the “Actions” in GitHub Actions kick off the next level (or wave) in the evolution of CI/CD platforms! Finally we’re able to really reduce boilerplate code in our pipelines. And we don’t have to establish a new team to write re-usable templates for us. We can simply pick the ones needed. Additionally these building blocks aren’t only used by our own company. There’s even a big community of people using the Actions, reporting issues, pushing the development of it further, in other words: bringing all the benefits of open-source software into the creation of CI/CD pipelines. A logical step after the introduction of Pipeline-as-Code. Since I see it as a kind of pattern, I would like to name it: “Pipeline-as-Code Building Blocks”.

This is maybe an unusual post of mine: nearly no example code! But as I’ve already written about GitHub Actions (and will continue to write about them in the future), I hope you’ll forgive me. I simply needed to put an emphasis on the huge step this tooling represents for the CI/CD world. I don’t think that GitHub will be the only one that’s adopting the pattern of “Pipeline-as-Code Building Blocks” in the future, though. But so far only GitHub really implements it this broadly. So if you haven’t done so already: take a look at GitHub Actions for your next project. And then stop re-writing your CI/CD pipelines. 🙂