From Classic CI/CD to GitOps with ArgoCD & Crossplane

Lately I found a passion in integrating Crossplane with ArgoCD and finally wanted to write about all the steps needed to create a full blown working setup of both. Just as I finished the code and tried to find a good start into the topic, I found that I needed to write kind of a preceding post. I felt that I need to explain the why and what about ArgoCD and Crossplane just not in a technical way only, but also with a bit of context. And I think I also need that context for myself.

Crossplane & ArgoCD – blog series

1. From Classic CI/CD to GitOps with ArgoCD & Crossplane
2. Bootstrapping Crossplane with ArgoCD
3. Going full GitOps with Crossplane & ArgoCD

Why integrate Crossplane with ArgoCD?

As I started to work with Crossplane I was really fascinated about the tooling and the possibilities it provides to build your internal developer platform. Having worked with GitOps tools before I nevertheless felt that there was a piece missing in the puzzle. Crossplane provided great concepts for creating your own Compositions of infrastructure and added a great API between platform team (XRDs, Compositions, XRs) and the application developers (Claims or XRCs). It lets you create all the infrastructure you can possible imagine just through the use of the Kubernetes API. No more need to learn the next fancy DevOps tool! If you teach your team(s) to read manifests, you're done. What a great invention! And it adheres to GitOps Principles 1. & 2.: It's absolutely declarative. And it should reside in Git, thus beeing versioned and immutable!

But who or what triggers Crossplane? A manually executed kubectl apply -f ? A CI/CD pipeline of your choice? Well not really in a GitOps world, where these kind of task should be handed over to a software agent (aka operator). Also the infrastructure Crossplane creates should be automatically reconciled if some manifest changes in Git, right?! In order to wrap my head around the whole topic I drew some sketch notes. Let's dive into "history".

Classic CI/CD approach (aka 100% "Push")

In the world before GitOps most of us created "classic" CI/CD pipelines, into which we strived to automate every single step of our software delivery process (I wrote about "next gen" CI/CD and Pipeline-as-Code Building Blocks back in 2021). And I know there was even a world before CI/CD, where all these steps were manually executed... But let's ignore that here.

A simply CI/CD pipeline consisted of at least three steps: Testing our software, which had been checked into Git. Building a container image from our source code (or any other deployable artifact). And finally deploying our artifact to our Kubernetes cluster. These steps are roughly sketched here:

As you see an application developer (named "DEV" here) triggers the pipeline by checkin in some code changes. The application consists of a frontend ("UI") and backend ("BE") and is deployed multiple times (aka replicas) behind an API gateway. Nothing strange here. I named this approach "100% Push" from a GitOps perspective, where we push our software artifacts solely from our CI/CD pipeline to our infrastructure. But wait! Isn't there something missing? Right, the infrastructure we want to deploy our applications to.

Classic CI/CD approach incl. infrastructure (still 100% Push)

In the pre-GitOps world we created our infrastructure manually by clicking around in Hyperscaler frontends. Or much better: Also created pipelines for our infrastructure provisioning. So let's add another CI/CD pipeline to our picture. This time an "infra pipeline". It provisions our Kubernetes cluster. In this example it's an AWS EKS cluster that needs networking with a VPC, SecurityGroups etc. and NodeGroups with some VMs to do the heavy lifting. Also additional Kubernetes resources like the API gateway need to be provisioned:

The ops or platform team also used a Git repository leveraging a multitude of DevOps tools to provision the needed infrastructure. The sketch note only mentiones a few: Terraform (or the OpenSource fork OpenTofu), AWS CDK, Pulumi and Ansible. But these are just examples, where Terraform might have the widest adoption in the Cloud ecosystem currently.

One might argue that this picture already looks like "GitOps", since everything is managed in Git. Well not really. I don't think that GitOps was the best name for the pattern the industry could have found. But it is like it is. Thinking of GitOps beeing equivalent with the "operator pattern" makes my brain having less trouble understand the whole thing. And yes, I also think that reaching this kind of "classic" level where every bit of infrastructure and application deployment resides in Git is absolutely far from beeing a bad thing.

In most GitOps related articles this classic CI/CD approach is referred to as "Push" based. Thus the headline names.

Adding ArgoCD for application deployment (50/50 Push/Pull)

But someone invented GitOps for a reason. Including tools like Flux CD, Sveltos or ArgoCD. Using these tools a software agent (or technical: a Kubernetes operator) get's installed on a (mostly separate) management cluster. After installation the GitOps operator (in this case ArgoCD) takes over the application deployment:

Therefore also the CI/CD pipelines change (they don't go away fully as you note). In our example the third pipeline step that did the application deployment now only commits a new image tag to another Git repository: the deploy repo.

There are much discussions around about the need for separate deployment repositories. And it's absolutely possible to avoid the need for these repos as my colleague Daniel described in his post.

The GitOps operator of your choice needs to be configured to watch this deploy repository. In the event something changed (like the image tag), ArgoCD deploys this change to the Kubernetes cluster we already used to have with the classic approach. This kind of flips deployment responsibilies around: Instead of our CI/CD pipeline pushing our application artifacts to our Kubernetes cluster (aka Push), the GitOps operator continously reconciles the state of our deploy repo with our application cluster (aka Pull).

Now if you're already using GitOps tooling this might sound familiar. *But what about our infrastructure provisioning?! *As I started my carreer one of the greater goals in IT back then was to harmonize tooling between application developers and operations people. And this even sticked to me as a goal since I haven't really seen a full DEV and OPS tooling alignment since then. But using Crossplane together with ArgoCD might bring us a great step into that direction!

Adding Crossplane to apply GitOps to infrastructure provisioning (100% Pull)

What the integration of Crossplane does to our setup is that from now on we don't need an infra pipeline to provision our infrastructure anymore. Using Crossplane we choose the same pipeline style as our application developer counterparts: We test our Crossplane Compositions using an appropriate testing tool (like kuttl or Kyverno chainsaw). We build a Crossplane Configuration Package (aka a container image) and push it to our container registry. And we finally commit the new Package version to a new "infrastructure deploy repo" (which is often named as Crossplane "apis"):

And just as with our application deployment ArgoCD takes over and watches this infrastructure deploy repo - aka the Crossplane apis repo. Noticing new Crossplane Configuration Package versions there the ArgoCD operator now triggers Crossplane to provision the declaritively defined infrastructure. I will cover in-depth how this works using ArgoCD's App-of-Apps pattern in the upcoming blog posts.

But the ArgoCD Crossplane connection doesn't stop there. Since we now have a bi-directional reconciliation loop in place! Crossplane will make sure that the infrastructure matches the manifests deployed in the management cluster. And ArgoCD will make sure, that the manifests deployed in the management cluster match the ones checked into Git. Yippie: a new GitOps level has been unlocked :)

As you notice we still need a pipeline to boostrap our second management cluster with ArgoCD and Crossplane. This second pipeline will also do some extra work. If you for example ever wondered what a Crossplane ArgoCD provider could be good for: If you imagine Crossplane beeing triggered by ArgoCD to provision an EKS cluster as an application deployment target. If Crossplane is done deploying the EKS cluster, how could ArgoCD get to know this newly created cluster which hasn't been present when ArgoCD was bootstrapped in the first place? Right, therefore you'll need this exact provider. I will also cover this topic in an upcoming blog also in-depth.

Crossplane and ArgoCD is a match made in GitOps heaven

I would really encourage you to try the integration of ArgoCD and Crossplane (and thus will write more detailled posts). Both tools are already a great choice on their own. But combined together it brings your platform setup to a whole new level! My hands are tingling already in advance of writing the in-depth posts.

The bi-directional reconciliation loop brings into infrastructure provision the same "level of GitOps" as we're already used from our application deployments. With this harmonisation both DEV and OPS (or platform) teams move closer together. And all the benefits of using Crossplane are beeing integrated into a full fledged GitOps setup.

If you want to learn more about the technical details and how to integrate Crossplane with ArgoCD, stay tuned for the upcoming blog posts.