Introduction to GitOps with ArgoCD

In this post you will learn what GitOps is about and see the steps to create a setup on your laptop to gain some experience with ArgoCD. Using an industry standard container orchestrator such as Kubernetes, this enables developers to continuously deploy their applications.

There is a huge number of options available to deploy artifacts. A widely adopted approach uses the OCI image format to ship your software. A continuous integration pipeline builds the software and creates an artifact. The image can then be deployed by a container orchestrator, i.e. Kubernetes.

Furthermore, development teams are responsible for the entirety of their applications. Defining the infrastructure and the process as code is crucial to not reducing efficiency and velocity while keeping all actions reproducible and tackling complexity. Tools such as Terraform and Helm can read a definition and be executed as part of a pipeline to push changes to achieve the desired state.

The GitOps principles

While traditional pipelines and the tools within such as GitHub Actions or GitLab CI/CD are executed whenever a specific event like a git push happens, the GitOps principle works the other way around. The largest difference is that the desired state is continuously pulled from a source such as a VCS repository to drive the infrastructure. This pattern is called a reconciliation loop and matches with the Kubernetes operator pattern.

There are many tools out there that leverage the Kubernetes CRDs as input to define the desired state and apply them by operators.

The elements of GitOps can be described by four principles.

• Declarative infrastructure
• Desired state is stored versioned and immutable
• Pulled automatically from source code
• Continuously reconciled by software agents

The CNCF hosts the project OpenGitOps which is an attempt to define an open standard.

ArgoCD

As stated before, in this blog post we will take a closer look at ArgoCD. This tool focuses on Continuous Delivery of applications to Kubernetes.

For this purpose, ArgoCD monitors a source and creates the API objects out of its definitions files. A lot of options are available starting from plain YAML files to Helm Charts and Kustomize. Even custom plugins are possible, which is very handy in migration scenarios or if the need for any custom integrations are a usual pattern in your organization. The core concepts of ArgoCD can be found at https://argo-cd.readthedocs.io/en/stable/core_concepts.

ArgoCD is an excellent fit in our mission to deliver our high-value applications at best-in-class velocity to users.

Installation

Now it is time to take our first steps into our GitOps journey and try out ArgoCD. First, we need a Kubernetes cluster to deploy to. I will use Minikube here, but every other distribution will do the job. Simply install Minikube and start your local cluster:

minikube start --driver=docker

After your kubectl tool is configured and you can access your cluster, clone the demo repository and install ArgoCD.

git clone git@github.com:dhenne/minikube-argocd-traefik-example.git

There is a bootstrap-argo-cd chart in the folder charts/, which can be used to bootstrap your ArgoCD installation using the helm CLI tool. Later we will use this instance of ArgoCD to manage itself.

cd minikube-argocd-traefik-example/charts/bootstrap-argo-cd/
helm dependency build
helm install argo-cd .

After all images are downloaded and the pods are started, you should see a few pods running:

kubectl get pods

NAME                                                       READY   STATUS    RESTARTS   AGE
argo-cd-argocd-application-controller-0                    1/1     Running   0          52s
argo-cd-argocd-applicationset-controller-bf44c7bc9-2k4w9   1/1     Running   0          53s
argo-cd-argocd-dex-server-8676995c89-7pb5x                 1/1     Running   0          53s
argo-cd-argocd-notifications-controller-7d9d94f45d-rk5kh   1/1     Running   0          53s
argo-cd-argocd-redis-6749667d89-7prsk                      1/1     Running   0          53s
argo-cd-argocd-repo-server-58b8864fc7-kvtln                1/1     Running   0          53s
argo-cd-argocd-server-78fd54c49d-dvv7d                     1/1     Running   0          53s

Nice! To interact with the installation and check out what is going on, we can use the UI. The password can be found inside a secret which is automatically created if not disabled by a configuration parameter. Get yours with kubectl -n default get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d forwarding a port to your workspace using the command kubectl port-forward service/argo-cd-argocd-server -n default 8080:443 & and browse to http://localhost:8080.

After login with the user admin you can see there are no applications deployed yet.

Congratulations, now we can start delivering!

Apps-of-apps pattern

ArgoCD introduces an app-of-apps pattern. As applications can be defined in CRDs, applications can be part of an application. This can be used as a starting point to define which applications have to be deployed.

To demonstrate this, there is a root application defined inside the example repository. The definition contains the path pointing to the demo repository:

charts/root/templates/root.yaml

...
  source:
    repoURL: https://github.com/dhenne/minikube-argocd-traefik-example.git
    path: charts/root
    targetRevision: HEAD
...

The root application of the demo project includes another definition for ArgoCD. Unlike before, the source is a Helm repository. This definition causes ArgoCD to match the previously installed instance and therefore to manage itself. Now you are enabled to get automated updates using Renovate to update your installation!

charts/root/tempaltes/argo-cd.yaml

...
  source:
    repoURL: https://argoproj.github.io/argo-helm
    chart: argo-cd
    targetRevision: 5.4.3
...

Use kubectl apply to instruct pulling the state from GitHub and apply the defined state:

kubectl apply -f charts/root/templates/root.yaml

Now watch the state changing until there are some apps including Root and ArgoCD:

NOTE: To keep your namespace clean, unlink the ArgoCD instance from your local Helm CLI: kubectl delete secret -l owner=helm,name=argo-cd

Now you can explore the great UI from ArgoCD. Everything is clean and focused on purpose. As mentioned in the core concepts before, there are buttons labeled refresh and sync. Refreshing causes to poll the source repositories and their state. After that, a sync applies the changes needed to ensure the defined state is in place. If auto sync is enabled, then a sync is done immediately after a refresh causing an out-of-sync state.

Another very helpful use case is to explore your deployment and get an idea of all parts and their state. If we drill down the argo-cd app, there are a lot of resources deployed. It's a lot of work to get an idea of all parts. But within the ArgoCD UI, everything is well organized and it's easy to check out all parts in detail. In fact, there are 47 resources. All relations definitions states, logs, and events are a few clicks away. Even parts that are paid less attention to such as replica sets or endpoints fit nicely into the big picture.

Application gitopsified

It's time for additional proof of the ability to deploy our apps. The demo repository mentioned before can be seen as a widely used pattern of a GitOps repository. There you can find definitions of the desired state, but the source code can be found in a separate repository. In this article, I will demonstrate a slightly different approach. We only define which applications will be deployed and the desired state is placed in your application's source code repositories. Developers then have to do all their changes inside one repository, without the need to keep an eye on the GitOps repository. Of course, you can achieve that using custom pipelines and other tools, but we want a GitOps-style approach here that polls information instead of pushing it to the GitOps repo. On top of that, ArgoCD is our tool of choice here, and it can achieve that. Why should we bother with custom self-made approaches? Be aware that configuration in source code repositories can be considered a violation of best practices, so prepare yourself for a good discussion (maybe with this article).

While there's a proposal, which aims at first-class support of multiple sources for our definitions, we have to use ArgoCD generators to use multiple sources to define our state. Generators are included within the ApplicationsSet feature, which has been fully integrated into the ArgoCD installation since 2022. ApplicationSets are a great option to template your application definitions without Helm or anything else. Therefore, using the Git Generator can be considered as a first-class approach, too.

There are two options available for the Git Generator. While the directories key scans a directory and creates an application for every folder found, the files key does the same thing for a file-matching pattern.

The demo repository contains another application in the root app named spring-boot-apps, which references a Helm Chart consisting of ApplicationSet definitions to demonstrate the two options. Exploring them in the UI can be done in the root app using a link in the application representation. This shortcut is quite handy to move around while using the app-of-apps pattern.

Following the link, you can see two ApplicationSets, which consist of two applications each for a test and production stage. Let's have a look at both sets.

Git Generator: Directories

In my opinion, the Directories Generator is the best approach. It generates applications from a pattern that matches folders. Inside the folders, everything that is supported can be placed. This provides superior flexibility for developers, which encourages people to strive towards a DevOps culture. In the demo repository, you'll find Helm Charts, but it doesn't matter for the application definition and can be swapped to other options in different stages without redefining the ApplicationSet itself. Another advantage worth mentioning is the avoidance of ArgoCD or organization-specific schema of your definitions inside the source code repository. Developers can use their knowledge of existing standards. Below you can find the definition which points to a Spring Boot application.

charts/spring-boot-apps/templates/springboot-appset-git-directories.yaml

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: springboot-appset-git-directories
spec:
  generators:
    - git:
        repoURL: https://github.com/dhenne/spring-boot-service
        revision: HEAD
        directories:
          - path: "deployment/helm/*"
  template:
    metadata:
      name: '{{`{{path.basename}}`}}'
    spec:
      project: default
      source:
        repoURL: https://github.com/dhenne/spring-boot-service
        path: '{{`{{path}}`}}'
        targetRevision: HEAD
      destination:
        server: https://kubernetes.default.svc
        namespace: default
      syncPolicy:
        automated:
          prune: true
          selfHeal: true

NOTE: Unlike the files generator, using paths inside hidden folders is not supported, as the folders are skipped.

Git Generator: Files

Another generator that can be used is the Files Generator that looks for files matching the provided pattern. Each file can then define keys and values in JSON or YAML format. When ArgoCD is generating the applications, values can be used in a template spec to define applications. Think of Helm Charts, then you got the concept. In the demo repository, we use Helm Charts, so we have to exclude ArgoCD templating from the Helm templating engine using Go strings.

charts/spring-boot-apps/templates/springboot-appset-git-files.yaml

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: springboot-appset-git-files
spec:
  generators:
    - git:
        repoURL: https://github.com/dhenne/spring-boot-service
        revision: HEAD
        files:
          - path: "deployment/multisource-files/**.yaml"
  template:
    metadata:
      name: '{{`{{name}}`}}'
    spec:
      project: default
      source:
        repoURL: https://dhenne.github.io/helm-charts/
        chart: spring-boot-helm-demo
        targetRevision: 0.1.6
        helm:
          values: |-
            {{`{{values}}`}}
      destination:
        server: https://kubernetes.default.svc
        namespace: default
      syncPolicy:
        automated:
          prune: true
          selfHeal: true

Developer Workflow

The workflow for developers to deploy new versions consists of a single commit to the application repository. To instruct ArgoCD to deploy a new version, developers only need to adjust the version number inside the configuration file of the correct environment. For example, the test environment of the directories example can be found in the values.yaml in the application repository. There is a GitHub workflow to demonstrate an automated version bump, release, and deployment of the spring-boot-service to test environments. The production environments are requiring manual confirmation to allow the workflow to proceed. All information defining the application and the corresponding environments can be found in one place, which is a big advantage of this approach.

Conclusion

ArgoCD promises to provide an easy-to-understand GitOps tool. As we have seen, this is perfectly true. There is a lot of flexibility while keeping things aligned on purpose. The UI is very helpful to drill down into the complexity of large deployments and developers can still use their favorite standard tooling for defining applications in Kubernetes. If you consider digging deeper into the topic, the awesome-argo list might be a good start.

Argonauts, to the moon!

Links

https://github.com/argoproj/argo-cd

https://github.com/dhenne/minikube-argocd-traefik-example