Standing on the shoulders of the Tekton community: Tekton buildpack pipeline

In the first article we mastered the Tekton installation, got to know the first API objects and created a first small pipeline. You might want to have a look at this Remarkable – note for a recap.
Now we will create a practical pipeline that, as usual for a CI system, creates new Docker images when new commits are pushed. The reusability of Tekton components, both tasks and entire pipelines, is a strength that I would like to demonstrate practically in this article.

How do we build?

We can build container images in several ways.

Docker, the company that brought containers to the mainstream, defined writing a Dockerfile which built a container image step by step triggered by a docker build. Writing this starts very simple, but it quickly becomes very complex if you want to ensure a secure, stable build and operation.

I would like to set only two impulses that you might want to think through to get a clue of stuff to be taken care of:

• Should the CI system really pass a Docker socket to the build containers?
• Should the root user really be used within the Docker image?

For these topics there are many possible solutions. I will highlight the buildpack approach.

Cloud-Native Buildpacks are a simple solution to create container images. So what to do? You just need a pack build my-image to build the application in the current directory as a my-image and store it in an image repository. Admittedly, the solution is very easy to use, but at the same time a lot happens in the background to ensure an optimal image build. Issues like e.g. caching and security are solved transparently.

Integration in Tekton

We will need a pipeline that runs the required tasks step by step. The pipeline will first clone the source code from a Git repository and place it in a directory. The second step will read the source code in that directory and use buildpack tooling to build the image and push it to a container registry.

In the first article we looked at how to create a pipeline and link tasks but we haven’t worked with files yet.

Workspaces

Directories can be shared between tasks in Tekton using workspaces. Workspaces can leverage various Kubernetes storage types, for example, a PersistentVolumeClaim, a config map, or an emptyDir.

When defining a pipeline, we define the required workspaces. In our case, we define a workspace to store the source code from the Git repository for the runtime of the pipeline. At this point, only the name and the link to the tasks are known. Tasks can expect workspaces which can then be deployed under a specific name by the pipeline.

Here we see a short pipeline that clones the source code from a Git repository into a workspace named source-workspace.

1apiVersion: tekton.dev/v1beta1
2kind: Pipeline
3metadata:
4  name: git-clone-sample-pipeline
5spec:
6  params:
7    - name: git-clone-url
8      type: string
9      description: HTTP URL of the git repository to clone
10    - name: git-revision
11      type: string
12      default: "main"
13      description: Git revision (branch, tag, commit-id) to clone
14  workspaces:
15    - name: source-workspace # Directory where application source is located. (REQUIRED)
16  tasks:
17    - name: fetch-repository # This task fetches a repository using the `git-clone` task you installed
18      taskRef:
19        name: git-clone
20      workspaces:
21        - name: output
22          workspace: source-workspace
23      params:
24        - name: url
25          value: $(params.git-clone-url)
26        - name: revision
27          value: $(params.git-revision)
28        - name: subdirectory
29          value: ""
30        - name: deleteExisting
31          value: "true"

PipelineRun

As already described in the first article, we create a PipelineRun to start the pipeline and to pass parameters and workspaces.

In the following example, we start the pipeline defined above with the desired values. The Git repository is set to a fixed value and the source-workspace is linked to an existing PersistentVolumeClaim.

1apiVersion: tekton.dev/v1beta1
2kind: PipelineRun
3metadata:
4  generateName: git-clone-sample-pipeline-
5spec:
6  serviceAccountName: tekton-service-account
7  pipelineRef:
8    name: git-clone-sample-pipeline
9  workspaces:
10    - name: source-workspace
11      subPath: source
12      persistentVolumeClaim:
13        claimName: source-workspace-pvc
14  params:
15    - name: git-clone-url
16      value: https://github.com/marcopaga/feeding-the-ci-process-single-project.git

Now the pipeline starts, clones the repository, and terminates successfully.

Providing credentials for clone and push

For open source projects with a publicly available repository, this is it. For internal use with credentials on a repository, we still need to provide them to the Git process.

While researching this aspect, I found various pieces of information. For me the variant which is described there worked well. In essence, a Secret is created with the credentials and extended with annotations describing what deployment and what server it is to be used for. The key tekton.dev/git-0 indicates that this is to be used for github.com repositories.

1apiVersion: v1
2kind: Secret
3metadata:
4  name: github-clone-secret
5  annotations:
6    tekton.dev/git-0: https://github.com
7type: kubernetes.io/basic-auth
8stringData:
9  username: git-clone-user-name
10  password: git-clone-password

This on its own is not enough, because the Secrets to be used must be added to the ServiceAccount of the PipelineRun. As you can see in the following example, the Secret is simply listed as well.

For the container image push, a Secret is also needed. Here it is sufficient to create a normal Docker Registry Secret as described here in the Tekton Docs. It is also important to add this to the ServiceAccount.

In the following you can see a ServiceAccount with the two defined Secrets.

1apiVersion: v1
2kind: ServiceAccount
3metadata:
4  name: buildpacks-service-account
5secrets:
6  - name: docker-registry-secret
7  - name: gitlab-clone-secret

Using an existing pipeline and tasks

Now we have already created a pipeline with which we can clone the code. We can now continue with our knowledge and gradually add the next steps.
On the Tekton Hub, we can find tasks for the build . So far, so good! The tasks are well documented and actually we can build them directly.

But what’s that – you can also search for pipelines ? If we do that, we find a complete pipeline that is already done and tested.
The buildpacks pipeline is beautifully documented. We find information about the background, the required dependencies, and the installation.
As described in the first article, these are installed and used directly. The existing PipelineRun is adapted following the documentation and everything is ready.

Recap

This reusability allows us to collaborate and share CI components within our own organization and even in the open source ecosystem.
We were able to create a working pipeline with very little time and effort by leveraging community components. Now we have a starting point to make our own extensions, benefit from the community of the Tekton Hub and hopefully even give back our own contributions.
In the next article we will continue with Tekton Triggers on this basis and start a build of this pipeline when pushing it into a repository.