Tame the multi-cloud beast with Crossplane: Let’s start with AWS S3

What if learning the Kubernetes API is all you need to provision any infrastructure? And we’re not only talking about AWS, Azure & Google – but also IONOS, DigitalOcean and even vSphere. Let’s have a look at Crossplane and how we can create an S3 Bucket on AWS step by step.

No code required, it’s all declarative!

Crossplane claims to be the “The cloud native control plane framework”. It introduces a new way for managing any cloud resource (be it Kubernetes-native or not). It’s an alternative Infrastructure-as-Code tooling to Terraform , AWS CDK /Bicep or Pulumi and introduces a different level of abstraction, based on Kubernetes CRDs . One could name it the cloud native way to do GitOps.

Crossplane – blog series

1. Tame the multi-cloud beast with Crossplane: Let’s start with AWS S3
2. Testing Crossplane Compositions with kuttl, Part 1: Preparing the TestSuite
3. Testing Crossplane Compositions with kuttl, Part 2: Given, When, Assert
4. Create, build & publish Crossplane Configuration Packages with GitHub Actions & Container Registry

Why Crossplane is different

Crossplane can be also compared to tools which enable the management of cloud resources through the Kubernetes API, like AWS Controllers for Kubernetes (ACK) , Azure Service Operator for Kubernetes (ASO) or Google Config Connector . Crossplane providers are even generated from ACK and ASO . But the CNCF incubating project Crossplane adds on top of these:

The Crossplane community believes that the typical developer using Kubernetes to deploy their application shouldn’t have to deal with low level infrastructure APIs.

Crossplane uses the Kubernetes API and extends it with a set of Custom Resource Definitions (CRDs) to abstract from actual cloud provider APIs. Additionally, these CRDs are a great foundation to build an Internal Developer Platform (IDP) . Crossplane promotes self-service by introducing building blocks like Composite Resource Claims (XRCs) that are a great API for the typical application developer. On the other hand, Composite Resources (XR) and Composites are a magnitude more powerful and ideal for platform operators.

Drawing on our experiences as platform builders, SREs, and application developers, we’ve designed Crossplane as a toolkit to build your own custom resources on top of any API – often those of the cloud providers. We think this approach is critical to enable usable self-service infrastructure in Kubernetes.

Finally, Crossplane also enables effective multi-cloud interoperability . But don’t worry, you can even order Pizza using Crossplane :).

Crossplane basic concepts

The crossplane concept of a Composite Resource is composed out of several building blocks :

• Composite Resources (XR) : they compose Managed Resources into higher level infrastructure units (especially interesting for platform teams) and an optional CompositeResourceClaim (XRC) (which is also referred to as ‘Claim’ and can be seen as an abstraction of the XR for the application team to consume)
• a CompositeResourceDefinition (XRD) which defines an OpenAPI schema the Composition needs to be conform to (think of Kubernetes CRDs)
• a Composition that describes the actual infrastructure primitives aka Managed Resources used to build the Composite Resource. One XRD could have multiple Compositions – e.g. one for every environment like development, stating and production

Composite Resources can also be nested together, which allows for higher level abstractions and better separation of concerns. Think of an AWS EKS cluster where you need lot’s of network/subnetting setup (which can form one XR) and the actual EKS cluster creation with NodeGroups etc (which would be the XR using the networking XR). More in-depth details on nested XRs can be found here .

As an optional step, you can package these Composite Resource artifacts using a Configuration into an OCI container image. Now your custom Configuration package can be installed easily in any other Crossplane clusters.

Composite Resources are composed of infrastructure building blocks called Managed Resources (MR) that are bundled by a Provider:

• Managed Resourced (MR) : a Kubernetes custom resources (CRDs) that represent infrastructure primitives (mostly in cloud providers). All Crossplane Managed Resources could be found via https://doc.crds.dev
• Providers : are Packages that bundle a set of Managed Resources and a Controller to provision infrastructure resources – all providers can be found on GitHub, e.g. provider-aws or on docs.crds.dev . A list of all available Providers can also be found on GitHub.

You may also stumble upon Packages . They were formerly named Stacks and are simply OCI container images. Packages handle the distribution, version updates, dependency management and permissions for Providers and Configurations.

Getting started with Crossplane: fire up a k8s cluster with kind

In order to use Crossplane, we’ll need any kind of Kubernetes cluster to let it operate in. This management cluster with Crossplane installed will then provision the defined infrastructure. Using any managed or local Kubernetes cluster like EKS, AKS, Minikube or k3d is suitable. In my example project (which is available on GitHub ), I used kind to host the management cluster.

Using kind is pretty easy. Let’s get our hands dirty! Just be sure to have kind, the package manager Helm and kubectl installed. On a Mac you can use brew like this (have a look at the docs for other systems ):

1brew install kind helm kubectl

We should also install the crossplane CLI:

1curl -sL https://raw.githubusercontent.com/crossplane/crossplane/master/install.sh | sh
2sudo mv kubectl-crossplane /usr/local/bin

The kubectl crossplane --help command should be ready to use.

Now spin up a local kind cluster with:

1kind create cluster --image kindest/node:v1.23.0 --wait 5m

Install Crossplane with Helm

The Crossplane docs tell us to use Helm for installation :

1kubectl create namespace crossplane-system
2 
3helm repo add crossplane-stable https://charts.crossplane.io/stable
4helm repo update
5 
6helm upgrade --install crossplane --namespace crossplane-system crossplane-stable/crossplane

As a Renovate -powered alternative we can create our own simple Chart.yaml to enable automatic updates of our installation if new crossplane versions get released:

To install Crossplane using our own Chart.yaml simply run:

1helm dependency update crossplane-config/install
2helm upgrade --install crossplane --namespace crossplane-system crossplane-config/install

Be sure to exclude /charts directories and Chart.lock files via .gitignore. Now Renovate will have an eye on our Crossplane versions:

Before we can actually apply a Provider, we have to make sure that Crossplane is actually healthy and running. Therefore we can use the kubectl wait command like this:

1kubectl wait --for=condition=ready pod -l app=crossplane --namespace crossplane-system --timeout=120s

Otherwise we will run into errors like this when applying a Provider:

1error: resource mapping not found for name: "provider-aws" namespace: "" from "provider-aws.yaml": no matches for kind "Provider" in version "pkg.crossplane.io/v1"
2ensure CRDs are installed first

Finally check the Crossplane status with kubectl get all -n crossplane-system:

1$ kubectl get all -n crossplane-system
2NAME                                           READY   STATUS    RESTARTS   AGE
3pod/crossplane-7c88c45998-d26wl                1/1     Running   0          69s
4pod/crossplane-rbac-manager-8466dfb7fc-db9rb   1/1     Running   0          69s
5 
6NAME                                      READY   UP-TO-DATE   AVAILABLE   AGE
7deployment.apps/crossplane                1/1     1            1           69s
8deployment.apps/crossplane-rbac-manager   1/1     1            1           69s
9 
10NAME                                                 DESIRED   CURRENT   READY   AGE
11replicaset.apps/crossplane-7c88c45998                1         1         1       69s
12replicaset.apps/crossplane-rbac-manager-8466dfb7fc   1         1         1       69s

Configure Crossplane to access AWS: create aws-creds.conf

I assume that you have aws CLI installed and configured . The command aws configure should work on your system. With this prepared we can create an aws-creds.conf file as described in the docs :

1echo "[default]
2aws_access_key_id = $(aws configure get aws_access_key_id)
3aws_secret_access_key = $(aws configure get aws_secret_access_key)
4" > aws-creds.conf

Don’t ever check this file into source control – it holds your AWS credentials! In the example project I added *-creds.conf to the .gitignore file.

If you’re using a CI system like GitHub Actions (as this repository is also based on), you need to have both AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY configured as repository secrets:

Also make sure to have your default region configured locally, or as a env: variable in your CI system. All three needed variables in GitHub Actions for example look like this:

Create AWS Provider secret

Now we need to use the aws-creds.conf file to create the Crossplane AWS Provider secret:

1kubectl create secret generic aws-creds -n crossplane-system --from-file=creds=./aws-creds.conf

If everything went well there should be a new aws-creds secret ready:

Install the Crossplane AWS Provider

To be able to provision infrastructure on a cloud provider like AWS, we need to install the corresponding Crossplane Provider first. Remember: the Provider packages all needed Managed Resources (MRs) and their respective controllers. As a Provider is a Crossplane Package, we can install it using the Crossplane CLI like this:

1kubectl crossplane install provider crossplane/provider-aws:v0.22.0

Alternatively we can create our own provider-aws.yaml file like this:

This kind: Provider with apiVersion: pkg.crossplane.io/v1 is completely different from the kind: Provider which we want to consume. The latter uses apiVersion: meta.pkg.crossplane.io/v1.

Don’t forget to install the AWS provider using kubectl:

1kubectl apply -f crossplane-config/provider-aws.yaml

The package version in combination with the packagePullPolicy configuration here is crucial, since we can configure an update strategy for the Provider here. A full table of all possible fields can be found in the docs . We can also let Crossplane manage the upgrade to new versions for us. If you installed multiple package versions, you’ll see them prefixed with providerrevision.pkg.x when running kubectl get providerrevision:

1$ kubectl get providerrevision
2NAME                                                           HEALTHY   REVISION   IMAGE                             STATE      DEP-FOUND   DEP-INSTALLED   AGE
3providerrevision.pkg.crossplane.io/provider-aws-2189bc61e0bd   True      1          crossplane/provider-aws:v0.22.0   Inactive                               6d22h
4providerrevision.pkg.crossplane.io/provider-aws-d87796863f95   True      2          crossplane/provider-aws:v0.28.1   Active                                 43h

Now our first Crossplane Provider has been installed. You may check it with kubectl get provider:

1$ kubectl get provider
2NAME           INSTALLED   HEALTHY   PACKAGE                           AGE
3provider-aws   True        Unknown   crossplane/provider-aws:v0.22.0   13s

Before we can actually apply a ProviderConfig to our AWS provider we have to make sure that it’s actually healthy and running. Therefore we can use the kubectl wait command again like this:

1kubectl wait --for=condition=healthy --timeout=120s provider/provider-aws

Otherwise we may run into errors when applying the ProviderConfig right after the Provider.

Create ProviderConfig to consume the Secret containing AWS credentials

Now we need to create a ProviderConfig object that will tell the AWS Provider where to find it’s AWS credentials . Therefore we create a provider-config-aws.yaml :

Crossplane resources use the ProviderConfig named default if no specific ProviderConfig is specified, so this ProviderConfig will be the default for all AWS resources.

The secretRef.name and secretRef.key have to match the fields of the already created Secret. Apply the ProviderConfig with:

1kubectl apply -f crossplane-config/provider-config-aws.yaml

Provision a S3 Bucket with Crossplane

The Crossplane core controller and the Provider AWS controller should now be ready to provision any infrastructure component in AWS! So let’s start with a simple S3 Bucket.

The first step towards using Composite Resources is configuring Crossplane so that it knows what XRs you’d like to exist, and what to do when someone creates one of those XRs. This is done using a CompositeResourceDefinition (XRD) resource and one or more Composition resources.

So in order to provision a S3 Bucket in AWS using Crossplane we have to craft three building blocks:

1. CompositeResourceDefinition (XRD) (reference docs )
2. Composition (reference docs )
3. Composite Resource (XR) or Claim (XRC) (reference docs )

1. Defining a CompositeResourceDefinition (XRD) for our S3 Bucket

All possible fields a XRD may consist of are documented here . The field spec.versions.schema must contain a OpenAPI schema , which is similar to the ones used by any Kubernetes CRDs. They determine what fields the XR (and Claim) will have. The full CRD documentation and a guide on how to write OpenAPI schemas could be found in the Kubernetes docs .

Note that Crossplane will automatically extend this section. These includes the following fields, which will be ignored if they’re found in the schema:

• spec.resourceRef
• spec.resourceRefs
• spec.claimRef
• spec.writeConnectionSecretToRef
• status.conditions
• status.connectionDetails

The example project on GitHub hosts an a Composite Resource Definition (XRD) for an S3 Bucket. The definition.yaml could look like this:

1apiVersion: apiextensions.crossplane.io/v1
2kind: CompositeResourceDefinition
3metadata:
4  # XRDs must be named 'x<plural>.<group>'
5  name: xobjectstorages.crossplane.jonashackt.io
6spec:
7  # This XRD defines an XR in the 'crossplane.jonashackt.io' API group.
8  # The XR or Claim must use this group together with the spec.versions[0].name as it's apiVersion, like this:
9  # 'crossplane.jonashackt.io/v1alpha1'
10  group: crossplane.jonashackt.io
11  
12  # XR names should always be prefixed with an 'X'
13  names:
14    kind: XObjectStorage
15    plural: xobjectstorages
16  # This type of XR offers a claim, which should have the same name without the 'X' prefix
17  claimNames:
18    kind: ObjectStorage
19    plural: objectstorages
20  
21  # default Composition when none is specified (must match metadata.name of a provided Composition)
22  # e.g. in composition.yaml
23  defaultCompositionRef:
24    name: objectstorage-composition
25
26  versions:
27  # This version should only change, when the parameters defined also change - and thus create a new version
28  - name: v1alpha1
29    served: true
30    referenceable: true
31    # OpenAPI schema (like the one used by Kubernetes CRDs). Determines what fields
32    # the XR (and claim) will have. Will be automatically extended by crossplane.
33    # See https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/
34    # for full CRD documentation and guide on how to write OpenAPI schemas
35    schema:
36      openAPIV3Schema:
37        type: object
38        properties:
39          spec:
40            type: object
41            # We define all needed parameters here one has to provide as XR or Claim spec.parameters
42            properties:
43              parameters:
44                type: object
45                properties:
46                  bucketName:
47                    type: string
48                  region:
49                    type: string
50                required:
51                  - bucketName
52                  - region

The file uses lots of comments in the attempt to give some assistance for crafting your own XRD. A CompositeResourceDefinition can be roughly divided into the top area featuring Crossplane specific configuration (with spec.group, spec.names, spec.claimNames and a compositionRef using spec.defaultCompositionRef) and the bottom section leveraging the OpenAPI schema defining the parameters of our resources. In our case in which we want to provision an S3 Bucket, there are only two parameters: bucketName and region.

We need to install the XRD into our cluster with:

1kubectl apply -f aws/s3/definition.yaml

Optionally one can check the CRDs beeing created with kubectl get crds and filter them using grep to our group name crossplane.jonashackt.io:

1$ kubectl get crds | grep crossplane.jonashackt.io
2objectstorages.crossplane.jonashackt.io                         2022-06-27T09:54:18Z
3xobjectstorages.crossplane.jonashackt.io                        2022-06-27T09:54:18Z

2. Craft a Composition to provision a S3 Bucket for static website hosting

The main work in Crossplane has to be done crafting the Compositions. This is because they interact with the infrastructure primitives the cloud provider APIs provide.

There are detailed docs to many of the possible manifest configurations . A Composition to manage an S3 Bucket in AWS with public access for static website hosting could for example look like the example project’s composition.yaml :

1---
2apiVersion: apiextensions.crossplane.io/v1
3kind: Composition
4metadata:
5  name: objectstorage-composition
6  labels:
7    # An optional convention is to include a label of the XRD. This allows
8    # easy discovery of compatible Compositions.
9    crossplane.io/xrd: xobjectstorages.crossplane.jonashackt.io
10    # The following label marks this Composition for AWS. This label can 
11    # be used in 'compositionSelector' in an XR or Claim.
12    provider: aws
13spec:
14  # Each Composition must declare that it is compatible with a particular type
15  # of Composite Resource using its 'compositeTypeRef' field. The referenced
16  # version must be marked 'referenceable' in the XRD that defines the XR.
17  compositeTypeRef:
18    apiVersion: crossplane.jonashackt.io/v1alpha1
19    kind: XObjectStorage
20  
21  # When an XR is created in response to a claim Crossplane needs to know where
22  # it should create the XR's connection secret. This is configured using the
23  # 'writeConnectionSecretsToNamespace' field.
24  writeConnectionSecretsToNamespace: crossplane-system
25  
26  # Each Composition must specify at least one composed resource template.
27  resources:
28    # Providing a unique name for each entry is good practice.
29    # Only identifies the resources entry within the Composition. Required in future crossplane API versions.
30    - name: bucket
31      base:
32        # see https://doc.crds.dev/github.com/crossplane/provider-aws/s3.aws.crossplane.io/Bucket/v1beta1@v0.18.1
33        apiVersion: s3.aws.crossplane.io/v1beta1
34        kind: Bucket
35        metadata: {}
36        spec:
37          forProvider:
38            # public-read should enable public access for static website hosting
39            # see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket
40            acl: "public-read"
41            websiteConfiguration:
42              indexDocument: 
43                suffix: "index.html"
44          deletionPolicy: Delete
45      
46      # Each resource can optionally specify a set of 'patches' that copy fields
47      # from (or to) the XR.
48      patches:
49        # All those fieldPath refer to XR or Claim spec.parameters
50        - fromFieldPath: "spec.parameters.bucketName"
51          toFieldPath: "metadata.name"
52        - fromFieldPath: "spec.parameters.region"
53          toFieldPath: "spec.forProvider.locationConstraint"
54  
55  # If you find yourself repeating patches a lot you can group them as a named
56  # 'patch set' then use a PatchSet type patch to reference them.
57  #patchSets:

Again this example uses lots of comments in the attempt to give some assistance for crafting your own Composition. After defining some metadata labels, spec.compositeTypeRef and spec.writeConnectionSecretsToNamespace the actual resource configuration happens in spec.resources. Since we want to provision a S3 Bucket in AWS we definitely should have a look into the Crossplane AWS provider API docs . It also helps to have the Terraform docs opened up to clarify some questions or to skim the internet for some example code. The latter wasn’t that comprehensive when writing this post. I hope to see more Crossplane example implementations around in the near future.

The Composition also needs to be installed with kubectl apply -f aws/s3/composition.yaml.

3. Craft a Composite Resource (XR) or Claim (XRC)

Only the platform team itself typically has the permissions to create XRs directly. Everyone else uses a lightweight proxy resource called CompositeResourceClaim (XRC or simply “Claim”) to create them with Crossplane. If you’re familiar with Terrafrom you can think of an XRD as similar to variable blocks of a Terrafrom module. The Composition could then be seen as the rest of the HCL code describing how to instrument those variables to create actual resources.

Regardless of the role you only have to write a Composite Resource (XR) or Claim (XRC) ! You don’t need to craft both, since the XR will be automatically generated from the XRC by Crossplane. If you step into the role of an platform engineer, you can start crafting the XR directly and no Claim will be generated.

Since we want to create a S3 Bucket, the example project hosts a claim.yaml :

1---
2# Use the spec.group/spec.versions[0].name defined in the XRD
3apiVersion: crossplane.jonashackt.io/v1alpha1
4# Use the spec.claimName or spec.name specified in the XRD
5kind: ObjectStorage
6metadata:
7  # Only claims are namespaced, unlike XRs.
8  namespace: default
9  name: managed-s3
10spec:
11  # The compositionRef specifies which Composition this XR will use to compose
12  # resources when it is created, updated, or deleted.
13  compositionRef:
14    name: objectstorage-composition
15  
16  # Parameters for the Composition to provide the Managed Resources (MR) with
17  # to create the actual infrastructure components
18  parameters:
19    bucketName: microservice-ui-nuxt-js-static-bucket2
20    region: eu-central-1

As you can see the Claim is one of the simpler building blocks in Crossplane. You need to look at the correct apiVersion and name the kind exactly to what you defined in the CompositeResourceDefinition (XRD). Also a namespace is needed to define a valid Claim (which isn’t true for XRs, since they’re cluster scoped). Also a compositionRef or alternatively compositionSelector need to be defined to reference the Composition the Claim should use. Finally both our parameters bucketName and region must be defined.

Now kubectl applying our Claim will finally trigger the provisioning of our S3 Bucket in AWS! If you went through all the steps, you can now test-drive your setup with kubectl apply -f aws/s3/claim.yaml.

The CLI validation will have your back, if you held the YAML ruler incorrectly like me:

1$ kubectl apply -f aws/s3/claim.yaml
2error: error validating "claim.yaml": error validating data: [ValidationError(S3Bucket.metadata): unknown field "crossplane.io/external-name" in io.k8s.apimachinery.pkg.apis.meta.v1.ObjectMeta_v2, ValidationError(S3Bucket.spec): unknown field "parameters" in io.jonashackt.crossplane.v1alpha1.S3Bucket.spec, ValidationError(S3Bucket.spec.writeConnectionSecretToRef): missing required field "namespace" in io.jonashackt.crossplane.v1alpha1.S3Bucket.spec.writeConnectionSecretToRef, ValidationError(S3Bucket.spec): missing required field "bucketName" in io.jonashackt.crossplane.v1alpha1.S3Bucket.spec, ValidationError(S3Bucket.spec): missing required field "region" in io.jonashackt.crossplane.v1alpha1.S3Bucket.spec]; if you choose to ignore these errors, turn validation off with --validate=false

This will also help you debug your configuration – it hints for the actual problems that are still present.

Waiting for resources to become ready

There are some possible things to check while your resources get deployed after running a kubectl apply -f aws/s3/claim.yaml. The best overview gives a kubectl get crossplane which will simply list all the crossplane resources:

1$ kubectl get crossplane
2NAME                                                                                          ESTABLISHED   OFFERED   AGE
3compositeresourcedefinition.apiextensions.crossplane.io/xs3buckets.crossplane.jonashackt.io   True          True      23m
4 
5NAME                                               AGE
6composition.apiextensions.crossplane.io/s3bucket   2d17h
7 
8NAME                                      INSTALLED   HEALTHY   PACKAGE                           AGE
9provider.pkg.crossplane.io/provider-aws   True        True      crossplane/provider-aws:v0.22.0   4d21h
10 
11NAME                                                           HEALTHY   REVISION   IMAGE                             STATE    DEP-FOUND   DEP-INSTALLED   AGE
12providerrevision.pkg.crossplane.io/provider-aws-2189bc61e0bd   True      1          crossplane/provider-aws:v0.22.0   Active                               4d21h
13 
14NAME                                        AGE     TYPE         DEFAULT-SCOPE
15storeconfig.secrets.crossplane.io/default   5d23h   Kubernetes   crossplane-system

There are also some other useful commands you should know:

• kubectl get claim: get all resources of all Claim kinds, like PostgreSQLInstance.
• kubectl get composite: get all resources that are of kind Composite (aka XR), like XPostgreSQLInstance.
• kubectl get composition: don’t confuse with composite! Get’s you all Compositions.
• kubectl get managed: get all resources that represent a unit of external infrastructure.
• kubectl get name-of-provider: get all resources related to the Provider.

Troubleshooting your Crossplane configuration

There’s a great explanation in the docs on how to efficiently track down errors in Crossplane configuration:

Per Kubernetes convention, Crossplane keeps errors close to the place they happen. This means that if your Claim is not becoming ready due to an issue with your Composition or with a composed resource you’ll need to “follow the references” to find out why. Your Claim will only tell you that the XR is not yet ready.

The docs also tell us what they mean by “follow the references”:

• Find your XR by running kubectl describe claim-kind claim-metadata.name and look for its “Resource Ref” (aka spec.resourceRef).
• Run kubectl describe on your XR. This is where you’ll find out about issues with the Composition you’re using, if any.
• If there are no issues but your XR doesn’t seem to be becoming ready, take a look for the “Resource Refs” (or spec.resourceRefs) to find your composed resources.
• Run kubectl describe on each referenced composed resource to determine whether it is ready and what issues, if any, it is encountering.

Inspect the S3 Bucket and deploy a static website

Now let’s check our Claim with kubectl get claim-kind claim-metadata.name like this:

1$ kubectl get ObjectStorage managed-s3
2NAME         READY   CONNECTION-SECRET               AGE
3managed-s3           managed-s3-connection-details   5s

To watch the provisioned resources become ready, we can run kubectl get crossplane -l crossplane.io/claim-name=claim-metadata.name:

1kubectl get crossplane -l crossplane.io/claim-name=managed-s3

We can also check if the S3 Bucket has been created successfully via aws CLI with aws s3 ls.

1$ aws s3 ls
22022-06-27 11:56:26 microservice-ui-nuxt-js-static-bucket
3...

Our bucket should be provisioned by now! It will then also be visible in the AWS console:

To really “proof” it’s working, let’s deploy a website (the example project holds a simple index.html ) to our S3 Bucket using the aws CLI like this:

1aws s3 sync static s3://microservice-ui-nuxt-js-static-bucket --acl public-read

Now we can open up microservice-ui-nuxt-js-static-bucket.s3-website.eu-central-1.amazonaws.com in our Browser and should see our website already deployed:

To delete the S3 Bucket again, we simply need to remove the Claim. But before deleting the Claim, we should remove our index.html. Otherwise we’ll run into errors like BucketNotEmpty: The bucket you tried to delete is not empty:

1aws s3 rm s3://microservice-ui-nuxt-js-static-bucket/index.html

Using kubectl delete claim we can remove our S3 Bucket again:

1kubectl delete claim managed-s3