As you are developing your new shiny containerized service on Kubernetes (k8s), you might also want to apply Network Policies . But during the process, you experience connection problems inside your containers. You followed best practices and kept your containers as minimal as possible. So debugging your networking problems might get tedious.
In this post, we will explore how to use k8s ephemeral containers to debug problems with Network Policies.
Current workflows
There are already tools like illuminatio to test Network Policies. But these tools are developed to validate already existing Network Policies. But how do you develop or debug these Network Policies?
One possible solution is using nsenter as described in this blog post . But as you can see, it involves a lot of steps to get your debugging started. In my opinion, ephemeral containers are a lot easier.
A new way
K8s 1.16 introduced a new alpha feature called ephemeral containers. At the time of this writing (k8s version 1.18.6), this feature is still alpha. So, what is the purpose of ephemeral containers ? This is the interesting part of the definition for this post:
Ephemeral containers are useful for interactive troubleshooting when kubectl exec is insufficient because a container has crashed or a container image doesn’t include debugging utilities.
How to use an ephemeral container
Assume, you have deployed a Pod to your k8s cluster and want to debug networking problems. With ephemeral containers, you can add a container equipped with needed tools to your already running Pod:
kubectl -n <namespace> alpha debug -it <name-of-your-pod> --image=praqma/network-multitool --target=<name-of-your-pod>
For an extended example, take a look at the k8s debugging documentation.
The above command will start a container based on the praqma/network-multitool image in the same network namespace as the other container(s) of your Pod.
The image of the ephemeral container contains the needed tools you intentionally left out of the image of your service. You can now start debugging your issues with the shell spawned inside the ephemeral container. This way, you do not have to evaluate the host the Pod is running on, ssh into the host, and so on and so forth. You are also not dependent on the host OS and the tools which might be installed or not. Ephemeral containers are especially useful when you try to debug Network Policies based on labels. K8s assigns the same labels to the ephemeral container as to the other container(s) inside the Pod. With the same labels, the container also gets the same Network Policies.
Conclusion
Ephemeral containers are a nice way to ease the debugging of Network Policies. But as the k8s documentation states, this feature is “not suitable for production clusters”. One problem is that you cannot remove the ephemeral container. This open issue addresses the problem. But the issue does not mention a milestone when it might be implemented. When this issue is fixed, this feature might also be useful for production use, but until then, keep it for your dev or test environment.
Was this post helpful?
More articles
fromChristian Zunker
9.8.2021 Your job at codecentric?
Jobs
Agile Developer und Consultant (w/d/m)
Alle Standorte
More articles in this subject area
Discover exciting further topics and let the codecentric world inspire you.
Gemeinsam bessere Projekte umsetzen.
Wir helfen deinem Unternehmen.
Du stehst vor einer großen IT-Herausforderung? Wir sorgen für eine maßgeschneiderte Unterstützung. Informiere dich jetzt.
Hilf uns, noch besser zu werden.
Wir sind immer auf der Suche nach neuen Talenten. Auch für dich ist die passende Stelle dabei.
Blog author
Christian Zunker
Do you still have questions? Just send me a message.
Do you still have questions? Just send me a message.