Continuous Validation for Security Configurations

Testing integration with a component that has a completely separate life cycle apart from your application is hard. Think about a database system version upgrade. In more cases than one, it has caused a decision to skip automation entirely and rely on manual testing instead. An IAM solution is just like that. It’s configuration-heavy and it is often managed outside of the scope of your team. However, end-to-end testing definitely forces us to consider how we functionally test our security.

Last year, Keycloak became popular (marked assess by TechRadar) as an open source IAM solution and it offers a wide range of APIs. My colleagues already wrote some nice articles . I wanted to explore the opportunities of using Keycloak for our CI to improve testability in our IAM integration.

With Keycloak you are able to continuously test IAM integration in your E2E test suite.

Recently I discovered Keycloak as an Identity and Access Management (IAM) candidate for our Gareth.io platform and another product which is still in incubation. So far, I am pleased with what I have found. The ease of integration via its endpoints is what makes it an especially interesting candidate. Additionally I found a good opportunity to use Keycloak as an E2E test candidate.

Keycloak is developed and promoted by Red Hat. The project pages are located at http://keycloak.jboss.org , the sources can be found in GitHub . The product is completely written in Java and extensively uses Red Hat’s Java Stack. WildFly is used as the default application server and Wildfly’s clustering and high availability functions are also used. Read a nice introduction from my colleague here .

Step 1. Additions to the infrastructure

There are a number of options to configure Keycloak. There is an administrative UI, a config file (keycloak.xml), a CLI, and everything is configurable using the REST interface. All applications living in the same logical domain are in the same Realm. A realm contains the functional administration options (eg. allowed roles, user profile fields, authentication providers, etc.).

To make it testable, I used the jboss/keycloak Docker image to get a basic running Keycloak instance. I have a CI job that creates a basic Keycloak, specific to my project. I use it to configure my realm and roles. It has its own lifecycle and is managed by a separate pipeline. There is a base image which contains basic Keycloak realm configuration. Based on my Keycloak base image I build my production Keycloak (with enriched configuration) and multiple configuration variants for testing Keycloak in my CI pipeline.  

Step 2. Add Keycloak to the test project

Next, we need to add a Keycloak client to our test infrastructure. I use Cucumber (Java variant) to drive my acceptance tests. I added the official Keycloak client jar  and RESTEasy libraries.

In order to connect to it, you can initiate an instance of the client like this:

Now let’s influence Keycloak with our newly bootstrapped library. We first define a Cucumber step and then create an implementation:

And the corresponding Java implementation for our step. We will create a new user within the realm of the tested application and assign it an administrator role. Because we instantly want to use this user in our test we need to mark it as enable.

Now this is just one example of things you can do in your CI with keycloak. I personally am not the biggest fan of testing too many non-functionals in your E2E tests. Functionally you can thoroughly test your registration and sign-in functionality. This is probably not where the main business value is delivered, but it is the start of the user experience in many applications. So we should make sure to capture regression in an early stage of delivery. Furthermore, there are some really nice non-functional cases that used to be hard or required a lot of manual testing which is now possible automagically.

Server side session invalidation

In a modern web application security tokens can be distributed across apps, single-page application frontends and backend applications. Testing this is cumbersome and intrinsically complicated. With the Keycloak client you are now able to influence one or more realms. A very nice feature is ‘RealmResource.logoutAll()’ which invalidates all running sessions on the server. Afterwards you can verify that clients behave accordingly. But there are plenty more.

Wrapping up

We have seen an example on how to integrate Keycloak in your acceptance testing suite. Because we use Docker it’s easily integrated in popular CI tools like Gitlab, Jenkins or CircleCI. Based on what I have seen so far, everything you can do in the Keycloak administrative interface is administrable over REST. Next to this interface there is also the possibility to use shell scripts for administrative purposes. For ultimate flexibility Keycloak is easily and quickly booted in a container so one could even swap out security providers.

My final advice is to keep your Cucumber steps generic. Once created they can be reused  across multiple projects and teams.

Handy sources and follow-up work

Keycloak website: https://www.thoughtworks.com/radar/platforms/keycloak

Dieter Dirkes’ introduction: https://blog.codecentric.de/2016/06/accessmanagement-mit-keycloak/

Jannik Hüls: https://blog.codecentric.de/2016/08/single-sign-mit-keycloak-als-openid-connect-provider/