How mature is your DevOps? – Some thoughts on measuring progress

Spoiler: It doesn’t really matter.

Recently, we received the following inquiry quite a few times from our customers: “How do you measure your progress towards Dev(Sec)Ops? Is there some sort of maturity model or a required skill set for everyone involved?” This is a concern especially for bigger organizations that wish to roll out this kind of working model to a great number of teams. The short and honest anwer is: I think the question is beside the point.

For the sake of simplicity, I will use the term DevOps for the remainder of this blog post. This includes but is not limited to DevSecOps because the same principles apply here. Chances are, they apply to you too.

To understand this, we first need to take a look at what “DevOps ” is. DevOps means that your product teams are taking over responsibility for solving a problem from A to Z. This includes selecting a methodology, tech stack, security and operational concerns, and much, much more. Usually this is boiled down to “You build it, you run it!”. This sums it up pretty well, but it leaves out a small, but important part: “You decide!”.

Introducing maturity models

So it is up to the team to decide about all these pretty important pieces. But how does this clash with a maturity model of any kind? Well, a maturity model implicitly introduces a few side effects in your organization:

• One size fits all (“97% of our teams are now doing DevOps according to this KPI!”)
• Leveling (“My DevOps is better than yours!”)
• End bosses (“If we solve problem X, we are finished with DevOps!”)

These features directly clash with the nature of DevOps. The principles behind DevOps are an Infinite Game as described by Simon Sinek , meaning there is no final goal to be reached. Instead, you want to have a sustainable future in whatever you are doing. And whenever you solve a problem or reach a new level, you will face new challenges or see that some of your solutions from the past are no longer feasible.

To give an example, in a project I did some time ago we used a CI/CD pipeline which took ages to deploy. We also had a test suite that ran for quite some time but compared to the deployment, it was still a minor issue. After a few weeks, everyone involved was fed up with the long deployment time. So we put some work and engineering into the problem and were able to significantly cut the time down to a fraction. Now the test suite, which was fine a week before, was our biggest obstacle.

Does this mean our test suite was not mature enough in the first place? No. It was sufficient given the environment it was operating in. But once the environment changes, you have to re-assess your current state. And that is the base of the maturity model problem. Such models usually offer an absolute end state which is simply unfit to hit a moving target.

Skill sets for DevOps maturity

Traditionally, teams are staffed according to skill sets that ideally complement each other. So can we instead put together a list of minimum skill requirements for a DevOps team? This also clashes with the definition we have seen above. By defining a narrow set of technologies and skills, the organization is essentially taking the responsibility for these decisions away from the teams. And when they no longer own a decision, this leads to a potential breaking point.

But what can we do about it? If you look at the DORA State of DevOps report 2019 , there is good empirical evidence as to how such a transformation can succeed. Instead of defining a minimum skill set and hiring or training people for this, communities of practice and other bottom-up approaches have been far more successful. So a more promising way is to establish a culture of learning and collaboration. This can and should be supported by the organization by providing the time and resources.

Moving forward

Going back to the initial question, how do we measure DevOps maturity? It doesn’t really matter. Because the question itself does not give us any guidance on our DevOps journey.