In my German Softwerker article (Vol. 14, p. 90) , I already dealt with the continuous design and development cycle of APIs. This was mainly about basic assumptions and tooling, including the introduction of API gateways or platforms into existing development cycles. In this post, I would now like to introduce automated processes for the design and development of APIs and services under the context of APIOps. When talking or writing about development processes, the term DevOps is usually always mentioned. Unfortunately, this is also always used in a very inflationary manner.
So what is DevOps again?
DevOps is a broad concept that encompasses several aspects of collaboration and communication between software developers and other IT professionals. The easiest way to define DevOps is to talk about its goals. DevOps aims to improve collaboration between development and operations teams throughout the software development process, from planning to delivery, to increase deployment frequency, achieve faster time to market, reduce the failure rate of new releases, shorten the lead time between fixes, and improve mean time to recovery.
One of the models that can be used when talking about DevOps is called CALMS, which stands for Collaboration, Automation, Lean, Measurement and Sharing. The CALMS model is a way to assess, analyze and compare the maturity of the DevOps team.
CALMS model |
|---|
CollaborationCollaboration in the CALMS model means that people are above processes. An organization should value healthy people rather than processes, which can cause people to burn out and eventually quit their jobs. As part of the culture, it also embraces failure – giving people the freedom to fail and, more importantly, to learn from it. In this culture, ideas are valued by all; people do not favor the ideas of only certain people. Hierarchy and titles do not matter, and everyone participates in shaping the system. |
AutomationAutomation of the software lifecycle is critical to achieving greater speed and consistency of deployments. To be able to move from an implemented feature or code change to a deployed feature in production in a matter of minutes requires reliable automation. Key elements that need to be automated are infrastructure (IaC), continuous integration (CI) process, post-code testing, continuous deployment (CD), and testing along deployment paths. Ideally, and if possible, automation is already built into the platform and tools you use. |
Lean principles and processes.The focus of Lean principles and processes comes from manufacturing (specifically Toyota Production Systems). The core of Lean is to remove any waste from your processes. An example of how this can be achieved is to first draw and document the current state of your processes.
After all of this is recorded, you can estimate how long each part takes and easily identify bottlenecks, unnecessary processes, or manual processes. Once these are identified, they can either be eliminated or automated to speed up the process. After repeating this a few times, the desired lean state of the processes can be determined. |
MeasurementTo determine whether deployments and releases are successful, specific metrics are needed. The purpose of measurements is to quickly discover potential problems with the code or process so that, if needed, they can be gone back and fixed. So far, only system and application measurements have been mentioned, but people and cultural metrics should not be forgotten. Measurements inform whether people are healthy or the impact of investments in the organization, in addition to how they affect how much money is made or how to innovate faster. |
SharingSharing insights and best practices is also important, both within the organization and between organizations within a company, as well as between potential competitors and the rest of the industry, with the goal of improving the industry for everyone. |
When you consider all the CALMS principles, they have one thing in common: people. You can’t be successful with DevOps if you don’t focus on people and make sure they work well together. Now the picture of DevOps should be a little clearer. At the same time, we have learned about CALMS as a way to measure progress in terms of using DevOps. However, this is still not enough to get to the APIOps announced in the title. For this, let’s look at another paradigm, GitOps.
GitOps
Through GitOps, versioning, or more precisely Git, is used for the entire management process of an application.This should also enable developers to perform tasks from the area of IT operations. GitOps can also be used to increase the maturity of process automation. A large part of API design and development takes place in operations through the use of API gateways. This leads to another level of APIOps.
A continuous process
The continuous development process continues to determine everything we do – and thus represents the starting point for all further considerations.
Continuous Lifecycle
As already discussed in the previous article, there are two separate processes for the design and development of APIs, one for the producer side and one for the consumer side of an API. The focus will now be on the producer process.
API Producer / Provider Continuous Lifecycle
OpenAPI Spec
When we look at the design of APIs, it also makes sense to follow the approaches of “API first” or “Spec first”. But what does this mean in concrete terms for automating an API development process? The design of an API basically starts with the creation of a specification based on the OpenAPI standard. I already described this procedure in my last article in Softwerker Vol. 14. But how do you write an OpenAPI spec and how does the topic of automation fit with it? Since most projects that require an API generate it based on the sources, the question of a general workflow did not really arise. If you start with “API first”, the first step is version control, or more precisely Git, since a central repository for the API specifications is needed.
But do you really want to write APIs in JSON or YAML notation?
With Stoplight Studio , such an editor is now available as a desktop application. In addition to the actual specification, JSON schemas and project documentation can also be captured on the basis of Markdown. The integration of Spectral and Prism provides support for linting and mocking. In combination with Git, all artifacts can be placed under version control.
Stoplight Studio Visual Editor
In addition to Stoplight Studio, Postman is more a Suite than an editor even when the desktop version is used. We will have a look at the additional API lifecycle possibilities of Postman in a future blog post. Strictly speaking, when an API is published, a configuration for the API platform or gateway is also required. This can be done through the OpenAPI Extensions. Extensions (also called Specification Extensions or Vendor Extensions) are user-defined properties that start with x-, such as x-codecentric. They can be used to describe additional functionality that is not covered by the standard OpenAPI specification. Many API-related products that support OpenAPI use extensions to document their own attributes. Currently, Stoplight Studio does not allow this with the visual editor, but it is on the roadmap for upcoming releases. However, the extensions can be viewed and edited via the code view.
Stoplight Studio Code Editor
Everything as/with code
For the configuration of an API gateway or platform, information in the form of JSON or YAML files is mandatory. The OpenAPI specs created can support configuration through centrally set key/values. But to apply this directly to the API gateway, we need a transformation of the spec to Configuration as Code. Here we now need to check what possibilities the API Gateway offers for integration based on a CI/CD pipeline. Detailed insights into the possibilities of automation based on Everything as Code can be found in an article by Jonas Hecht on the codecentric blog.
Policy as Code
Now that the infrastructure and the configuration of the API delivery pipeline based on “as code” are in place, the question arises of how the sometimes complex access structures to an API via an API gateway or general compliance policies according to standards such as PCIDSS, SOC or DSGVO can also be mapped in an automation. Hashicorp has coined the term “policy as code” for this. However, this article would like to take a look at Open Policy Agent (OPA) in this context. OPA decouples the setting of policies from the enforcement of them. Now, when a component like an API gateway needs to make decisions according to a policy, it queries this to the agent and provides JSON data as input for this. The agent now generates its decision based on the input and the policy and additional data.
Thus, policies can also become part of an existing automated process. This should also suffice for now as an introduction to the topic “Policy as Code”.
Conclusion
The created term APIOps is made up of many topics and sub-aspects that have been known for a long time, such as
- DevOps (in particular CI/CD)
- GitOps
- Everything as Code
All of the elements shown can support and further automate existing processes, with the “API first” approach being the foundation of everything.
Was this post helpful?
More articles
fromDaniel Kocot
11.12.2024 Your job at codecentric?
Jobs
Agile Developer und Consultant (w/d/m)
Alle Standorte
More articles in this subject area
Discover exciting further topics and let the codecentric world inspire you.
Gemeinsam bessere Projekte umsetzen.
Wir helfen deinem Unternehmen.
Du stehst vor einer großen IT-Herausforderung? Wir sorgen für eine maßgeschneiderte Unterstützung. Informiere dich jetzt.
Hilf uns, noch besser zu werden.
Wir sind immer auf der Suche nach neuen Talenten. Auch für dich ist die passende Stelle dabei.
Blog author
Daniel Kocot
Senior Solution Architect / Head of API Consulting
Do you still have questions? Just send me a message.
Do you still have questions? Just send me a message.