Approaches for API consumers
“We do know our consumers. We know exactly what they want.” Very often I hear these two sentences at the beginning or in the middle of projects. But who is a consumer of an API or a digital product in the fist place?
This is a question that should be on every API provider’s mind whenever the team is adopting API as a Product . It doesn’t matter if it’s internal or external. In this article, I will takle API Experience from the consumer’s perspective. We have developed a methodology for the vendor that addresses consumer needs – but does it really meet them? Because we know that API consumers always finds themselves somewhere in the middle between search and feedback.
If we now take a look at the API management process (see figure above), a consumer may go through the following phases:
- Discover
- Develop
- Consume
- Monitor/Optimize
All four phases have in common that a consumer has the chance to give feedback at any time. If you take a closer look at the phases, you also get the impression that the use of a platform will be of great importance for the cycle “Discover – Develop – Consume”. But then who are actually the users of such a platform?
Personas
In general, we encounter two types of consumers and simultaneous users of a platform. On the one hand, there is the business, also often referred to as the department. On the other hand, there is a developer persona. This type can be found in the following forms in teams:
- Enterprise Developer
- Web Developer
- Device Developer
- Data Scientist
Basically, they are all share a basic habit of finding the data they are looking for in the form of a good interface, the API. However, business is now increasingly taking over the role of a developer. In the literature, the term Citizen Developer is used to characterize this role. By Citizen Developer we mean a person from the business who can create applications without programming knowledge, but usually with the support of the IT department. Thus, we are now dealing with three different consumers:
- the business
- the Citizen Developer
- the developer, as the classic user of an API
API Interaction Patterns
In order to understand consumers’ needs, it is important for a comprehensive API experience that we complete it on an abstract level without drifting too much into the technologies.
To achieve this, we generally make use of patterns in software development. With API Interaction Patterns we can do this with regard to the description of APIs. The following patterns are currently used in projects around APIs.
Tunnel
The idea of Tunnel Pattern goes back to ideas like Remote Procedure Call (RPC). From a developer’s point of view, it’s a convenient way to provide simple extensions for existing procedures for the purpose of exchange. But the handling around the tunnel, i.e. the technical access to the respective endpoint, makes it difficult for the consumer to use. This led to the availability of a simpler protocol for the tunnel in the form of HTTP and consequently to a proliferation of APIs based on the Resource Pattern.
Resource
The consumer-oriented approach is the focus of the resource pattern as opposed to the tunnel. Basically, it is about making the appropriate resources accessible that consumers use so that interaction can take place starting with the consumer. In this way, there is an opportunity to hide any implementation details behind the resources. But the resources sometimes do not contribute to an improved representation, because it is not possible to show relations between the resources without additional processes or workflows. This is exactly the problem the Hypermedia Pattern wants to solve.
Hypermedia
The Hypermedia Pattern builds on the Resource Pattern and adds the use of links to it. As with the web, the paths between resources can be navigated by links between them, without having to know each individual resource. Thus, a consumer receives a lot of information, of which one does not know at the moment whether one really needs it. It already contributes to the fact that such an API is regarded as “chatty”. However, what should actually be achieved can be solved in another way: with the Query Pattern.
Query
While with the Resource style we face one endpoint per resource, with the Query style it is only one endpoint where the consumer is enabled to query the resources behind the API. The consumer also gets back exactly the result that matches their query. This in turn means that there must be a good knowledge of the data as well as the query models on the consumer side.
If we now look at the styles shown so far, we see that they are all based on the same assumption. The consumer is the trigger of the interaction by sending a request and then waiting for a response.
What does it actually look like if the consumer wants to be informed about every change and does not need to explicitly request?
Event-Based
This is where the Event-Based Pattern comes into play. With this interaction pattern, the Publish/Subscribe Pattern (PubSub) also makes its way into the infrastructure. Based on events, consumers and producers can be connected directly or a message broker is located between the two for some decoupling. Unlike the other styles, we now find a kind of centralized architecture.
On search and retrieval
The Interaction Patterns now form the basis that API providers need to describe the APIs to be provided, including the data models. Here we talk about ProtoBuf , OpenAPI , GraphQL schema , AsyncAPI and JSON schema , among others. By describing a respective API and the data model, it is ensured that “search and find” becomes possible at any time.
The Discover phase represents the main interest of API consumers, so that they, too, can always explore further digital business areas. However, the creation of a description represents a sufficient criterion. To support discovery, it is now necessary as a provider to ensure the storage of descriptions. In general, we are talking about a catalog that makes it possible to guarantee access to descriptions. At this point, I usually hear in conversations with customers that they need an API gateway for this. An API gateway does not support the consumer here, but is intended to help with the general provisioning of APIs at the vendor. But why does this misconception occur?
There are several reasons. First, almost every API gateway provides a portal as part of API management. Second, vendors state that an API gateway contributes to the visibility to the consumer, but this is completely false. An API gateway can contribute to visibility of APIs, but in order to support search and discovery, more is required.
The simplest form is a catalog. A second step would be a portal. The portal can be seen as a separate component next to an API gateway and thus becomes a part of the infrastructure as well. Thus, Configuration as Code comes into play here too. As consumers, we are enabled to understand the API products consisting of API definition and data model in an almost textual way. However, it is also possible to offer a common portal for several API gateways.
The next level, starting from a portal, is a hub. It should be understood more as a comprehensive umbrella for complex architectures. The picture of the umbrella shows that a hub can also be a first step towards an Internal Developer Platform. The Internal Developer Platform is a part of Platform Engineering, whose goal is to achieve the best possible developer experience for the entire development process.
The fourth and final stage is the registry. We are already familiar with the concept of a registry, namely storing and making accessible in an audited manner, from the context of event-based systems. In the context of APIs, it is primarily a matter of storing artifacts such as API definitions and associated data model schemas. Compared to a hub, the registry is a very technical component, characterized primarily by the use of the interface in the form of REST. A prominent representative here is the Apicurio Registry . Likewise, the registry supports the concept of an Internal Developer Platform.
What all the presented stages have in common is that they create a desire among consumers for semantic search to significantly enhance the experience in terms of searching and retrieval. But what exactly is meant by semantic search in our context? The term describes a search method that focuses on the meaning of a query.
In our context, this means that consumers are searching for a data model and need a direct link to possible APIs. The idea of a direct search for an API originates from a time when a developer’s persona was seen as the only consumer. Similarly, it is worthwhile at this point to show a possible link between an explicit (external) and a global data model. We now see that the desire for a semantic search for the API provider side is not a simple undertaking and already goes far beyond the concrete representation. At the same time, we are creating a transition to Developer Experience.
Mythical DX
In the context of APIs and consumers, the term Developer Experience quickly comes up and is often used as a way to advertise software products. But what does it mean specifically?
Developer Experience describes a developer’s expectation of a platform, and it’s not primarily about the form of delivery, but the overall experience. This includes communication options with the API provider, well-designed deployment and billing processes. Regarding the types of developers outlined above, we also still need to incorporate different gradations to really bring about the same experience for everyone. But especially in the context of API as a Product, the business also needs a corresponding understandable experience, as the requirements are not comparable to those of a developer.
The business or even the Citizen Developers are usually driven by data and use APIs only as a technical vehicle to obtain the desired data. Therefore, it is important to also give high priority to the data model that an API will deliver in terms of documentation. If we now look at the big picture from the perspective of an API provider, we need multiple views of the data and also the API description so that they can be effective in terms of their target group. In other words, this means that the classic API portal, which usually comes with an API gateway, cannot be effective with respect to the relevant target groups apart from developers. Looking at data models, we come across so-called Data Catalogues. This form of cataloging is meant to satisfy the rules of compliance and governance. Thus, the platform idea is expanded by another component and now clearly shows that the provision of APIs cannot only be done on the basis of a standard solution.
Conclusion
In order to obtain a real end-to-end API experience, many components are needed, but above all, communication. In the end, it requires mature solutions or a platform that accompanies consumers on their entire developer journey.
Was this post helpful?
More articles
fromDaniel Kocot
13.11.2024 Your job at codecentric?
Jobs
Agile Developer und Consultant (w/d/m)
Alle Standorte
More articles in this subject area
Discover exciting further topics and let the codecentric world inspire you.
Gemeinsam bessere Projekte umsetzen.
Wir helfen deinem Unternehmen.
Du stehst vor einer großen IT-Herausforderung? Wir sorgen für eine maßgeschneiderte Unterstützung. Informiere dich jetzt.
Hilf uns, noch besser zu werden.
Wir sind immer auf der Suche nach neuen Talenten. Auch für dich ist die passende Stelle dabei.
Blog author
Daniel Kocot
Senior Solution Architect / Head of API Consulting
Do you still have questions? Just send me a message.
Do you still have questions? Just send me a message.