Charge your APIs Volume 32 - Definition-Based API Mocking, Simulation, and Testing with Microcks - Part 2: Specification of Mock Data, Request Validations and Routes

Key Takeaways

• This article is the second one in a three-part series about definition-based API mocking, simulation, and testing with Microcks (make sure you have read the first article)

• While the previous article concentrated on Microcks’ architecture, and its capabilities in automated mock derivation and provisioning, the present article focuses on mock data specification by means of example objects and features for mock refinement, i.e., parameters constraints for request validation mocking and dispatchers for flexible mock routing

• Example objects make it possible to further increase the degree of automation in mock derivation and provisioning, thereby lowering efforts in mocking

• Parameters constraints and dispatchers allow for raising the level of conformance between API functionality and mocks with low-code and even no-code approaches, depending on the complexity of mocked functionality

Introduction

This article continues the three-part series about definition-based API mocking, simulation, and testing with Microcks. The first article presented Microcks’ architecture and an example API that got uploaded to a running instance of Microcks which then derived and served mocks from the API’s definition. In the following, further core features of Microcks are described and practically illustrated on the basis of the example API. These features comprise mock data specification from defined example objects, parameters constraints for the mocking of request validations, and dispatchers to influence mock routing. Both latter features constitute comparatively low-effort mechanisms for subsequent mock refinement targeting API functionality that has an increased complexity.

Example Objects as Sources for Automated Mock Provisioning

Microcks Importers derive mock structures including operation paths and payload elements from existing API definitions. However, structures alone are not sufficient for effective API mocking, simulation, and testing, all of which assume the possibility to interact with operation mocks by exchanging request and response messages adhering to mock structures. For this purpose, Microcks is able to distill requests and responses of concrete API interaction scenarios from example objects. These objects may be included in API definition files that are uploaded to Microcks. Alternatively, they may be described in excerpts that follow a certain definition format. Microcks considers such excerpts "secondary artifacts" as they are separated from a "primary API definition" and accompany it with example objects. Upon import of a secondary artifact, Microcks transforms comprised example objects into mock requests and responses by merging the artifact’s definition with a previously selected primary definition that was already uploaded to Microcks. Effectively, the merging of a primary definition with secondary artifacts yields a combined definition that gathers all relevant information for API mocking, i.e., mock structures from the primary definition and example objects for concrete message exchange from the secondary artifact. Secondary artifacts come in handy, e.g., (i) when bloating an already complex API definition with additional information shall be prevented; (ii) when examples comprise critical business or security-relevant information, and must therefore not be included in publicly shared primary definitions; or (iii) when primary definitions stem from external systems, are immutable, and lack examples.

The example API illustrates both, the description of example objects in primary API definitions as well as in accompanying secondary artifacts. As for the former, the primary definition file "customers.yaml" defines the response OpenAPI example object some_customer for the operation GET /customer:

/customer:
  get:    
    responses:
      "200":
        ...
        examples:
          some_customer:
            email: "some_customer@example.org"
            first_name: "{{ randomFirstName() }}"
            last_name: "{{ randomLastName() }}"        
            address: "{{ randomStreetAddress() }}, {{ randomCity() }}, {{ randomCountry() }}"

some_customer consists of four fields of which the email field has the fixed value "some_customer@example.org". By contrast, values of the three remaining fields are governed by Microcks template expressions for dynamic mock content. Upon mock request, Microcks evaluates such expressions and replaces them with evaluation results. For instance, the template expression "{{ randomFirstName() }}" for the field first_name leads to a capitalized random string of characters that makes up the potential first name of a person, e.g., "John" or "Jane". Similarly, the combined template expression "{{ randomStreetAddress() }}, {{ randomCity() }}, {{ randomCountry() }}" for the field address evaluates to a hypothetical address consisting of a random street, city, and country.

While the example object for GET /customer is specified for only the operation’s response, Microcks also recognizes pairs of example objects in which a response object is assigned to a certain request object. From those pairs, Microcks derives operation mocks that yield a given pair’s response object when the pair’s request object is sent to a mock. In the context of the example API and its definition, the some_customer example object of the POST /login operation denotes an example object that is spread over the operation’s request and response definition parts. In its web UI, Microcks displays matching pairs of requests and responses in the detailed view of operation mocks:

To illustrate the refinement of primary API definitions with example objects from secondary artifacts, the Git repository of the example API comprises the file "customers_examples.postman_collection.json." It contains the example API in Postman’s Collection format, which was used to specify an example for a successful invocation of the GET /customer_kinds operation:

Like primary API definitions, secondary artifacts can be uploaded to Microcks with Importers when ticking the checkbox "Is a Secondary Artifact?" in the import dialog. Secondary artifacts are a powerful feature of Microcks because they separate API definitions from the definition of mock messages. Consequently, they support usage scenarios in which, e.g., developers design and implement APIs based on a technical definition format whereas domain experts capture business-inspired API messages using dedicated formats and tools. Another use case for secondary artifacts is the externalization of mock message definitions that depend on Microcks-specific functionality. For example, the Microcks template expressions embodied above in the some_customer example object for the GET /customer operation aggravate the corresponding API definition’s publication in API catalogs. That is because definition users that are unfamiliar with Microcks likely do not know how to correctly interpret the template expressions. Additionally, the expressions make the API definition depend on Microcks with all the potential negative impacts of technology dependence such as decreased maintainability and compatibility. Starting from version 1.10.0, Microcks also comes with its own definition format for API examples.

Mocking Request Validations with Parameters Constraints

For the mocking of operations that check the values of HTTP headers or URL query parameters for well-formedness, validity, or other structural or semantic properties, Microcks provides the parameters constraints feature.

A Microcks parameters constraint consists of the following elements:

• Name: Unique identifier for the constraint. This identifier is equivalent to the name of the constrained HTTP header or URL query parameter.
• "Required" flag: Boolean value indicating whether the constrained HTTP header or URL query parameter of the given name must be present in valid mock requests.
• "Recopy" flag: Boolean value indicating whether the constrained HTTP header or URL query parameter of the given name and its value from a mock request shall be sent back to the caller as an outgoing HTTP header. Helpful to simulate, e.g., transaction propagation.
• Match constraint: Regular expression against which the value of a constrained HTTP header or URL query parameter must match for Microcks to forward the received request to a mock.

In the context of the example API, the mock for the GET /customer_kinds operation applies a parameters constraint to determine the expected value of the HTTP header field api_key. Effectively, this constraint simulates the behavior of Bearer tokens. In Microcks' web UI, parameters constraints are configurable from the mock details dialog:

Flexible Mock Routing with Dispatchers

Microcks relies on the concept of dispatching to determine the mock that shall be invoked upon request receipt. More precisely, mock dispatchers specify rules for dispatch criteria, i.e., those parts of incoming requests that Microcks shall match with rules to derive the most suitable response.

Microcks is able to automatically infer dispatchers from API definitions. For OpenAPI definitions, the following are commonly inferred dispatchers:

• URI_PARTS: Inferred for API operations that have only path parameters, e.g., GET /customer_kinds/{id} from the example API. Here, Microcks identifies the id path parameter as the distinguishing request part so that responses are sent depending on a concretely requested customer kind ID. If there is more than one path parameter, all parameters must match in logical conjunction, i.e., a response is only sent if it matches with the values of all requested path parameters.
• URI_PARAMS: Equivalent to URI_PARTS but for API operations with only query parameters.
• URI_ELEMENTS: Inferred for API operations with path and query parameters. The matching logic is based on the logical conjunction of path parameters or the logical conjunction of query parameters. That is, for a response to match either all path parameter values, all query parameter values, or both must match.

Other automatically inferred dispatchers include QUERY_ARGS (for GraphQL or gRPC operations) and QUERY_MATCH (for SOAP operations). Microcks' documentation describes those dispatchers in detail.

JSON_BODY and SCRIPT represent dispatchers that are not automatically inferred. Instead, they must be specified explicitly, either interactively by means of Microcks' web UI or programmatically by means of Microcks’ own API (see the upcoming last article of the three-part series). Therefore, JSON_BODY and SCRIPT are more advanced and powerful dispatchers than the inferred ones. They are also both applied in the Microcks-based mock provisioning for the example API.

JSON_BODY is a dispatcher that matches requests’ JSON bodies against expressions, which are themselves formulated in JSON based on three fields:

• exp: JSON Pointer that selects a specific body value against which the dispatcher shall match.
• operator: Comparison operator depending on the following cases clause. A complete list of comparison operators can be found in Microcks' documentation.
• cases: Mapping of selected body values to the names of corresponding mock responses in Microcks. A response is selected and sent back by the dispatcher to the requesting client when the mapped value matches with the value from the response’s body, as selected by exp, following the matching logic of the specified operator.

The example API applies the JSON_BODY dispatcher for mocking the POST /login operation. Here, the dispatcher is responsible for matching login requests to customer_token responses via the value provided in the password field of the received request. The corresponding dispatcher code is as follows:

{
  "exp": "/password",
  "operator": "equals",
  "cases": {
    "123456": "some_customer",
    "default": "invalid_credentials"
  }
}

With the "/password" JSON pointer, exp selects the value in the password field provided in a structurally valid mock request. The "equals" operator then checks for an exact match of the selected value with one of the mapping values in the cases field, e.g., the password "123456". The "default" case applies when none of the mapping values matches. The string value that is assigned to the mapping value of a case is interpreted by Microcks as the name of the mock response to be sent back when the case applies. For instance, when "123456" is received as value in the password field of a mock request, Microcks sends back the response named "some_customer", which is equivalent to sending back the customer_token of the customer whose password is "123456". As per the dispatcher, all other password values will result in the response named "invalid_credentials". Note that for illustrative purposes and the stripped down example API, this mapping is sufficient. However, for more complex test data with a variety of customers the mapping easily becomes ambiguous when for more than one test customer a trivial test password such as "123456" is specified. Such cases are better handled with Microcks SCRIPT dispatchers.

SCRIPT dispatchers determine mock responses from the execution of Groovy scripts. Since these dispatchers can resort to the functionality of a general-purpose programming language and its ecosystem, they are both the most versatile and error-prone kind of Microcks dispatchers.

Prior to the execution of SCRIPT dispatchers, Microcks sets up a runtime environment that enables access to certain elements of incoming requests via specialized Groovy objects stored in global variables:

• mockRequest: Object providing access to received requests. For instance, the requestContent attribute holds the request payload, whereas the getRequestHeaders() method returns a map of request header names to values.
• requestContext: Map that acts as a request-scoped storage for arbitrary information. Mock templates can access this information for sophisticated dynamic mocking.
• log: Object for the invocation of log methods such as info() or error(). Since logging seemingly is the only means for SCRIPT dispatcher debugging, keeping script complexity as low as possible is advisable.
• store: Map for string value storage that is persistent to an operation mock and as of Microcks 1.10.0 supports stateful mocks.

The mock of the example GET /customer operation involves a SCRIPT dispatcher to map customer-specific login tokens returned by POST /login to corresponding mock responses with customer details:

def headers = mockRequest.getRequestHeaders()
def loginToken = headers.get("customer_token", "null")
switch(loginToken) {
  case "eyJhbGciOiJIUzI1NiJ9....": return "some_customer"
  default: return "invalid_credentials"
}

First, the Groovy script retrieves the customer-specific loginToken from the customer_token header of a mock request to GET /customer. Next, the script decides which mock response to return by switch-casing over the loginToken. In case a concrete loginToken is known to the operation mock, the dispatcher returns the name of the mock response, e.g., "some_customer", that represents the customer details JSON object corresponding to the loginToken. This JSON object is then returned by Microcks to the caller. Alternatively, in case a provided loginToken does not map to a known customer mock response, the script returns the default mock response "invalid_credentials". In Microcks' web UI, the configuration of the SCRIPT dispatcher looks as follows:

Conclusion and Outlook

This second article in a three-part series about definition-based API mocking, simulation, and testing with Microcks described and exemplified further core features of the tool, namely mock data specification from defined example objects, parameters constraints for the mocking of request validations, and dispatchers to influence mock routing. While the derivation of mock data specification increases automation in API mocking, the latter features aim to decrease the efforts in mock refinement with low-code and no-code approaches, thereby targeting API functionality of increased complexity. The next and final article of the series will take a look at mock-guided contract testing with Microcks and the provisioning of consistent mock environments using Microcks’ own API. In addition, this concluding article discusses potential use cases and further characteristics of Microcks to support reasoning about its applicability in your own projects.