Legacy SOAP API integration with Java, AWS Lambda and AWS API Gateway

Introduction

Once you have decided to migrate your infrastructure to AWS, the migration process is usually not executed at once. Instead there will most likely be a transition period, in which both, new and legacy infrastructure, have to coexist and communicate with each other. In this transition period the existing systems are gradually migrated to the cloud environment. However, sooner or later it might be possible that you run into compatibility problems, because a legacy system cannot be incorporated into the cloud (for whatever reasons) or refuses to communicate with modern API interfaces. There could, for instance, be clients which can have their API endpoint configured, but cannot be changed with regard to the message format they send to this endpoint. For this kind of purpose the AWS API Gateway offers several options to integrate incoming requests and outgoing responses into the cloud infrastructure.

In this article I want to show a basic solution of how to integrate requests with the AWS API Gateway and AWS Lambda using the example of a SOAP request.

Prerequisites

A basic understanding of the AWS platform as well as an AWS account are required. Also you should be familiar with Java and Maven . The full sample code, used in this article, can be found on GitHub .

The Plan

We will create an AWS API Gateway resource, which receives and processes a SOAP message and returns a SOAP message as response. In order to achieve this, we implement a Java Lambda function, which is configured as an integration point in the method execution of our resource. The API Gateway is in turn responsible for mapping the incoming request and the outgoing response to corresponding content types.

Let’s start with setting up the Lambda function.

Set up Lambda

We start with a Java 8 implementation of the RequestHandler interface provided by the AWS Java SDK. Because Lambdas are only able to process JSON, the API Gateway has to map the incoming SOAP request correspondingly (I elaborate on this point in the “Integration Request” section of this article). To process the mapped request we create a Wrapper class, which can be instantiated with the JSON String. This wrapper object contains the original XML within a String field and can be handed to the RequestHandler implementation for processing.

Include libraries

We create a Java 8 Maven project and add the following dependencies to the pom.xml:

<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-lambda-java-core</artifactId>
    <version>1.1.0</version>
</dependency>
<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-lambda-java-events</artifactId>
    <version>1.3.0</version>
</dependency>
<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-lambda-java-log4j</artifactId>
    <version>1.0.0</version>
</dependency>

Please note that in most applications the “full” AWS SDK is added to implement all kinds of use cases. But as we want to keep the Lambda function as compact as possible, we only include the minimum set of dependencies required for the execution of the function.

Create Wrapper Class

The SoapWrapper class is a simple POJO, containing the XML request / response as a String:

public class SoapWrapper {

    private String body;

    public SoapWrapper() {}

    public SoapWrapper(String body) {
        this.body = body;
    }

    public void setBody(String body) {
        this.body = body;
    }

    public String getBody() {
        return body;
    }
// ...
}

Implement Request Handler

The implementation of the RequestHandler interface expects a SoapWrapper object as input and returns a SoapWrapper object as response. The AWS Lambda execution environment will take care of the JSON serialization and deserialization for us, as long as the respective class offers a default constructor and setters for the fields.

public class ApiRequestHandler implements RequestHandler<SoapWrapper, SoapWrapper> {

    @Override
    public SoapWrapper handleRequest(SoapWrapper request, Context context) {
        // ...
    }
}

To verify that the SoapWrapper object works as intended, we parse the String content of the body field to a Java SOAPMessage. Afterwards we return a hard coded SOAPMessage as response to test the end to end scenario. Feel free to take a look at the code in the sample project in GitHub for further reference.

Package Lambda

Java Lambdas need all classes that are required for the execution of the program in a single jar file. Hence Maven has to package these classes into a so called “fat jar”, which comprises all necessary runtime dependencies. This can easily be achieved by including the shade plugin into the pom.xml:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-shade-plugin</artifactId>
    <version>2.4.3</version>
    <configuration>
        <createDependencyReducedPom>false</createDependencyReducedPom>
    </configuration>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>shade</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Finally we create our jar file with mvn clean package.

Configure and deploy Lambda

To configure and deploy the Lambda function, log into the AWS console and go to the Lambda service:

1. Hit “Create a Lambda function”
2. Select the “Blank Function” blueprint
3. Skip the “Configure triggers” section with “Next”
4. Provide a meaningful name for the function
5. Select “Java 8” as Runtime
6. For the code entry type select “Upload a .ZIP or .JAR file” and upload the previously created fat jar. The maven shade plugin actually creates two jar files, so make sure to select the one without the “original-” prefix. Amazon recommends that packages larger than 10 MB should be uploaded to AWS S3. Java Lambdas almost always exceed this threshold, but for the time being upload the jar file manually
7. Afterwards provide the handler, which is the fully qualfied name of the class implementing the RequestHandler interface (e.g. de.codecentric.lambda.ApiRequestHandler)
8. Role: Depending on what the Lambda function should do, it needs the appropriate rights to do so. Basic execution is sufficient for our purpose, hence select “Create a custom role”. Click on “Allow” in the following AIM service window
9. Finally leave the “Advanced Settings” section untouched and proceed with “Next” to review the input

Test Lambda

Now that we have deployed our RequestHandler implementation, we can test the execution with a simple JSON document (containing an escaped SOAP XML), which we paste directly into the editor on the AWS website. Select the Lambda function in the AWS Lambda service and click on “Actions”, “Configure test event”, enter the following and hit “Save and test”:

{
  "body": "<SOAP-ENV:Envelope xmlns:SOAP-ENV=\"<SOAP-ENV:Envelope xmlns:SOAP-ENV=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:codecentric=\"https://www.codecentric.de\"><SOAP-ENV:Header/><SOAP-ENV:Body><codecentric:location><codecentric:place>Berlin</codecentric:place></codecentric:location></SOAP-ENV:Body></SOAP-ENV:Envelope>"
}

A successful test should not have raised exceptions, and we should see the incoming request as log output in the AWS CloudWatch log files. If the Lambda function works as intended, we can proceed to set up the API Gateway.

Set up API Gateway

Using the AWS Management Console for the API Gateway, we are able to set up our SOAP Wrapper API within minutes. We just have to keep in mind to map the incoming request content, which is XML, to JSON (as the Lambda function only speaks JSON). Conversely we map the outgoing response content to XML, in order to emulate an actual SOAP response. This can be done with an Integration Request and an Integration Response within the AWS API Gateway, respectively. We define a content type and a mapping template in each of these method execution steps to process the body of the request / response. Within the mapping template we can modify the content of a request / response with Velocity .

Create API, Resource and Method

1. Go to the API Gateway service and click “Create API”
2. Select “New API”, input a name (e.g. “soapApi”) and hit “Create API”
3. Select the API, push the “Actions” button, select “Create Resource”, provide a resource name (e.g. “legacy”) and hit “Create Resource”
4. Select the resource, hit “Actions” again, select “Create Method” and then “POST”. Confirm
5. Wire the lambda function with the API in the following window: Select “Lambda function” integration type, specify Region and function name, then hit “Save”
6. Confirm the permission request for the API Gateway in the following window

After the API is sucessfully created, we can see the visualized “Method Execution” when we select our POST method:

Integration Request

In the “Method Execution”, click on the “Integration Request” and open the “Body mapping Templates” section. Select “Add mapping template” and type in “text/xml”. Then simply “jsonify” the whole request with the following Velocity snippet:

{
   "body" : $input.json('$')
}

As the SoapWrapper Java Class expects a single JSON element “body”, we define the JSON object accordingly. Because the Java SOAP library sends requests with text/xml as content type we provide the type analogically. Depending on the migration scenario and which tools are used to execute the request, it might be necessary to adjust the content type appropriate to the scenario. Furthermore depending on the selected “body passthrough” option, the API Gateway either rejects requests not matching the content type, or passes them through “as is”. Having finished the Integration Request, the Lambda function should already be able to receive SOAP messages from the API Gateway. Finally, we take care of the response.

Integration Response

The Lambda function so far delivers a SoapWrapper object as JSON. Yet, what we actually need is XML. Hence we map the response to the respective content type and message body. For that purpose click on “Integration Response” in the “Method Execution”, unfold the existing response and the “Body Template” section. In the ensuing step, change the content type from application/json to application/xml and return the body of the SoapWrapper response (which contains the XML as String) with the following Velocity snippet:

#set($inputRoot = $input.path('$'))
<?xml version="1.0" encoding="UTF-8"?>
$inputRoot.body

Method Response

For the finishing touch of our response, we define a “Method Response” for the HTTP status code 200 with application/soap+xml as content type:

Deploy API

In order to test our created resource, we deploy the API to an arbitrary deployment stage, e.g. “test”. To do so, simply select the API, hit “Actions” and “Deploy API”. We receive an endpoint URL after the deployment, which can be used in the next step to test the interaction of API Gateway and Lambda function.

Test interaction of API and Lambda

The project on GitHub provides an integration test (WrapperApiEndpointIntegrationTest), which sends a post request to the specified endpoint URL (which we have received in the preceding “Deploy API” step). Of course we should also be able to test with any software capable of sending a POST request and receiving a response.

Conclusion

While SOAP is no longer supported on the AWS API Gateway, you can still include legacy SOAP requests in your new shiny cloud infrastructure, at least for a transition period. Of course the „soap legacy” resource requires further development; eg. we did not go into security: it is mandatory to put some thought in authentication and authorization. Also your legacy system might need SOAP headers or other parameters which have to be included in your request. Furthermore we are lacking a WSDL file to describe our API. It is also worth mentioning, that your AWS infrastructure probably resides within a VPC network, in which case you might need further configuration in terms of AWS networking. It also stands to question if Java should be the programming language of choice for this kind of purpose. If you have infrequent, unpredictable and spiky API calls and the function’s runtime is rather short, a programming language with less ramp up time could be the better option. But this also depends on the specific purpose of the API call, and which libraries are needed to process the request at hand.

Obviously Integration Requests and Responses are not limited to SOAP. With Velocity you can map a vast amount of requests and responses to all kinds of formats and spin up an API within minutes.