Developing AWS locally with Serverless offline plugins

You are using the Serverless framework on AWS ? Did you know that you can develop on your local machine as well with little effort? For this purpose, a variety of Serverless offline plugins are available which make it possible to use services like Lambda, DynamoDB or S3 locally. In this article, I will show you how to get started easily.

I will provide an example reduced to the bare necessities, and will only go into detail on the relevant parts of the offline plugins. I will get an AWS stack up and running locally, based on Node.js , API Gateway, lambdas, and DynamoDB.

Why offline plugins?

There are many reasons for using cloud-based services. Our blog contains a lot of great examples, maybe you want to have a look at these:

• The article Google Cloud Function for Machine Learning shows how to implement some cloud-based machine learning
• Going serverless: How to move files from on-prem SFTP to AWS S3 demonstrates how to switch from FTP to cloud-based storage

For developers, new obstacles are emerging that have been less critical in classical approaches. When the development environment is moved completely to a cloud provider, it usually takes more time and effort until a code change finally becomes visible in the product.

If you already work with the Serverless framework, you can benefit from the variety of available offline plugins to get your cloud project up and running on your local machine. The idea behind it is simple: the code you execute locally is also executed in the cloud.

Prerequisites

You should at least know the basics of the Serverless framework and also have it installed. You can check that easily by executing

1$ serverless --version

in your shell and see if it responds with a valid version.

Furthermore, you should have Node.js installed and also understand the basics of this language.

Preparations

If you want to reproduce the following example, please clone it first from this repository . Additionally, we need some plugins. These statements:

1$ npm install aws-sdk
2$ npm install express
3$ npm install serverless-http
4$ npm install serverless-dynamodb-local
5$ npm install serverless-offline
6$ sls dynamodb install

will install all required plugins. serverless-dynamodb-local and serverless-offline are the candidates which enable us to develop locally.

Configuring plugins

As soon as all plugins are installed, they need to be included in the serverless project. This is done in the plugins-block.

1plugins:
2  - serverless-dynamodb-local
3  - serverless-offline  # should be last in list

of the serverless.yml-file. This will make the plugins available for our project. Some minor configuration is required which we will add to the custom-section:

1tableNames:
2  persons: 'cc-persons'
3    
4dynamodb:
5  start:
6    migrate: true
7  stages:
8    - dev

tableNames defines the name of the DynamoDB table, so that we can reuse it in other places. migrate ensures that the required DynamoDB tables are created automatically on start up.

In projects where no local plugins are used, the AWS SDK automatically takes care of the AWS configuration. But for local plugins, that behavior is restricted: We need to tell the SDK the DynamoDB endpoint later. That’s why we configure it via

1custom:
2  endpoints:
3    dynamodb-url: 'http://localhost:8000'

and publish the required values via

1provider:
2  environment:
3    CONFIG_PERSONS_TABLE: ${self:custom.tableNames.persons}
4    CONFIG_DYNAMODB_ENDPOINT: ${self:custom.endpoints.dynamodb-url}

as environment variables for later use. Finally, we need to create a DynamoDB table. We create a resource for it, described in a separate file resources/persons-table.yml, and embed it via

1resources:
2  - ${file(resources/persons-table.yml)}

into the resources-block of the serverless.yml-file.

Calling DynamoDB from the Node.js lambda

The Serverless framework will take care of creating tables. However, the access to the tables is implemented by ourselves in the Node.js lambdas. The project already contains two simple services which are defined in the serverless.yml files functions block:

• GET on /persons
• POST on /persons

The implementation is placed in handler/persons.js. It makes use of the Express framework , further details are not relevant for us at this point.

All calls on the DynamoDB that are invoked in our lambdas are encapsulated by the AWS SDK. However, the SDK does not know about our local DynamoDB yet and would connect us with the AWS cloud and redirect all queries to the cloud. Thus, we need to tell the AWS SDK which DynamoDB endpoint it should use while we are in offline mode.

To find out whether we are in offline mode, the serverless offline plugin provides an environment variable which we can check: IS_OFFLINE. We can use this variable to build a switch, overriding the endpoint in local mode.

1const CONFIG_PERSONS_TABLE = process.env.CONFIG_PERSONS_TABLE;
2const CONFIG_DYNAMODB_ENDPOINT = process.env.CONFIG_DYNAMODB_ENDPOINT;
3const IS_OFFLINE = process.env.IS_OFFLINE;
4 
5let dynamoDb;
6if (IS_OFFLINE === 'true') {
7  dynamoDb = new AWS.DynamoDB.DocumentClient({
8    region: 'localhost',
9    endpoint: CONFIG_DYNAMODB_ENDPOINT,
10  });
11} else {
12  dynamoDb = new AWS.DynamoDB.DocumentClient();
13}

With this mechanism, the endpoint used in local mode is different than in normal (“cloud”) mode, namely the one we defined in the serverless.yml file as environment variable CONFIG_DYNAMODB_ENDPOINT.

Function testing

All other sources in this project look exactly the same as they would without using offline plugins, and therefore are not relevant for this article. We already made all required adjustments. Let’s now start testing if everything works as expected. With the statement

1$ serverless offline start

the whole project will be started locally. Whether or not everything works correctly can be seen in the log output, for example

1Dynamodb Local Started, Visit: http://localhost:8000/shell
2Serverless: DynamoDB - created table cc-persons
3Serverless: Starting Offline: dev/eu-central-1.
4Serverless: Offline [HTTP] listening on http://localhost:3000

We can see that a local DynamoDB was started, and that the table cc-persons was created. Let’s now find out whether our services work correctly by executing the following statements:

1$ curl -XPOST localhost:3000/persons
2$ curl localhost:3000/persons

consecutively. This should create a new person and then return it back.

More opportunities

There are much more serverless offline plugins for the Serverless framework. For example, I tried serverless-offline-ssm and serverless-s3-local successfully. Using serverless offline plugins enables a developer to run an entire AWS stack locally and in an isolated manner, without always having the cloud infrastructure available. By using serverless offline plugins, changes become directly visible without any deployment needed and therefore the length of development cycles can be reduced significantly.

Questions, wishes, suggestions? Just drop a comment and I will see what I can do for you. 🙂