Python and CDK (Part 1): Developing AWS Lambda functions with Python and CDK

This blog post assumes that you are familiar with Python development and know the basic concepts of Amazon CDK. What's more, you should have an AWS account and have configured the AWS CLI. If you're new to CDK, go here, if you need to configure the AWS CLI, go here.

In the following, I'll guide you through setting up a CDK project as well as a Python project and make them work together. We will install all required dependencies and tools together. The following topics are covered:

• CDK project setup
• Python project setup
• Writing and testing a lambda
• Deploying the lambda to the cloud

CDK project setup

Even tough there is a CDK binding for Python, we are going to use TypeScript as CDK language. TypeScript has the largest community amongst the CDK-supported languages, which makes it easier to get answers for particular problems, e.g. on Stack Overflow. Also, there are more third party constructs available. At the time of this writing, constructs.dev hosts 1274 constructs for TypeScript and only 971 for Python.

Therefore, our the first step is to install Node.js and npm. Open this link to the Node.js documentation and follow the installation instructions.

Once Node.js and npm are installed, we'll continue to install the CDK CLI tool.

1npm install -g aws-cdk

Now create your project folder. I named mine serverless-python.

1mkdir serverless-python
2cd serverless-python

Within the project folder, we're using the CDK CLI tool to initialize our CDK project.

1cdk init app --language typescript

For now, that's all we have to do.

Python project setup

At the time of this writing, AWS only supports Python up to version 3.9. The Python version provided by your OS will probably not match. Thus, we will use pyenv to install the Python version we like without making changes to the Python version of your OS. Especially on Linux, this could cause severe issues with your desktop manager.

On Linux and MacOS, make sure you've installed all required dependencies here

Next, follow the installation instruction:

• Linux and MacOS here
• Windows here

Now, we install the Python version we like to use.

1pyenv install 3.9.16

Within our project folder, we will set the local application-specific Python version. This will create a .python-version file in our project folder that contains the selected version. pyenv uses shims to intercept Python and pip call and redirects them to the correct version. If you're interested in the details, read the documentation here.

1pyenv local 3.9.16

To manage our Python dependencies and packaging, we'll use Poetry. Poetry lets us declare our project libraries and manage (install/update) them for us.

To install Poetry, follow the installation instructions here.

Having Poetry installed, we can create our Python project. Poetry will create a subfolder with the name you choose, in my case sample-app. For simplicity, both the python and the cdk project will share the same project folder. As we only require the generated pyproject.toml we will copy that to our project folder and delete the generated sample-app directory. The pyproject.toml contains our dependencies and other build system requirements and information.

1poetry new sample-app
2cp sample-app/pyproject.toml .
3rm -rf sample-app

Afterward, we tell Poetry which Python version to use. This works because we set the Python version with pyenv earlier, and pyenv's shim makes the python version visible to poetry.

1poetry env use 3.9.16

Every project requires a style guide. For Python, the most popular one is PEP 8. Thus, we conclude the project setup by installing a linter and formatter. The linter we are going to use is Flake8, which easily verifies that we comply with PEP 8. The formatter is Black, which applies a subset of PEP 8. Its configuration is uncompromisingly simple – there isn't any.

We install both by adding them as development dependencies with poetry.

1poetry add --group dev black flake8

To make Flake8 work nicely with Black, add the following configuration file to your project directory and paste the following snippet. If you're interested in the configuration options, have a look at the Black documentation here.

.flake8

1[flake8]
2max-line-length = 88
3extend-ignore = E203

Hello lambda!

Now it's time to write our first lambda, the “Hello world” (of course).

We keep our code structure simple and put our lambdas into the src directory and tests into the test directory.

1mkdir src
2mkdir test

Next, we'll define the deployment unit for our "Hello World" lambda by creating a package in src. Deployment unit means that only the contents of that package will be deployed to AWS Lambda. Don't forget to create an empty __init__.py file in src and test and all subfolders, so Python can pick up the packages.

1mkdir src/hello
2mkdir test/hello
3touch src/__init__.py
4touch test/__init__.py
5touch src/hello/__init__.py
6touch test/hello/__init__.py

Finally, let's write some code. Our simple lambda will take a name from the event parameter, compose a greeting, print the greeting and return it.

src/hello/hello.py

1def handler(event, _):
2    greeting = f"Hello {event['name']}!"
3    print(greeting)
4    return {"greeting" : greeting}

Every piece of code is only complete with a test. The unit test below passes a name in the event and verifies the greeting message.

test/hello/test_hello.py

1from src.hello.hello import handler
2
3def test_should_return_hello_name():
4    response = handler({"name": "John"}, None)
5    assert response["greeting"] == "Hello John!"

Before we can run the test, we need to add pytest as test runner and configuration it.

Install pytest with poetry as development dependency.

1poetry add --group dev pytest

Then append the following pytest configuration in the pyproject.toml.

pyproject.toml

1[tool.pytest.ini_options]
2minversion = "6.0"
3addopts = "-ra -q"
4testpaths = [
5    "test",
6]

To run the tests, execute the pytest command via Poetry.

1poetry run pytest

With our "Hello lambda" written and tested, we can deploy it. So we define a new lambda function in our CDK stack. The function's code property refers to our hello package, and the handler refers to the file and method which will be called by AWS.

lib/python-serverless-stack.ts

1import { RemovalPolicy, Stack, StackProps } from 'aws-cdk-lib';
2import { Code, Function, Runtime } from 'aws-cdk-lib/aws-lambda';
3import { LogGroup, RetentionDays } from 'aws-cdk-lib/aws-logs';
4import { Construct } from 'constructs';
5
6export class PythonServerlessStack extends Stack {
7  constructor(scope: Construct, id: string, props?: StackProps) {
8    super(scope, id, props);
9
10    new Function(this, "HelloLambda", {
11      functionName: "hello-lambda"
12      runtime: Runtime.PYTHON_3_9,
13      code: Code.fromAsset('src/hello/'),
14      handler: 'hello.handler',
15    });
16  }
17}

And deploy...

1cdk deploy PythonServerlessStack

After the deployment is done, we can test our lambda by invoking it through the AWS CLI.

1aws lambda invoke \
2    --function-name hello-lambda \
3    --payload '{ "name": "World" }' \
4    response.json

response.json

1{"greeting" : "Hello World!"}

Continue reading: Part 2 - Deploy Python dependencies for AWS Lambdas with CDK

Additional resources

• AWS CDK Reference Documentation
• AWS Lambda Powertools for Python
• Boto3 – AWS SDK for Python
• Assertpy – Simple assertion library for unit testing with a fluent API