In this blog post we will focus on creating the step function (state machine) that coordinates our Lambda workload. Our Lambdas will read from S3, transform data, and store this into the RDS instance we created in part 3 and part 4 of our blog series.
By the way, if you are curious about reducing VPC costs with Lambda, read more about it at: https://blog.codecentric.de/en/2019/09/reduce-vpc-costs-lambda-step-functions/ .
In Part 1, 2, 3 and 4 of this blog series we described how to create a custom VPC including security groups and subnets, with S3 buckets, and an RDS instance. These first steps represent our infrastructure that is the foundation for our new architectural setup with AWS CDK using TypeScript. If you are just beginning to use AWS CDK and want to know how to get started, we recommend you start reading Part 1. This blog post is part five of our six-part blog series on AWS CDK:
- How to create a custom VPC
- How to create S3 buckets
- How to create an RDS instance
- How to create Lambdas
- How to create a step function
- CDK lessons learned
From this point onward, we assume you have completed everything discussed in the previous parts, that everything is compiling and you successfully deployed the VPC, S3 bucket, RDS instance, and Lambdas to your AWS account.
The Lambdas
Following up on our ‘simplified’ part 4, we will expand our lambda-stack.ts file and add more Lambdas using our previously defined createVpcLambda function.
//lambda-stack.ts
const stepFunctionA = ApplicationStack.createVpcLambda(this, 'rdsLambdaA', 'triggers/rdsLambdaA.trigger', props, secret, s3AccessRole);
const stepFunctionB = ApplicationStack.createVpcLambda(this, 'rdsLambdaB', 'triggers/rdsLambdaB.trigger', props, secret, s3AccessRole);
const stepFunctionC = ApplicationStack.createVpcLambda(this, 'rdsLambdaC', 'triggers/rdsLambdaC.trigger', props, secret, s3AccessRole);
const taskEventA = new Task(this, 'stepFunctionA', {
task: new InvokeFunction(stepFunctionA)
});
const taskEventB = new Task(this, 'stepFunctionB', {
task: new InvokeFunction(stepFunctionB)
});
const taskEventC = new Task(this, 'stepFunctionC', {
task: new InvokeFunction(stepFunctionC)
});
// step functions here
Next, we will continue by adding our step function definitions. Now we create the file ./lib/application-stack.ts containing the following code:
//lambda-stack.ts
const taskEventA = new Task(this, 'stepFunctionA', {
task: new InvokeFunction(stepFunctionA)
});
const taskEventB = new Task(this, 'stepFunctionB', {
task: new InvokeFunction(stepFunctionB)
});
const taskEventC = new Task(this, 'stepFunctionC', {
task: new InvokeFunction(stepFunctionC)
});
const stepChain = new Parallel(this, 'taskParallelTasks', {}).branch(taskEventA).branch(taskEventB).branch(taskEventC);
const stateMachine = new StateMachine(this, 'StateMachine', {
definition: stepChain,
timeout: Duration.minutes(3)
});
As you can see, for each Lambda there is a corresponding taskEvent. Step functions can of course be very complex with decisions and recursive flow, however, for our purpose, we only want to parallelize the Lamba workload. Hence we will add a chain to our flow by calling the branch() function. The statemachine API provides a quit semantic API to communicate your flow. We additionally added a timeout which is valid for the complete workflow.
Packaging the Lambdas
Let us shift our focus back to our CDK infrastructure: we need to include the Lambdas we want to deploy. As shown highlighted in part 4 of the blog, the Lambdas are in mono-repo style. We are managing multiple Lambdas in one directory.
1// Directory structure for part 5: 2. 3├── bin 4│ └── part5.ts 5├── cdk.json 6├── lambdas 7│ ├── package-lock.json 8│ ├── package.json 9│ ├── src 10│ │ └── triggers 11│ │ ├── rdsLambdaA.ts 12│ │ ├── rdsLambdaB.ts 13│ │ ├── rdsLambdaC.ts 14│ │ └── vpcProcessing.ts 15│ ├── tsconfig.json 16│ └── yarn.lock 17├── lib 18│ ├── lambda-stack.ts 19│ ├── rds-stack.ts 20│ ├── s3-stack.ts 21│ └── vpc-stack.ts 22├── package-lock.json 23├── package.json 24├── tsconfig.json 25└── yarn.lock
Before we can deploy our Lambda functions to the AWS cloud, we need to package them. In order to do so, run the following commands in your console.
1cd lambdas && npm i && npm run build-ts && npm run package-cdk
The result of this is a new folder inside the Lambdas directory called ‘deployment’, which contains the production dependencies (node_modules) and the .js Lambdas needed for a successful deployment to AWS.
Last but not least, let us check if our new setup actually compiles to an updated Cloudformation template. Run the following commands:
1npm run build && cdk synth
The console output should log that the stack was synthesized successfully. At this point you could deploy your new stack.
Final build & deploy
We are all set up and ready to deploy our new CDK stack to our AWS cloud. In Part 1 we have already set up our credentials, so this time we can build and deploy by simply running the following commands:
1npm run build && cdk synth
After successfully having synthesized the Cloudformation template, you can comfortably check what changed by running the command:
1cdk diff --profile sample
Finally, we deploy the changes made to the AWS cloud by running the command
1cdk deploy --profile sample
Upon signing in to AWS Cloudformation you should see the Lambda stack being created.
A finished application
There you go! In five steps you have created an elaborate Serverless application stack that you can manage professionally using CDK. If you enter the step functions service in the AWS console, you will get a nice visualisation of our state machine. For any workload you will find a detailed flow log of inputs, outputs, possible errors, and handy links to Cloudwatch for your logs.
We will finish off this blog series with our evaluation of the use of CDK for Serverless in our next blog.
Was this post helpful?
More articles
fromKevin van & Maik Kingma
31.12.2019 Your job at codecentric?
Jobs
Agile Developer und Consultant (w/d/m)
Alle Standorte
More articles in this subject area
Discover exciting further topics and let the codecentric world inspire you.
Gemeinsam bessere Projekte umsetzen.
Wir helfen deinem Unternehmen.
Du stehst vor einer großen IT-Herausforderung? Wir sorgen für eine maßgeschneiderte Unterstützung. Informiere dich jetzt.
Hilf uns, noch besser zu werden.
Wir sind immer auf der Suche nach neuen Talenten. Auch für dich ist die passende Stelle dabei.
Blog authors
Kevin van
Do you still have questions? Just send me a message.
Maik Kingma
Do you still have questions? Just send me a message.
Do you still have questions? Just send me a message.