Living on the edge: building serverless applications with Cloudflare Workers

Cloudflare is best known for its CDN, DNS server (1.1.1.1) or WAF/DDos mitigation services. These services are highly predicated on “Edge Computing”, bringing data closer to the user interested in those services – a user in Australia will be happier if they receive their daily dose of funny cat videos from a server in Melbourne than one in Iceland. Amazon once calculated that every 100ms of latency cost them 1% of revenue, so latency matters. Cloudflare has data centers in over 300 locations and self-reports that 95% of internet users can reach a Cloudflare location in less than 50 ms (https://www.cloudflare.com/network/).

Static content is all well and good, but not particularly exciting for developers. However, in 2017 Cloudflare introduced its new product “Workers”, a serverless application platform targeted at application developers. Since then, the application development toolbox only has grown and it now includes:

• D1, a relational database based on SQLite
• R2, an S3 compatible object store (without egress charges!)
• KV, a key-value store (caching)
• “Durable Objects”, a special type of collaborative storage
• Queues, Pub/Sub
• Easy integration with LLMs

These days, it is definitely possible to create full-stack applications on Cloudflare. Many of these tools can be used on the free plan with fairly generous quotas, so trying it out is very easy and that is what we will be doing in this blog post: we are building an API that asks questions to a Cloudflare provided LLM. Question and Response will be stored in a relational database and a history can be queried.

The Workers Runtime

Cloudflare Workers are what would be Lambdas in AWS or Functions in Azure – bits of codes that get executed when called on servers maintained by Cloudflare. Cloudflare uses Chromium’s V8 JavaScript engine. This is important because it somewhat restricts what npm packages can be used as not all Node.js APIs are supported. It is not a complete Node runtime, but a lot of libraries are supported. There is also support for WebAssembly and Python, but we will use JavaScript (Typescript) in this article. Sidenote: using such a widely available runtime should be quite good for local testing, shouldn’t it?

Scaffolding a Cloudflare Worker application

While there are Cloudflare plugins for popular IaC providers like Terraform or the Serverless platform, we will explore the “default” tooling provided by Cloudflare, two Node-based tools called “C3” and “Wrangler”. C3 is a CLI that can be used to initialize a new Cloudflare project. It will interactively ask you a series of questions:

$ npm create cloudflare@latest                                                                                                                                                                                                                         1 ↵
Need to install the following packages:
create-cloudflare@2.33.0
Ok to proceed? (y) y

> npx
> create-cloudflare

──────────────────────────────────────────────────────────────────────────────────────────────────────────
👋 Welcome to create-cloudflare v2.33.0!
🧡 Let's get started.
📊 Cloudflare collects telemetry about your usage of Create-Cloudflare.

Learn more at: https://github.com/cloudflare/workers-sdk/blob/main/packages/create-cloudflare/telemetry.md
──────────────────────────────────────────────────────────────────────────────────────────────────────────

╭ Create an application with Cloudflare Step 1 of 3
│
├ In which directory do you want to create your application?
│ dir ./my-blog-example
│
├ What would you like to start with?
│ category Hello World example
│
├ Which template would you like to use?
│ type Hello World Worker
│
├ Which language do you want to use?
│ lang TypeScript
│
├ Copying template files
│ files copied to project directory
│
├ Updating name in `package.json`
│ updated `package.json`
│
├ Installing dependencies
│ installed via `npm install`
│
╰ Application created

╭ Configuring your application for Cloudflare Step 2 of 3
│
├ Installing @cloudflare/workers-types
│ installed via npm
│
├ Adding latest types to `tsconfig.json`
│ added @cloudflare/workers-types/2023-07-01
│
├ Retrieving current workerd compatibility date
│ compatibility date 2024-11-12
│
├ Do you want to use git for version control?
│ yes git
│
├ Initializing git repo
│ initialized git
│
├ Committing new files
│ git commit
│
╰ Application configured

╭ Deploy with Cloudflare Step 3 of 3
│
├ Do you want to deploy your application?
│ no deploy via `npm run deploy`
│
╰ Done

────────────────────────────────────────────────────────────
🎉  SUCCESS  Application created successfully!

💻 Continue Developing
Change directories: cd my-blog-example
Start dev server: npm run start
Deploy: npm run deploy

We created an application called “my-blog-example”, using the “Hello World” template and TypeScript. We chose not to deploy it yet.

This generates quite a lot of files that are common for node-based projects. We will mainly focus on two in the following: the source code of the created worker and the “wrangler.toml” configuration file.

The worker is unremarkable:

1export default {
2   async fetch(request, env, ctx): Promise<Response> {
3       return new Response('Hello World!');
4   },
5} satisfies ExportedHandler<Env>;

It takes the request (body, headers, HTTP method etc), the environment (secrets, access to other Cloudflare services) and an execution context that allows us to do some runtime tweaks (which we will ignore in this article). Currently, this just returns “Hello, World” regardless of the request. We can test it locally with “npm run dev” and “curl localhost:8787”: this indeed returns “Hello World”.

The wrangler.toml file at the moment mainly configures the name of the Worker and the location of the source code.

1name = "my-blog-example"
2main = "src/index.ts"
3compatibility_date = "2024-11-12"
4compatibility_flags = ["nodejs_compat"]

Implementing Routing with itty-router

That alone of course will not do for a web application. In the first step, we will add some routing based on the type of the request. A simple way of doing this in code is “itty-router”, an npm package that we will install with npm install itty-router –save.

For our simple example, we will start with providing an endpoint that receives a prompt, runs it against an LLM and returns the response from the LLM to the client.

index.ts will serve as our entrypoint into the application. In it, we will define a router:

1import {askLlm} from "./ask-llm";
2
3const router = Router()
4router
5   .post('/ai', askLlm )
6   .all('*', () => new Response('Not Found', {status: 404}))

This router will forward all POST requests to /ai to the “askLlm” handler which will be defined in another source file. All other requests will return a 404. In the default fetch function of the worker, we then simply delegate the request to the router:

1export default {
2   async fetch(request, env, ctx): Promise<Response> {
3      return router.fetch(request, env, ctx)
4   },
5} satisfies ExportedHandler<Env>;

Connecting the Worker to Cloudflare Workers AI

Before we can start implementing our LLM call, we need to connect the Worker to Cloudflare’s "Workers AI" service. This is done in two parts. First, we need to enable the AI service in wrangler.toml. To do so, we just need to uncomment two lines:

1[ai]
2binding = "AI"

With this, we gain access to the whole catalogue of Models provided by “Workers AI”. To enable our application to use it, we need to add AI to the Environment interface in the generated worker-configuration.d.ts file at the root of the project:

1interface Env {
2   AI: Ai;
3}

That’s it. We now need to implement the code that takes the request, talks to the LLM and returns the response. It is actually fairly simple:

1import { IRequest } from 'itty-router'
2export const askLlm = async (request: IRequest, env: Env) => {
3   let requestPayload: AiTextGenerationInput = await request.json()
4   console.log("Running prompt " + requestPayload.prompt);
5
6   let response: AiTextGenerationOutput = await env.AI.run('@cf/meta/llama-3-8b-instruct', requestPayload);
7
8   return new Response(JSON.stringify(response), {
9       headers: { 'content-type': 'application/json' }
10   });
11};

We call a method on the Ai object that specifies the used model (in this case a variant of Llama 3.8) and the request payload, in this case this is a simple text prompt. We then return the response (and confidently ignore any potential errors).

Time to take it for a spin. We run the local environment and give it a curl:

$ curl --request POST \
  --url http://localhost:8787/ai \
  --header 'Content-Type: application/json' \
  --data '{"prompt": "Answer just yes or no: is it a good idea to write a blog post about Cloudflare workers?"}'
{"response":"Yes."}%

When doing this for the first time, the runtime might open a browser for you to log in. Workers AI doesn’t run locally, so there are always calls to Cloudflare infrastructure. Workers AI is much more powerful than that and also has features like Text-to-Image or Image-To-Text and an OpenAI-compatible API, but this will be enough for this demo.

Creating a database

In the next step, we persist prompt and response in a relational database using D1. As mentioned above, D1 uses the highly popular SQLite under the hood. We create a database using the other main developer tool, Wrangler:

$ npx wrangler d1 create ai                                                                                                                                                                                                ↵

 ⛅️ wrangler 3.90.0
-------------------

✅ Successfully created DB 'ai' in region WNAM
Created your new D1 database.

[[d1_databases]]
binding = "DB"
database_name = "ai"
database_id = "f81d4fae-7dec-11d0-a765-00a0c91e6bf6"

This creates a DB locally and remotely, so you can actually test against a local instance. But wait a second – “in region WNAM”? That does sound odd, and in fact, our database was by default created in “Western North America”. That is one thing to keep in mind – as of now D1 is not a distributed database. We can finetune the (rough) geographic location though, and could tell it to spin it up in “Western Europe”. Distributing it is currently work in progress at Cloudflare as described on their blog.

We have seen something similar to [[d1_databases]] earlier in this article. That time we edited wrangler.toml to enable AI, and that is exactly what we are going to do now. We can copy that section from the Wrangler output and put it in that file.

1[ai]
2binding = "AI"
3
4[[d1_databases]]
5binding = "DB"
6database_name = "ai"
7database_id = "f81d4fae-7dec-11d0-a765-00a0c91e6bf6"

Then once again we edit the worker-configuration.d.ts:

1interface Env {
2   AI: Ai
3   DB: D1Database;
4}

Now we have done the necessary plumbing so our Worker can access the database, but it does not even have a schema yet! We use Wrangler and so-called "migrations" to create one:

$ npx wrangler d1 migrations create ai initial_creation     

This creates the file migrations/0001_initial_creation.sql. We can now add some SQL to that file:

1-- Migration number: 0001    2024-11-26T08:24:46.030Z
2CREATE TABLE ai_requests (
3   id INTEGER PRIMARY KEY AUTOINCREMENT,
4   prompt TEXT,
5   response TEXT,
6   createdAt TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL
7);

Then, we can apply this migration with npx wrangler d1 migrations apply ai [--remote|--local]. The last parameter decides whether we apply it to our remote db on Cloudflare or locally. Note that we do not need to include the filename – Wrangler keeps track of the migrations that have already been applied. The sequence of migrations is determined by the leading number in the filename. Applying the migration locally, we get the following output:

npx wrangler d1 migrations apply ai --local               

 ⛅️ wrangler 3.90.0
-------------------

Migrations to be applied:
┌───────────────────────────┐
│ name                      │
├───────────────────────────┤
│ 0001_initial_creation.sql │
└───────────────────────────┘
✔ About to apply 1 migration(s)
Your database may not be available to serve requests during the migration, continue? … yes

🌀 Executing on local database ai1 (f81d4fae-7dec-11d0-a765-00a0c91e6bf6) from .wrangler/state/v3/d1:
🌀 To execute on your remote database, add a --remote flag to your wrangler command.
🚣 1 command executed successfully.
┌───────────────────────────┬────────┐
│ name                      │ status │
├───────────────────────────┼────────┤
│ 0001_initial_creation.sql │ ✅     │
└───────────────────────────┴────────┘

(Sidenote: The author spent the better part of two hours battling weird SQL errors at that stage. He was missing the semicolon terminating the CREATE TABLE statement. Under the hood, D1 creates a table where it tracks executed migrations. It appends an INSERT statement to whatever a migration contains, and if the migration does not terminate with a semicolon, it will complain about a syntax error before an INSERT that developers do not see in their code, but it truncates the statement too soon to easily make sense of it.)

Now we can go back to the implementation and write prompts and responses to our database:

1import { IRequest } from 'itty-router'
2export const askLlm = async (request: IRequest, env: Env) => {
3   let requestPayload: Prompt = await request.json()
4   console.log("Running prompt " + requestPayload.prompt);
5   let response = await env.AI.run('@cf/meta/llama-3-8b-instruct', requestPayload);
6   try {
7       await env.DB.prepare(`
8         insert into ai_requests(prompt, response)
9         values
10         (?1, ?2)
11       `)
12           .bind(JSON.stringify(requestPayload), JSON.stringify(response))
13           .run()
14   } catch (e) {
15       let message;
16       if (e instanceof Error) message = e.message;
17       console.log({
18           message: message
19       });
20   }
21   return new Response(JSON.stringify(response), {
22       headers: { 'content-type': 'application/json' }
23   });
24};

There we are. We can test it locally and it works. Just writing to the database, of course, is not enough, we want the option to read the records again.

Extending the API

We first extend our API with a “history” endpoint by adding another route:

1import { Router } from "itty-router";
2import { askLlm } from "./ask-llm";
3import { getHistory } from "./getHistory";
4
5const router = Router()
6router
7   .get('/history', getHistory)
8   .post('/ai', askLlm )
9   .all('*', () => new Response('Not Found', {status: 404}))

In that endpoint, we make a simple database query and return the result:

1import { IRequest } from 'itty-router'
2export const getHistory = async (request: IRequest, env: Env) => {
3
4
5   let result : D1Result<Record<string, any>>;
6   try {
7       result = await env.DB.prepare(`
8       select * from ai_requests
9       `)
10       .all();
11   } catch (e) {
12      console.log("Error reading from db:"  + e);
13       throw e;
14   }
15
16   return new Response(JSON.stringify(result.results), {
17       headers: { 'content-type': 'application/json' }
18   });
19};

That should be it! Now it i's time to take it for a spin again:

$ curl localhost:8787/history | jq
[
   {
    "id": 1,
    "prompt": "{\"prompt\":\"Answer just yes or no: is it a good idea to write a blog post about Cloudflare workers?\"}",
    "response": "{\"response\":\"Yes.\"}",
    "createdAt": "2024-11-26 09:47:57"
  }
]

That request purely runs locally. Before we deploy to production, we should add some tests. We are not going to do that in this blog post, but it is worth mentioning that the application scaffold already contains a Vitest configuration for Cloudflare and the documentation describes the whole process well.

Deploying to Cloudflare

So let us assume we are happy with our application and tests and want to go to production. Of course Wrangler will once again be our tool of choice and we run npm run deploy:

> my-blog-example@0.0.0 deploy
> wrangler deploy


 ⛅️ wrangler 3.90.0
-------------------

Total Upload: 3.39 KiB / gzip: 1.42 KiB
Your worker has access to the following bindings:
- D1 Databases:
  - DB: ai (f81d4fae-7dec-11d0-a765-00a0c91e6bf6)
- AI:
  - Name: AI
Uploaded my-blog-example (4.81 sec)
Deployed my-blog-example triggers (5.76 sec)
  https://my-blog-example.hurz.workers.dev
Current Version ID: c037396b-c135-42a5-8e1d-3790617f629b

That gives us an URL to reach the application and we can promptly test it:

$ curl --request POST \
  --url https://my-blog-example.hurz.workers.dev/ai \
  --header 'Content-Type: application/json' \
  --data '{
        "prompt": "Answer just yes or no: is it a good idea to write a blog post about Cloudflare workers?"
}'
{"response":"Yes."}

We can use the D1 dashboard to check if the persistence worked: And of course our own endpoint delivers the same result:

1curl https://my-blog-example.hurz.workers.dev/history | jq
2[
3  {
4    "id": 1,
5    "prompt": "{\"prompt\":\"Answer just yes or no: is it a good idea to write a blog post about Cloudflare workers?\"}",
6    "response": "{\"response\":\"Yes.\"}",
7    "createdAt": "2024-11-26 10:10:14"
8  }
9]

A keen observer might notice that our API is public on the internet. That is indeed always the case and Cloudflare itself does not provide anything like User Pools etc. for authentication and authorization. You have to bring your own.

Conclusion

We created a simple API that interacts with an LLM and a database on Cloudflare. This gave us a good overview of what Cloudflare Workers is and what a development workflow with it looks like, although not all details could have been covered. This little application is completely free to run as it will not exceed the allowance of 100,000 requests per day. Our little database is free as well for up to 5 million reads / 100,000 writes a day and a maximum storage of 5 GB. The standard paid subscription at the time of writing costs $5 per month and increases these values significantly and adds additional features like queueing, making Cloudflare a very interesting playground for new ideas. It is not a drop-in replacement for a public cloud provider like AWS or Azure, but the pricing for the features it does provide is very competitive and it is definitely powerful enough to host full stack applications. Give it a go!