Generating Kotlin DSLs

Introduction

This article will give an introduction to Kotlin DSLs (Domain-Specific Languages) and teach you how to create and generate them with an annotation processor.
We’ll also take a look at Kotlin internals and how we can use this knowledge to our advantage.

I’m only going to give a short introduction to DSLs here. For a more in-depth look at what (Kotlin) DSLs are, read this article showcasing an Apache Kafka Kotlin DSL.
With that said, let’s get into it:

What is a DSL?

DSLs are purpose-built languages used to describe objects of a certain domain. For example, HTML is a DSL to describe web pages.
An easy way to define a DSL is to make it a subset of an existing language: HTML is a subset of XML, npm configuration files are a subset of JSON, and Gradle build files can be written with a subset of Groovy or Kotlin.

Why Kotlin DSLs?

While Gradle is the most well-known case of a Kotlin DSL for configuration, anybody can define their own DSL for any use case. For example, liquibase supports many configuration formats, including Kotlin .
Kotlin DSLs have a major advantage over most alternatives: they’re essentially code, which means you can:

• write logic into your configuration (e.g. easily changing any part of your configuration based on environment)
• get IDE (and compiler) support for syntax and type checking, formatting and autocompletion

Creating a Kotlin DSL

Let’s take a look at a simple example:

1person {
2    age = 24
3    name = "Lukas Morawietz"
4    hands = 2
5}

Looks nice, right?
Here’s what we need to define such a DSL:

1data class Person(val age: Int, val name: String, val hands: Int)
2 
3data class PersonBuilder {
4    var age: Int? = null
5    var name: String? = null
6    var hands: Int? = null
7 
8    fun build(): Person {
9        checkNotNull(age) { "age must be assigned." }
10        checkNotNull(name) { "name must be assigned." }
11        checkNotNull(hands) { "hands must be assigned." }
12        return Person(age, name, hands)
13    }
14}
15 
16fun person(initializer: PersonBuilder.() -> Unit): Person
17    = PersonBuilder().apply(initializer).build()

This part doesn’t look as nice anymore, basically everything past the first line is boilerplate code.

Generating a Kotlin DSL

Luckily, we don’t need to write all that boilerplate code, we can just write code to generate it for us – using annotation processing.
We can use either ksp or kapt to generate code based on annotations and the Abstract Syntax Tree (AST) during compilation.
In the end we want our code to look like this:

1@AutoDSL
2data class Person(val age: Int, val name: String, val hands: Int)

And have the processor generate all the boilerplate we had above for us.

How do we do that?

We just have a look at the constructor of our annotated class, and generate a builder class with the same fields, except mutable and nullable.
The build method isn’t complex either. Just check every field was set and then return a new instance of our target class.
Finally, the DSL entry method is also simple: we create a new builder, apply the initializer lambda and build.
The full code for this is a bit long for this format, but you can find my implementation of AutoDSL here .

Problem: Default values

Most people have two hands, so let’s set that as a default value:

1@AutoDSL
2data class Person(val age: Int, val name: String, val hands: Int = 2)

The builder we’d write by hand could then maybe look like this:

1data class PersonBuilder {
2    var age: Int? = null
3    var name: String? = null
4    var hands: Int = 2
5 
6    fun build(): Person {
7        check(age != null) { "age must be assigned." }
8        check(name != null) { "name must be assigned." }
9        return Person(age, name, hands)
10    }
11}

However, there are two problems with this. First, we’re duplicating the default value. Especially if the default value is a more complex expression this might not be desirable. Second and more importantly, our annotation processing tools don’t give us access to default values at all, we can only see if there is one.

How Kotlin default values work

To find an alternative solution for this, we need to take a look at how default values are implemented in Kotlin internally. Here’s the decompiled Java equivalent of our Kotlin class (I’ve used IntelliJ here to view and decompile the Kotlin bytecode):

1class Person {
2 
3  public Person(int age, @NotNull String name, int hands) {
4    Intrinsics.checkNotNullParameter(name, "name");
5    super();
6    this.age = age;
7    this.name = name;
8    this.hands = hands;
9  }
10 
11  // synthetic method
12  public Person(int var1, String var2, int var3, int var4, DefaultConstructorMarker var5) {
13    if ((var4 & 4) != 0) {
14      var3 = 2;
15    }
16 
17    this(var1, var2, var3);
18  }
19 
20  //fields and methods excluded for brevity
21}

Okay, so at the top we see a normal constructor as we would expect, but at the bottom it gets interesting: here we have a synthetic constructor with two additional arguments: an integer and a DefaultConstructorMarker. We’re going to ignore the second one, it’s just there to make sure the constructor signature can’t conflict with other existing constructors.
The other (here var4) is interesting though – as we can see from the implementation it’s used as a bitfield to toggle the usage of our default values. In this case if the third bit is set (4 is 100 in binary), 2 is assigned to the third parameter, which is hands.

Solution

We can use this constructor with our own bitfield to set default values without ever needing to know what they are:

1class PersonBuilder {
2    private var _defaultsBitField: Int = -1
3 
4    var age: Int? = null
5 
6    var name: String? = null
7 
8    var hands: Int by Delegates.observable(0) { _, _, _ -> _defaultsBitField = _defaultsBitField and -5 }
9 
10    fun build(): Person {
11        check(age != null) { "age must be assigned." }
12        check(name != null) { "name must be assigned." }
13        val constructor = Person::class.java.getConstructor(Int::class.java, String::class.java, Int::class.java, Int::class.java, DefaultConstructorMarker::class.java)
14        return constructor.newInstance(age, name, hands, _defaultsBitField, null)
15    }
16}

Notable things are

• -1 is all ones in binary, meaning we turn on all default values initially. In this simple case we also could’ve just set it to 4
• Delegates.observable is just a nice delegate to wrap a field to do something when it is written
• -5 is the binary inverse of 4, which means in combination with and we can use it to turn off the third bit in our field, leaving all others untouched.
• Since the constructor we want to call is synthetic, we can’t call it directly. Instead, we’re using reflection to find it and then call newInstance. The DefaultConstructorMarker can always be null.

Summary

In this article we’ve talked about the advantages of Kotlin DSLs, namely inline logic and tooling support for syntax and type checking, formatting and autocompletion.
We’ve also seen how to create a Kotlin DSL, and how to generate it using annotation processing.
Lastly we looked at Kotlin internals to overcome one of the major pitfalls encountered when trying to generate a Kotlin DSL in this way.
I’ve encountered more pitfalls on the way to building AutoDSL for Kotlin , so let me know if you found this interesting, and I might write more on the topic.