Golang, Gin & MongoDB – Building microservices easily

Golang, a.k.a. Go, has been around in the industry for quite some time now, but people are still reluctant to just go ahead and use it. To help you get started, follow me on this journey and create your first microservice using Golang, Gin and Docker.

Set up your environment

There are a few packages needed on your development system in order to start developing – the good news is that it’s not much and easy to install:

1. Install Golang
2. Install Docker

In the upcoming sections I’m assuming that you are using Golang 1.11 or newer. If you, however, use an older version, be aware that some things might not work as described here.

If you want to get the whole application right from the start, you can go ahead and clone the Git repository right here from GitHub.

Let the coding begin

Basics

Assuming you successfully installed and tested Golang, let’s see how to get going and set up a webservice using Gin .
First of all, we need our main package with the main function in our main.go file (that’s enough mains for now) 🙂

package main

import "log"

func main() {
    log.Println("Hello from Go")
}

By running go build in your terminal, an executable suitable for your operating system will be created. By running said executable, we should already see the output “Hello from Go”.

Before adding external dependencies, it is worth considering adding support for versioned modules to our application by running go mod init task-management. Feel free to change the name of the application from task-management to anything you want!
We will see that a new file go.mod appears in your directory. Afterwards, we will find all dependencies we’re adding to our application in there.

Now let’s put some Gin in there and get our webservice started!

We can include Gin just like any other dependency in Go (assuming you run version 1.11 or newer) by running go get -u github.com/gin-gonic/gin.
This will download the Gin dependency and add it to the go.mod file. Now we recognize yet another new file in our directory – go.sum. That file contains checksums for direct and indirect dependencies of our application and actually some more things not that relevant for our cause.

Setting up Gin and your first endpoint

Now that we have Gin included, let’s set up and start the server.

package main

import "github.com/gin-gonic/gin"

func main() {
    r := gin.Default()
    r.Run()
}

Yes, that’s all we need to set up Gin. Using go build to build the application and then running the created executable will show us where we can reach Gin (should be curl localhost:8080 on default). That will give us an HTTP status 404 and some log output indicating that we’re set up and are successfully serving HTTP errors!

Serving some data

Now, while serving HTTP error codes can be fun, accompany me on our journey and start serving some actual data. Let’s go ahead and think about what data we want to serve. As mentioned above, I would just serve tasks as a start.

To separate the model from the application logic, we can put the struct into a separate file task.go

package main

// Task - Model of a basic task
type Task struct {
    Title string
    Body  string
}

Now let’s serve a task (as JSON) through a few modifications in our main.go file:

package main

import "github.com/gin-gonic/gin"

func handleGetTasks(c *gin.Context) {
    var tasks []Task
    var task Task
    task.Title = "Bake some cake"
    task.Body = `- Make a dough 
    - Eat everything before baking 
    - Pretend you never wanted to bake something in the first place`

    tasks = append(tasks, task)
    c.JSON(http.StatusOK, gin.H{"tasks": tasks})
}

func main() {
    r := gin.Default()
    r.GET("/tasks/", handleGetTasks)
    r.Run()
}

If we go ahead and go build and then run the application, we will now be able to see the dummy task as JSON by hitting localhost:8080/tasks/

We can now see that a pointer to the gin.Context object is passed to each handler function we add to Gin (e.g. handleGetTasks). That context can then be used to interact with parameters or objects passed in the request but also to specify what you want to return in terms of status code, headers and content. Furthermore, be aware that the gin type is always available to you if you import it and it offers quite a few handy functions to interact with request and response types. Therefore make sure to check if Gin offers functions you need before writing everything yourself!

Setting up a MongoDB for your application

There are multiple ways we could choose to just store some data, but to make things a bit more interesting, we could store our data in a MongoDB .

As we already have Docker installed on our development environment, we can use that to run our own MongoDB instance locally.

Executing docker run --name mongodb -e MONGO_INITDB_ROOT_USERNAME=myuser -e MONGO_INITDB_ROOT_PASSWORD=mypassword -e MONGO_INITDB_DATABASE=tasks -p 27017:27017 -d mongo:latest should be fine for development purposes (of course not for productive use!)

Storing data in our MongoDB

Connecting to MongoDB

As we now have our MongoDB, we need to establish a connection from our Golang application. Luckily, there also is a dependency we can use to make things easier for us: go get -u go.mongodb.org/mongo-driver.

...package declaration and imports...

const (
    // Timeout operations after N seconds
    connectTimeout           = 5
    connectionStringTemplate = "mongodb://%s:%s@%s"
)

// GetConnection - Retrieves a client to the DocumentDB
func getConnection() (*mongo.Client, context.Context, context.CancelFunc) {
    username := os.Getenv("MONGODB_USERNAME")
    password := os.Getenv("MONGODB_PASSWORD")
    clusterEndpoint := os.Getenv("MONGODB_ENDPOINT")

    connectionURI := fmt.Sprintf(connectionStringTemplate, username, password, clusterEndpoint)

    client, err := mongo.NewClient(options.Client().ApplyURI(connectionURI))
    if err != nil {
        log.Printf("Failed to create client: %v", err)
    }

    ctx, cancel := context.WithTimeout(context.Background(), connectTimeout*time.Second)

    err = client.Connect(ctx)
    if err != nil {
        log.Printf("Failed to connect to cluster: %v", err)
    }

    // Force a connection to verify our connection string
    err = client.Ping(ctx, nil)
    if err != nil {
        log.Printf("Failed to ping cluster: %v", err)
    }

    fmt.Println("Connected to MongoDB!")
    return client, ctx, cancel
}

While this looks like a lot of code, please note that now we have everything we need to start interacting with our MongoDB including timeouts, configurability and connectivity checks. Nevertheless, it might make sense to put that into a separate file e.g. db.go. Also, we should create a file containing our environment variables (e.g. a .env file) with entries like: export MONGODB_USERNAME=myuser.

In the getConnection() function, be aware that we are using os to interact with our environment. In contrast to many other languages, Golang makes it really easy to get in contact with the environment and read or write environment variables in pretty much just one line of code.

Adding a Datamodel

Now let’s tackle the next obstacle and start storing some data in our MongoDB. Therefore we need some changes.
To make each entry easily identifiable, we should add an ID field to our task struct:

...
type Task struct {
    ID    primitive.ObjectID 
    Title string
...

Interacting with our MongoDB

In our db.go we should now add some code to actually create a new task in our MongoDB collection.

//Create creating a task in a mongo or document db
func Create(task *Task) (primitive.ObjectID, error) {
    client, ctx, cancel := getConnection()
    defer cancel()
    defer client.Disconnect(ctx)
    task.ID = primitive.NewObjectID()

    result, err := client.Database("tasks").Collection("tasks").InsertOne(ctx, task)
    if err != nil {
        log.Printf("Could not create Task: %v", err)
        return primitive.NilObjectID, err
    }
    oid := result.InsertedID.(primitive.ObjectID)
    return oid, nil
}

As you can see, we pass a pointer to the task object into the create function, generate a new ObjectID and then use our MongoDB client to store the task in our collection. As a result, we pass the ObjectID, or an error in case something unintended happens while saving.

In Golang we can defer the execution of code to when the function is exited. This is a really nice way to free resources and acquired connections like our MongoDB connection. We can make sure to close the connection in any case and schedule it to close automatically once the functions exits.

Finally, we should hook a handler in our main.go and call the Create function we’ve just written:

...
func handleCreateTask(c *gin.Context) {
    var task Task
    if err := c.ShouldBindJSON(&task); err != nil {
        log.Print(err)
        c.JSON(http.StatusBadRequest, gin.H{"msg": err})
        return
    }
    id, err := Create(&task)
    if err != nil {
        c.JSON(http.StatusInternalServerError, gin.H{"msg": err})
        return
    }
    c.JSON(http.StatusOK, gin.H{"id": id})
}

func main() {
    r := gin.Default()
    r.GET("/tasks/", handleGetTasks)
    r.PUT("/tasks/", handleCreateTask)
    r.Run()
}

As you can see here, Gin helps us marshal and bind the JSON string into our task to make our lives a bit easier. It also assists us in handling conversion errors on our own with ShouldBind... or Gin handles everything for us with Bind... if we decide to.

Let’s build everything and try it!

Now go build, specify the environment variables (MONGODB_USERNAME, MONGODB_PASSWORD and MONGODB_ENDPOINT), run the application, and store a new task in our MongoDB:
curl --location --request PUT 'http://localhost:8080/tasks/' \ --header 'Content-Type: application/json' \ --data-raw '{ "Title": "New Task", "Body": "Well this has been some fun already!" }'

Now we can change the GET endpoint to return the data that is actually stored in MongoDB, add DELETE and POST endpoints and so on and so forth.
You can find implementations of those endpoint in the GitHub project , but I think now it’s time to move on to our next and last chapter.

Containerize it!

What would a proper microservice be without a comfy and tiny Docker container? As Golang has huge support for container technologies, this is quite easy. You can actually start from scratch! 😀

FROM scratch
ENV MONGODB_USERNAME=MONGODB_USERNAME MONGODB_PASSWORD=MONGODB_PASSWORD MONGODB_ENDPOINT=MONGODB_ENDPOINT
# My application runnable is called gin here
ADD gin / 
CMD ["/gin"]

Now if you have built your container and try to run it, you might see a strange error standard_init_linux.go:211: exec user process caused "exec format error". This happens if you’ve built the Golang application on a different operating system than you try to run it on. In case of Windows, Docker might even have trouble locating a file named gin at all.

We have multiple ways of dealing with this issue:

1. Build the Go application for the OS we run the application on
2. Do a multistage Docker build

If you want to follow route one on our journey, you simply need to execute GOOS=linux go build when building the application and next time you build our container, everything should be fine. This is a really handy feature as you can compile Golang applications for different operating systems by just specifying the GOOS environment variable.

If you want a multistage Docker build, you need a few more lines of code but your overall build process will be much simpler and more automated:

FROM golang:latest AS builder
ADD . /app
WORKDIR /app
RUN go mod download
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -o /main .

FROM scratch
ENV MONGODB_USERNAME=MONGODB_USERNAME MONGODB_PASSWORD=MONGODB_PASSWORD MONGODB_ENDPOINT=MONGODB_ENDPOINT
COPY --from=builder /main ./
ENTRYPOINT ["./main"]
EXPOSE 8080

Now you can build and run your container and are good to go!

Famous last words

Now, having built this application, you might have realized that Golang is actually quite nice. Yes, some things are rather expressive and might seem a bit repetitive (error handling, I’m looking at you), but you can build applications really fast and they actually are quite readable if you are used to C/C++/C# or other related languages. Overall, it’s a fun language to use. Make sure to go ahead and gather your own experiences with it! 🙂

If you want to check out the whole project, fill any gaps or just start experimenting based on this application – Clone me on GitHub! . Pull requests are very welcome! 🙂

Share your thoughts with me using the comment section below and check out the blog – there might be more you’re interested in, e.g. Serverless Golang on GCP !