In the world of distributed systems and microservices, the way services talk to each other has come a long way. REST, which is based on the HTTP protocol and follows the CRUD (Create, Read, Update, Delete) operations model, has been the go-to standard for web services for over a decade. It's known for its simplicity and the ease with which web browsers and other HTTP clients can interact with it. However, there's another player in the field: gRPC. Created by Google, gRPC can be an alternative to REST, especially for applications that need high performance, efficiency, and two-way communication.
Instead of relying solely on HTTP verbs, gRPC allows developers to define APIs using Protobuf, a binary serialisation format known for being compact and fast.
In these two upcoming blog posts, we'll delve deeply into gRPC and how it utilizes Protobuf for data transmission.
Exploring the Basics and History of gRPC
gRPC (Google Remote Procedure Call) is a general remote procedure call (RPC) framework created by Google. It's high-performance and open-source, enabling client and server applications to communicate as if they were on the same system, across a variety of platforms and languages.
The definition of gRPC includes several key concepts. First, it's an RPC framework, providing a way to execute method calls on a server as if they were being run locally on the client. Second, it's general: it's not limited to a specific type of applications or data. Third, it's high-performance, designed for applications requiring high efficiency and performance.
The story of gRPC began in 2015 when Google introduced it as part of the company's effort to provide an efficient, interoperable, and flexible method for communication between microservices. It was developed as a successor to Stubby, Google's internal RPC system, aiming to offer the same performance and flexibility, but with the added ability to function across a multitude of environments and programming languages.
Since its launch, gRPC has gained widespread acceptance in the software development community and is used by a variety of organisations and projects, including big names like Netflix, Cisco, Juniper, and Square. It has proven especially useful in microservice architectures, where its ability to enable efficient, bidirectional streaming communication over HTTP/2 makes it an ideal choice for service-to-service communication.
As an Incubation Project of the Cloud Native Computing Foundation (CNCF), gRPC has become a crucial part of the modern software stack, playing a pivotal role in a range of cloud-native applications to real-time communication systems.
Understanding and History of Protobuf
Protocol Buffers, often referred to simply as Protobuf, is a binary serialisation format created by Google. It's used for quickly and efficiently serialising and deserialising structured data. Protobuf was made to generate smaller and faster binary messages compared to similar technologies like XML or JSON. It provides a straightforward syntax for defining data structures and allows the generation of source code in many programming languages to use and manipulate these data structures.
- Origins: Protobuf was initially developed internally at Google to address the issues arising from using older serialisation formats. Google was looking for a more efficient, compact, and faster format to share data among its many internal systems and services.
- Release: In 2008, Google decided to release Protobuf as an open-source project, allowing developers outside Google to benefit from this format. Since then, it has become one of the most popular serialisation formats, especially in systems requiring high performance and efficiency.
- Versions: Currently, there are three main versions of Protobuf: Proto1, Proto2, and Proto3. Each version introduced enhancements and new features. Proto3, the latest version, was released in 2016, introducing simpler syntax, better support for more programming languages, and other improvements.
- Adoption and Usage: Since its release, Protobuf has been used in a variety of projects and systems both within and outside of Google. It's particularly popular in high-performance applications and in systems requiring fast and efficient communication between services, like gRPC, which was also developed by Google.
In summary, Protobuf is a powerful and efficient serialisation format that has proven to be a valuable tool for developers since its introduction. It offers a combination of speed, efficiency, and flexibility that sets it apart from many other serialisation formats.
Creating a gRPC Service Definition with Protobuf
Creating a gRPC service definition using Protobuf involves several steps that cover defining the data structures and service interfaces. Here's a detailed breakdown of this process:
- Create a Protobuf File: The first step is to create a new Protobuf file (.proto). This file serves as the blueprint for your gRPC service, defining the data structures and service interfaces your service will use.
- Specify the Syntax: At the start of your Protobuf file, you should specify the Protobuf syntax being used. For most applications, you should use the latest version, syntax = "proto3";.
- Define Message Types: Next, define the message types that will be used in your service interfaces. A message type in Protobuf is similar to a structure or object in other programming languages, defining a group of fields that belong together. Each field has a name, a type, and a unique number.
- Define Service Interfaces: After defining your message types, you can define your service interfaces. A service interface in Protobuf outlines a set of methods your service offers. Each method has a name, a request type, and a response type.
- Use the Protobuf Compiler: Once you've created your Protobuf file, you can use the Protobuf compiler (protoc) to generate source code from this file in your desired programming language. This generated code includes both the data structures for your message types and the necessary logic for invoking your gRPC methods.
Here's a simple example of a gRPC service definition with Protobuf:
1syntax = "proto3";
2
3// Definieren Sie eine Nachricht für die Anforderung
4message HelloRequest {
5 string name = 1;
6}
7
8// Definieren Sie eine Nachricht für die Antwort
9message HelloResponse {
10 string greeting = 1;
11}
12
13// Definieren Sie den Service
14service HelloService {
15 // Definieren Sie eine Methode, die die HelloRequest-Nachricht akzeptiert und die HelloResponse-Nachricht zurückgibt
16 rpc SayHello (HelloRequest) returns (HelloResponse);
17}
In this example, the HelloService defines a SayHello RPC, which accepts a HelloRequest message and returns a HelloResponse message. Each message has one field: 'name' for the HelloRequest and 'greeting' for the HelloResponse.
Setting Up the Development Environment
To dive right into practice, let's start by setting up a "development environment." In our case, this consists of the Buf CLI and the Buf Schema Registry (BSR). It would also be wise to consider APICurio as a possible schema registry.
According to popular online tutorials, you only need protoc and a bit of build magic from the respective framework or programming language to see initial results. It remains to be seen whether it’s actually that straightforward.
Buf is a tool aimed at optimising the handling of Protobuf files. It focuses particularly on the workflow around Protobuf files, including linting, identifying breaking changes, code generation, and dependency management. The goal is to enhance the user experience for developers dealing with Protobuf and gRPC.
Since I am setting up the environment on a Mac, I will use Homebrew for the installation. Additional installation details can be found in the Buf CLI documentation:
brew install bufbuild/buf/buf
It's important to note that protoc is still running in the background, getting a wrapper through the CLI. Since we will be using Visual Studio Code as our preferred IDE, we’ll also install the "Buf for Visual Studio Code" extension there.
To initialise our project, we’ll execute the following command in the project directory:
buf mod init
With this, we initialise a module in the context of Buf and at the same time create a configuration file buf.yaml in the same directory.
1version: v1 2breaking: 3 use: 4 - FILE 5lint: 6 use: 7 - DEFAULT
In this file, we have default settings for breaking changes and linting. However, we won’t make any changes here for our start.
Next week, in the second part, we'll take a look at how gRPC and Protobuf work in the familiar retail domain.
Conclusion
In conclusion, gRPC and Protobuf offer a powerful combination for creating stable and efficient service definitions in distributed systems and microservices. gRPC provides high performance and two-way communication, while Protobuf offers compact and fast data transmission. These technologies have gained widespread acceptance and are used by various organisations and projects. With the ability to define message types and service interfaces, developers can easily create gRPC service definitions using Protobuf. Additionally, tools like Buf CLI and Buf Schema Registry enhance the workflow around Protobuf files and make development more seamless. Overall, gRPC and Protobuf are valuable tools for building modern and efficient communication systems.
References
Was this post helpful?
More articles
fromDaniel Kocot
13.11.2024 Your job at codecentric?
Jobs
Agile Developer und Consultant (w/d/m)
Alle Standorte
More articles in this subject area
Discover exciting further topics and let the codecentric world inspire you.
Gemeinsam bessere Projekte umsetzen.
Wir helfen deinem Unternehmen.
Du stehst vor einer großen IT-Herausforderung? Wir sorgen für eine maßgeschneiderte Unterstützung. Informiere dich jetzt.
Hilf uns, noch besser zu werden.
Wir sind immer auf der Suche nach neuen Talenten. Auch für dich ist die passende Stelle dabei.
Blog author
Daniel Kocot
Senior Solution Architect / Head of API Consulting
Do you still have questions? Just send me a message.
Do you still have questions? Just send me a message.