Combining Axum, Hyper, Tonic, and Tower for hybrid web/gRPC apps: Part 3

Share this

This is the third of four posts in a series on combining web and gRPC services into a single service using Tower, Hyper, Axum, and Tonic. The full four parts are:

Tonic and gRPC

Tonic is a gRPC client and server library. gRPC is a protocol that sits on top of HTTP/2, and therefore Tonic is built on top of Hyper (and Tower). I already mentioned at the beginning of this series that my ultimate goal is to be able to serve hybrid web/gRPC services over a single port. But for now, let's get comfortable with a standard Tonic client/server application. We're going to create an echo server, which provides an endpoint that will repeat back whatever message you send it.

The full code for this is available on GitHub. The repository is structured as a single package with three different crates:

A library crate providing the protobuf definitions and Tonic-generated server and client items
A binary crate providing a simple client tool
A binary crate providing the server executable

The first file we'll look at is the protobuf definition of our service, located in proto/echo.proto:

syntax = "proto3";

package echo;

service Echo {
  rpc Echo (EchoRequest) returns (EchoReply) {}
}

message EchoRequest {
  string message = 1;
}

message EchoReply {
  string message = 1;
}

Even if you're not familiar with protobuf, hopefully the example above is fairly self-explanatory. We need a build.rs file to use tonic_build to compile this file:

fn main() {
    tonic_build::configure()
        .compile(&["proto/echo.proto"], &["proto"])
        .unwrap();
}

And finally, we have our mammoth src/lib.rs providing all the items we'll need for implementing our client and server:

tonic::include_proto!("echo");

There's nothing terribly interesting about the client. It's a typical clap-based CLI tool that uses Tokio and Tonic. You can read the source on GitHub.

Let's move onto the important part: the server.

The server

The Tonic code we put into our library crate generates an Echo trait. We need to implement that trait on some type to make our gRPC service. This isn't directly related to our topic today. It's also fairly straightforward Rust code. I've so far found the experience of writing client/server apps with Tonic to be a real pleasure, specifically because of how easy these kinds of implementations are:

use tonic_example::echo_server::{Echo, EchoServer};
use tonic_example::{EchoReply, EchoRequest};

pub struct MyEcho;

#[async_trait]
impl Echo for MyEcho {
    async fn echo(
        &self,
        request: tonic::Request<EchoRequest>,
    ) -> Result<tonic::Response<EchoReply>, tonic::Status> {
        Ok(tonic::Response::new(EchoReply {
            message: format!("Echoing back: {}", request.get_ref().message),
        }))
    }
}

If you look in the source on GitHub, there are two different implementations of main, one of them commented out. That one's the more straightforward approach, so let's start with that:

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let addr = ([0, 0, 0, 0], 3000).into();

    tonic::transport::Server::builder()
        .add_service(EchoServer::new(MyEcho))
        .serve(addr)
        .await?;

    Ok(())
}

This uses Tonic's Server::builder to create a new Server value. It then calls add_service, which looks like this:

impl<L> Server<L> {
    pub fn add_service<S>(&mut self, svc: S) -> Router<S, Unimplemented, L>
    where
        S: Service<Request<Body>, Response = Response<BoxBody>>
            + NamedService
            + Clone
            + Send
            + 'static,
        S::Future: Send + 'static,
        S::Error: Into<crate::Error> + Send,
        L: Clone
}

We've got another Router. This works like in Axum, but it's for routing gRPC calls to the appropriate named service. Let's talk through the type parameters and traits here:

L represents the layer, or the middlewares added to this server. It will default to Identity, to represent the no middleware case.
S is the new service we're trying to add, which in our case is an EchoServer.
Our service needs to accept the ever-familiar Request<Body> type, and respond with a Response<BoxBody>. (We'll discuss BoxBody on its own below.) It also needs to be NamedService (for routing).
As usual, there are a bunch of Clone, Send, and 'static bounds too, and requirements on the error representation.

As complicated as all of that appears, the nice thing is that we don't really need to deal with those details in a simple Tonic application. Instead, we simply call the serve method and everything works like magic.

But we're trying to go off the beaten path and get a better understanding of how this interacts with Hyper. So let's go deeper!

`into_service`

In addition to the serve method, Tonic's Router type also provides an into_service method. I'm not going to go into all of its glory here, since it doesn't add much to the discussion but adds a lot to the reading you'll have to do. Instead, suffice it to say that

into_service returns a RouterService<S> value
S must implement Service<Request<Body>, Response = Response<ResBody>>
ResBody is a type that Hyper can use for response bodies

OK, cool? Now we can write our slightly more long-winded main function. First we create our RouterService value:

let grpc_service = tonic::transport::Server::builder()
    .add_service(EchoServer::new(MyEcho))
    .into_service();

But now we have a bit of a problem. Hyper expects a "make service" or an "app factory", and instead we just have a request handling service. So we need to go back to Hyper and use make_service_fn:

let make_grpc_service = make_service_fn(move |_conn| {
    let grpc_service = grpc_service.clone();
    async { Ok::<_, Infallible>(grpc_service) }
});

Notice that we need to clone a new copy of the grpc_service, and we need to play all the games with splitting up the closure and the async block, plus Infallible, that we saw before. But now, with that in place, we can launch our gRPC service:

let server = hyper::Server::bind(&addr).serve(make_grpc_service);

if let Err(e) = server.await {
    eprintln!("server error: {}", e);
}

If you want to play with this, you can clone the tonic-example repo and then:

Run cargo run --bin server in one terminal
Run cargo run --bin client "Hello world!" in another

However, trying to open up http://localhost:3000 in your browser isn't going to work out too well. This server will only handle gRPC connections, not standard web browser requests, RESTful APIs, etc. We've got one final step now: writing something that can handle both Axum and Tonic services and route to them appropriately.

`BoxBody`

Let's look into that BoxBody type in a little more detail. We're using the tonic::body::BoxBody struct, which is defined as:

pub type BoxBody = http_body::combinators::BoxBody<bytes::Bytes, crate::Status>;

http_body itself provides its own BoxBody, which is parameterized over the data and error. Tonic uses the Status type for errors, and represents the different status codes a gRPC service can return. For those not familiar with Bytes, here's a quick excerpt from the docs

Bytes is an efficient container for storing and operating on contiguous slices of memory. It is intended for use primarily in networking code, but could have applications elsewhere as well.

Bytes values facilitate zero-copy network programming by allowing multiple Bytes objects to point to the same underlying memory. This is managed by using a reference count to track when the memory is no longer needed and can be freed.

When you see Bytes, you can semantically think of it as a byte slice or byte vector. The underlying BoxBody from the http_body crate represents some kind of implementation of the http_body::Body trait. The Body trait represents a streaming HTTP body, and contains:

Associated types for Data and Error, corresponding to the type parameters to BoxBody
poll_data for asynchronously reading more data from the body
Helper map_data and map_err methods for manipulating the Data and Error associated types
A boxed method for some type erasure, allowing us to get back a BoxBody
A few other helper methods around size hints and HTTP/2 trailing data

The important thing to note for our purposes is that "type erasure" here isn't really complete type erasure. When we use boxed to get a trait object representing the body, we still have type parameters to represent the Data and Error. Therefore, if we end up with two different representations of Data or Error, they won't be compatible with each other. And let me ask you: do you think Axum will use the same Status error type to represent errors that Tonic does? (Hint: it doesn't.) So when we get to it next time, we'll have some footwork to do around unifying error types.

Almost there!

We'll tie up next week with the final post in this series, tying together all the different things we've seen so far.

Read part 4 now

If you're looking for more Rust content, check out:

Subscribe to our blog via email
Email subscriptions come from our Atom feed and are handled by Blogtrottr. You will only receive notifications of blog posts, and can unsubscribe any time.

Do you like this blog post and need help with DevOps, Rust or functional programming? Contact us.

Share this

Tonic and gRPC

The server

`into_service`

`BoxBody`

Almost there!

Recommend

HShhss | Hoj Senna’s humble heap of stray scribbles

新能源车的下一场技术革命！

Protocol Engineer, Forest (Rust)

Designing state machines in Rust

Turn Location Services and GPS on or off on your iPhone, iPad, or iPod touch

badboy (Jan-Erik Rediger) · GitHub

Rust for Java Developers

Senior Toolkit Engineer - Database

IntelliJ Rust Changelog #156

GitHub - gfx-rs/meetup: Rust Graphics meetups

About Joyk