A Love Letter to Rust Macros

It’s no secret to people who know me that I’m a huge fan of the Rust programming language . I could talk for hours about the brilliance of the ownership system, my irrational longing for natively compiled languages without garbage collection, or the welcoming community that finally moved me to take a more active part in open source projects. But for a start, I just want to highlight one of my favourite features: Macros .

This is not meant to be an in-depth look at what makes Rust’s macro system good, or explain more advanced usages and caveats. Mainly, I’m writing this article for people who haven’t had the chance (or motivation) to try out macros, or even Rust in general. I want to highlight cool things you can do with them, and I want people to look at this and go: “Damn, I should try this.”

Metaprogramming in other Languages

Let’s look at the most well known examples of metaprogramming to get a perspective first. The first few that come to mind here are C macros , C++ templates and Java annotations . They are all very different, and while each of them is extremely powerful in its own right, they also have their downsides.

Anyone who’s ever written C++ or C has probably come into contact with the first two, since they are ubiquitously used for very basic things in those languages - see include guards or the STL . Variadic functions and polymorphism are examples of what’s commonly implemented using templates in C++, letting you write clean APIs for libraries or generic utility functions. Conditional compilation is made possible through compiler directives, which lets you easily compile binaries for different platforms and use their platform specific APIs, just through compiler flags.

And while C macros can sometimes feel like they offer little more than simple find-and-replace on compilation; both they and templates are very close to being turing complete and you can pretty much get them to do whatever you want. (Although they’ll probably end up doing a few things you don’t want them to along the way, too.)

At the same time, they can blow up your compile times and introduce unneeded complexity into your builds. It’s easy to introduce extremely hard-to-find bugs with C macros, since they will (among other things) happily overwrite or mutate local variables that share names used within them. And have you ever seen the compiler errors that can result from incorrectly using templates?

Java annotations, on the other hand, are not just limited to generating code. They can be used both during compile time and runtime, both for code generation and for an incredible amount of reflection/introspection. The Spring framework goes wild with annotations to provide an interface that is so intuitive that many others try to mimic it - one of the most well-liked backend frameworks in Rust ( rocket ) imitates its way of defining endpoints to a T.

Since annontations make it possible to let your code be interpreted in a way completely different from the language’s usual semantics, these big frameworks can start to feel like their own language. You don’t write any structural code anymore, you only write your business logic and use annotations to signal to the framework what goes where. The rest is handled through black magic and doesn’t need to concern you (until it goes wrong and you need to dig through hundreds of conditions for whether or not your classes should load). However, the fact that almost everything is configured through annotations can also make debugging very painful, and if (ab)used to the level that frameworks like Spring do it, will completely decimate your startup times if you don’t know exactly what you’re doing. The processing that would occur at compile time with compiler macros instead happens at runtime, and you in turn incur the resulting cost at runtime.

So, does Rust do all of those things while avoiding all of the aforementioned problems?The answer to that question is obviously not . But in my mind, it makes an effort to avoid certain problems, while playing to the strengths of the rest of the language.

Let’s look into what macros in Rust can do!

Declarative vs. Procedural

The first thing that might be a bit confusing to newcomers is that there are actually 2 very different types of macros. Declarative macros utilitze a sort of pattern matching and let you quickly define things like helper functions and DSLs ( Domain specific languages ) that will expand to code during compilation. Procedural macros are a whole lot more powerful - you can mark functions, structs and several other things with them as attributes or create functions with them. Essentially you declare a function that will receive an abstract syntax tree and can return an abstract syntax tree that will be included in the compilation.

Let’s first list a few basic things:

• Rust macros only do code generation . I.e. code goes in, code comes out.
• Rust macros are hygienic . That means they basically operate in their own isolated namespace: It’s impossible to touch local variables, and there won’t be any naming collisions for variables.
• Since the 2018 edition, macros are imported and used like everything else . They very much feel like a first class citizen of the language, instead of a “dumb” preprocessor directive ignorant of semantics.

Declarative Macros

From my experience, there are a few major areas where these get to shine. The most obvious being variadic functions:

The println or vec macros let you format a string or construct an array with an arbitrary amount of elements. The way they work is pretty easy to understand:

// This macro invocation...
let my_vec = vec![1, 2, 3];

// ...expands to:
{
    let mut my_vec = Vec::new();
    my_vec.push(1);
    my_vec.push(1);
    my_vec.push(1);
    my_vec
}

This means that in contrast to C++ templates, no recursion is necessary to achieve this and no extraneous functions are generated. Additionally, while my_vec is mutable inside the scope where the macro creates it, the “outer” my_vec doesn’t have to be. The macro doesn’t infer anything about the value it returns.

Let’s look at another use case. Sometimes you wind up writing the same piece of code over and over, and even though you know exactly what code it’s gonna be every time, the struct names, the exact calls you have to make and so on are different every time and prevent you from writing something more generic. Declarative macros to the rescue! Here’s an example from the no longer maintained Glium , which was the first really popular OpenGL abstraction in Rust. To provide a typesafe interface for passing vertex and uniform data to the graphics card, it has a macro to automatically implement necessary functions for your structs:

// This macro invocation...
struct MyVertex {
    position: [f32; 3],
    tex_coords: [f32; 2],
}

implement_vertex!(Vertex, position, tex_coords);

// ...expands to: (lots of code cut for brevity)
impl Vertex for MyVertex {
    fn build_bindings() -> VertexFormat {
        Cow::Owned(vec![(
            Cow::Borrowed("position"),
            // ... offset is calculated here
            { offset },
            // ... attribute type is determined here
            { attr_type_of_val(&dummy.position) },
            false,
            // ^ This block is repeated for every variant
            // you pass.

            Cow::Borrowed("tex_coords"),
            { offset },
            { attr_type_of_val(&dummy.tex_coords) },
            false,
        )]),
    }
}

Another thing I want to highlight here is how easy it is to make little DSLs with procedural macros. You can pattern-match almost anything (the limitations can be seen here ). In a linux window manager I’m writing, the configuration of keybinds happens in code (I know I know, but I’m re-compiling it all the time anyways) . This is the macro i’ve written to make it more ergonomic:

let my_keybinds = keybinds![
    (F1 | SwitchWorkspace(1))
    (F2 | SwitchWorkspace(2))
    (F3 | SwitchWorkspace(3))

    ([Super] + C | Spawn("terminal"))
    ([Super] + Q | CloseWindow)
    ([Super, Shift] + Q | ForceCloseWindow)

    (on KeyDown => Super | ShowTaskbar)
    (on KeyUp => Super | HideTaskbar)
];

You can see the macro rules for this example here .

Another really cool example is clap , which lets you define a command line application using a small DSL. Look at this as an example. If you’re looking for a more complex use case, there’s nom : A fully-featured parser combinator that uses macros almost exclusively.

Procedural Macros

The bigger brother of macros basically lets you introduce a completely separate compile step. This comes at a price, though: As of now, these have to reside in a different crate than the one you use them in. However, they give you an incredible amount of power. Let’s take attribute macros as an example. These can be used to annotate structs, fields, functions and other things:

#[proc_macro_attribute]
pub fn hello(attr: TokenStream, item: TokenStream) -> TokenStream {
    // ...
}

You will get all tokens of whatever you marked (including other attribute macros), and you can return any tokens you want. The easiest way to do this is using the quote! macro:

quote! { let foo = String::new("bar"); }

The most prominent place these are being used at are derive s. A derive basically says the following:

If you know how to do thing X for all fields of a struct/enum, you can automatically do thing X for the struct as a whole.

Some of the often-used derives like Copy and Clone rely on compiler intrinsics, but most of them are just plain procedural macros. Deserialize and Serialize from the serde are also great examples for this. If you have a struct where every field already implements Serialize , the whole struct can automatically implement Serialize . Here, you can also add additional attributes that will configure (de)serialization. This enables you to write similar APIs to what you would expect to find in Java or other languages that achieve these features through introspection.

Another example would be relm , which defines its widgets with the #[widget] attribute. And, as I mentioned before, rocket makes extensive use of proc macros to define endpoints for your web service.

One thing I adore particularly about proc macros is the fact that you’re in complete control of any error messages. During compilation you have access to all tokens that have an influence on the macro invocation, which means error messages can be as detailed as you want them to be! And it’s not brain surgery either: All you have to do to show an error message to the user is to panic! during the macro function. Take that, C++ template errors!

The Bottom Line

Basically, all I want to say is:

You should try using Rust macros.

Declarative macro patterns can look a little arcane at first, but I promise, they’re really not that hard to use. It’s a bit similar to the way I always thought regexes are dark magic until I actually decided to dig into them, and then they suddenly were a super useful tool and surprisingly easy to use. I’d probably been looking at the most complicated examples before, not at something you would write in a few minutes to make a specific job easier. Which I regularly do now with both regexes and Rust macros!

Recommended reading

If I’ve made you curious at all, here’s a few places where you can learn more about macros:

• Procedural Macros in Rust 2018 - A very complete and up-to-date guide on procedural macros specifically,
• Creating Macros in Rust - Short and sweet tutorial about declarative macros,
• The Little Book of Rust Macros - Good for looking up specifics,
• An Overview of Macros in Rust - A bit dated at this point, but provides a great perspective at how macros developed over time.

If you do try them or already have, I’d be happy to hear about your experiences! Just drop me a comment below, or a mail if you want. I’m also active on the Amethyst discord server , the community discord for a game engine written in Rust.