The way I personally discovered Rust is sort of akin to acausal trade:

In acausal trade, two agents each benefit by predicting what the other wants and doing it, even though they might have no way of communicating or affecting each other, nor even any direct evidence that the other exists.1

In grad school I spent a lot of time writing C to try to maximize throughput on some algorithm benchmarks for my thesis, and I really came to enjoy the language a lot. There is just so much control over exactly how things execute and how things are laid out in memory. I also spent a lot of time in grad school coding in Haskell, because I love learning new type system features and Haskell basically has an endless supply of those.

This left me in the strange position of simultaneously loving a low level language where you could almost feel the bits directly and which has almost no abstraction capability at all, while at the same time loving an ultra-high level, lazy, garbage collected language with a never-ending tower of abstractions. I constantly found myself wanting some aspect of the other language:

• C is just missing some straightforward, obviously good language features like algebraic data types, closures, type inference, and generics

• Haskell has this garbage collector and runtime and yet you still regularly run across space leaks due to laziness!

Another grad student and I briefly investigated writing a non-garbage collected functional language but it looked to us like there wasn’t enough original research left to do. All of the hard parts had been thought up before, just nobody had combined them into a nice polished language. The closest thing was Cyclone, a safe dialect of C that pioneered the use of linear types to do safe memory management (which eventually inspired the Rust borrow checker). But Cyclone was also very obviously an academic research language, there was no big push behind it, and new code only got added to it when someone wanted to write a new paper.

So, for me, there was this puzzle piece that was obviously missing, and it looked like someone would create it any day now, all I had to do was wait. It turns out, Graydon Hoare had started work on Rust in 2006, but I wouldn’t have recognized it as that puzzle piece at the time. Rust still had garbage collection and an erlang-style green thread runtime back then. It wasn’t until after Patrick Walton’s 2013 blog post that Rust shed its garbage collector and most of its runtime and became the language as we think of it today.

I found Rust shortly after that in 2014 and, I guess the acausal trade was complete? A bunch of really smart and dedicated people had done all the hard work of creating and polishing a very nice language that satisfied all my requirements. In return I, uh… did nothing. Ok well maybe it was less of an acausal trade and more of an acausal gift2.

But, I recognize this is not a super typical route to discovering Rust. Although, since most people writing Rust are doing it in their free time rather than at their jobs (currently), there might not actually be a typical way to get into Rust. I recently spoke with Tim McNamara, author of Rust in Action, and he talked about his own route to the language:

My career path has been really, let's say non-linear. Quite weird.

I have an undergraduate degree in philosophy and German.​ A BA from a university that no one’s heard of at all because it’s from New Zealand. I also got a master's in public policy as well for various reasons. I was destined to be an official basically.

I spent several years as a data scientist doing some really quite neat things with text and search and so forth. I worked for Canonical, the company behind Ubuntu for a couple of years as well. But, I just really enjoyed playing with data, so now I work as a senior data scientist at a small (actually relatively large now) consultancy here in New Zealand.

I got into Rust sort of sideways. I've been a Python developer, I guess, for about a decade. Five-ish years ago now, it just sort of started kind of creeping up on me, this idea that I'm leaving a lot of performance on the table. I'm a relatively proficient Python developer, but, you know, people sort of always whisper that, “C extensions are where it’s at.” Like if you're using Python you're always missing out.

But I was always told that like… C is dangerous. You should never basically write C extensions because that's reserved for wizards and experts and you're not an expert. So you should not even try because you’ll create some sort of segfault, destroy your whole computer, or destroy the world (or expose your users to some pretty needless security concerns). So, I was always terrified of C.

My first playing around with Rust was doing natural language processing. Carving up a lot of text and converting it into big arrays of matrices of numbers, which was the input to a lot of these machine learning models which I can reuse in the Python world.

I got a 100x speed up, like, just by doing this Rust thing.​ And also, memory is like megabytes, instead of gigabytes. There was this massive mental shift that goes on, “Well actually, Rust is really good.”

At the same time, he had a bubbling interest in what was going on at the lowest levels of software

This is actually quite magical. My computer can talk to almost every other computer in the world and we just expect it to work and like… how does it work? It's so commonplace that it almost feels like we don't really consider the complexity that goes on when you click the enter key on a keyboard.

And yet, it actually initiates a massive cascade of events that enables pixels to be rendered on the screen or to do whatever needs to happen when that event occurs.

This led him to start writing Rust in Action, where he was able to explore some of these questions in depth. Sometimes one investigation led him to further topics and he just kept digging

The database chapter started out with the files chapter. I was like, how does a computer represent these data structures actually? How are they represented in a serial string on disk? In memory everything's hierarchical and there’s pointers everywhere and so forth. So we kind of need to flatten everything out.

How do databases do things like ensure that your data is safe, even if the hardware, like the underlying platform, might lose power or become corrupted? You know, if you can't trust anything, how do you ensure that your data is safe?

So the full version, which is kind of still sitting on my computer, it's like this full, really interesting B-Tree structure and like, it's quite elaborate. So we implement pretty cool semantics on top of a key value store.

Writing the book was the impetus behind exploring these topics, but the book format has limitations that he ran up against

I always have a little bit of a lament—kind of like a little bit of regret that I wasn't able to insert the full version of things. I can't explain everything there is to know about systems programming in a book about Rust. Like, I can wedge a lot in there, but… I can't write every book in one book.

Turns out that if you’re writing a chapter, readers are not really interested in, like, 90 page chapters. I don’t think they’re going to have the patience for me teaching them everything about databases and database internals.

You kind of need to whittle it down to only introduce one new Rust feature in every example. I can't introduce four new Rust concepts. I couldn't introduce like closures and Result at the same time. There are definitely people who have come to me and been like “Well this is a stretch.”.

I recognize that I'm not going to be able to write a single piece of tech writing that is going to be perfect to everyone. It’s just something that I've kind of had to acknowledge.​ I started out with this very grandiose idea that I could write the very best tech book, like, in the world.

I felt for myself that I would really produce something extremely high quality. And, I think one of the hardest things has been that I've constantly felt that it’s not quite good enough.

There was this case where I thought a section would be a couple pages, and then like I’m a hundred pages in now. Thankfully for my readers, my editors were very pragmatic and said “Tim, you gotta… this is crazy.”

We also talked a bit about the audience for the book, and the message he wants to convey through it

If you're anything like the person that I was a couple of years ago, like inquisitive about how computers work or really interested in understanding, like, “What is a pointer?” “What does it mean for a reference to be dereferenced?” things that are kind of like this tacit knowledge, which is assumed by people within the Rust community. That’s the kind of stuff you really get.

My intention is to put people at ease when they enter the Rust community. People talk about affine type systems and a whole bunch of jargon. This is kind of the resource that people can look to as giving a base level of understanding when someone says “We’ve got to use a syscall for that.”

If anyone is looking at Rust and thinking “Man, these people are really smart and I’m not smart”. That is the mentality I want to bridge and say “No, actually, you’re welcome. You’ve got a place at the table.” The Rust community is available to you, and Rust is a resource that you can use to keep your programs fast and your users safe.

Given he’s a data scientist, I asked him what he thought of doing data science in Rust

I think that Rust is going to struggle with the data science for sort of two reasons. I think primarily the ergonomics question is the problem. R and Python are always just going to feel a little bit more fluid.

I'm a very strange data scientist in that I have invested very heavily in understanding the software tools that I use and the world of software that I inhabit. But most data scientists do not take much time to think about software engineering as a… as a thing3. There's this new term called data engineer, which is probably where I sit.

Now, within Rust, I think things like TensorFlow should probably be written in Rust. I would love to see Kafka implemented in Rust and, and ZeroMQ and these messaging systems and log aggregation and kind of a lot of this boring stuff that enables data to be stored and transmitted and analyzed.

I think Andy Grove has done a really amazing job. He’s been shipping Rust inside the Apache Arrow project. So Apache Arrow was originally an attempt at creating a kind of peace between the Python and R worlds of data scientists by creating an in-memory format for being able to kind of send data structures between Python and R without costs.

Andy had this idea of reimplementing what we would have a couple of years ago said Hadoop, but more recently we would say implementing Spark with this project built on Arrow called Ballista.

So, these kind of big, heavy pieces that are infrastructure for data science, I think is where Rust is going to find itself very welcome. I think it will be very challenging for Rust to try to be Python. I think that Julia has a much better story as like a convenient, fast language for interactive use. You talked with the creator of the Rust kernel for Jupyter notebooks. And I think that is an enabler, but I don't think it's enough. Being able to interact and introspect variables and so forth is important too.

Apart from the book, where does he use Rust?

It was interesting that you interviewed Alexis because creative coding is one of the things that I also play around with. There’s a couple of YouTube clips where I go the tutorial for Nannou, which is the framework you were discussing before. So that’s kind of like side projects.

At work, we’re slowly going through the process of optimizing. For like utilities or stuff that is being used repeatedly, I will gently nudge work, “If we want something that's reliable and easy to deploy and can use like a micro instance rather than like some big VM on the cloud”, I’ll nudge them towards Rust. So, it's been a process of finding things that are very discrete and slowly kind of finding a spot for Rust. Either as a replacement or as like a supplement to things that are typically implemented first in Python.

Typically, it’s in CLI utilities or ETL which is the data transformation pipeline.

What is he excited about in either the language or Rust ecosystem?

I honestly feel that WebAssembly plus WASI is going to be the standard way that we ship software in some future state. That’s where I feel most excited for Rust.

One of the things that I'm delighted to see is the way in which the Rust language developers have embraced the async discussion.

I would love for that mental tax of async to kind of evaporate. I don't know if that's actually going to occur. I'm sure that it won't be perfect, but all the signs are there. The people taking the right approach like Niko and Ryan Livek and others are very focused on creating a structure that would enable Rust to become the most pleasant async programming environment that the world has.

On his development setup and tooling:

I have a surprisingly boring setup. I’ve got a Dell XPS 15, and VSCode with rust-analyzer.

Like, I kind of wish I was a wizard, but really I found the Rust tooling, just the default stuff, to be really good. I did pay for CLion, but one reason I don’t use it is that I do a lot of stuff on Twitch and YouTube. And I want to make sure that my content is really accessible. And if I use proprietary paid technology, which a couple of my earlier videos did, it kind of creates a barrier that I don't think should exist.

He also had some advice for newer Rust developers

It's okay not to use idiomatic Rust. If you're a Java developer or if you've done C# or whatever other programming paradigm you're from, you are going to naturally write Rust that looks a little bit like Java or C#, you know, that's just what happens.

You are going to write getters that to me look hideous, like you're going add, getThing. But that's my problem as someone who's an experienced Rust user. I am going to squirm a little bit when I see your code and that's fine actually, because you do not need to learn everything in one go. Don’t aim for perfection if you’re starting out.

Finally, to appease the Rust Evangelism Strikeforce, I have to include some effusive praise for the language:

Normally I expect a technology to have this two year honeymoon phase, and then things start to really irritate you. I just am waiting for this disillusionment to occur, but with Rust, it just feels like I'm just continually excited.

It can enable you to build cool software and safe software and that's all important, but actually being able to kind of grow people as humans, I think is one of the things that's a really strong distinguishing characteristic of the language and its ecosystem more generally.​

And because there must be balance in the universe:

I get in trouble sometimes because I'm not a very good language evangelist. Rust is really pedantic and it can feel very bureaucratic and stuffy. Sometimes you just want Rust to relax. But it’s never going to relax.

Where to find Tim

You can find his book Rust In Action on Manning’s site and you can find him on Twitter @timClicks and see his streams on YouTube at timClicks. He also has a blog about technical topics at tim.mcnamara.nz

Upcoming posts

Tim was a very fun person to talk to, and he gave me lots of (spicy?) opinions on things like Haskell, Emacs and data science that I am going to break into a separate post for subscribers.

If you’re interested in that kind of thing, feel free to subscribe. It does not have a lot to do with Rust, it was just a long conversation with lots of fun tangents and I wanted to collect the tidbits somewhere.

I also have a post coming up about doing constructive math and type theory in Rust, which Yay!, but also, sorry Tim.

Share Always Bet On Rust