Design Patterns, Ownership Models, and Building Resilient Systems in Rust with Evan Williams
Why experienced developers have the hardest time, and what that tells you about the language
Evan Williams has been writing software for more than 40 years, across every layer of the stack and more programming languages than most engineers will encounter in a career. His book, Design Patterns and Best Practices in Rust, published by Packt, is not a pattern catalogue. It is an argument for a different way of thinking about code entirely, aimed squarely at experienced developers who arrive in Rust carrying instincts that the language will refuse to accommodate.
He recently sat down with Deep Engineering to talk about what that shift requires, which traditional patterns break in Rust, the typestate pattern he finds almost impossible to stop talking about, and why he discovered it is harder to write bad Rust than good Rust.
Watch the full conversation below.
A note on format: the transcript below has been lightly edited for clarity and readability.
Q. You have been in software for a long time. Tell us about your background and how your journey started.
Evan Williams: I have been in the software business for horrifyingly more than 40 years. Surprisingly, given that, my journey started when I was 14 years old, in the 1970s. My father was a very skilled electrical engineer, and we built a computer in the basement together, which had a 6502 processor and 1K of RAM. He hoped that I would become an electrical engineer because of my interest in the electronics. I became a programmer because I thought the programming was the most fun part. I have since then grown up with the industry, not from the very beginning of it, but certainly from fairly early on, and in particular grown up with professional software development from the beginning. I’ve touched every part of the stack, many, many programming languages, many systems, and I am intensely interested by this. If they didn’t pay me to do it, I’d do it anyway.
Q. You have worked with languages like C and Python. What initially drew you toward Rust?
Evan Williams: I’m interested in programming languages in general, and I had heard about Rust almost at the very start. I looked at it and I said, this looks kind of mildly interesting, I’ll just remember that. And then I forgot about it. But years later, Rust had just barely reached 1.0, and I was working on a hardware project that had a few interesting characteristics. It needed to be in a location that we couldn’t access easily. It wasn’t going to be on the internet. It needed to be rock solid because people depended on it. And it was remarkably complicated software. The team talked about it and we made the decision to go with Rust, which none of us knew at the time. We all learned it together. And that was where I caught the Rust bug and have not lost it since.
Q. What motivated you to write a book on design patterns in Rust at this stage of your career?
Evan Williams: There are sort of two questions tied up in there. One is, why would I be writing a book at all at this stage of my career? And the answer to that is, I have been remarkably fortunate to have people help me throughout my career and help me grow. And it makes me incredibly happy to be able to pay that forward and help the people who are learning at this point to grow themselves. This book is a vehicle for me to help people. Why Rust design patterns? Because I think the interesting thing about dealing with design patterns in Rust is very often they’re not what you need, and very often the traditional ones are not exactly what you need, and very often they can cause you problems that you didn’t have to have. So I wanted to save people that frustration and help them move along the path in a way that is a lot less painful, by not making the same mistakes that I made.
Q. Rust has been around for some time but is now seeing increased adoption. Why do you think this is the moment?
Evan Williams: For a long time there were a lot of people like me who were excited about Rust, and there were people who were early adopters who were pushing for it. But right now what we’re seeing is a world where the language has matured, the ecosystem has matured, it’s reached a kind of critical mass. Now when someone is thinking about doing a Rust project, they’re not pioneers who are wandering out into lands unknown. There’s a huge community, a huge set of packages and software, great learning resources, and there are people who have had success that you can see who have done amazing things with Rust. People feel more confident and feel safer venturing into using Rust for things, whereas before it was a little bit more of a risk in their mind.
Q. What gaps does Rust fill compared to more established languages like C, Java, or Python?
Evan Williams: I could just go over sort of the normal thing and say high performance and memory safety, and all those things are true. But Rust is also a powerful modern language that has a good feature set. And I feel like there’s a design gap of sorts. One of the things that’s great about Rust, and I’ll probably keep returning to this topic, is if you need something to be correct, if you need to be able to count on the code that you write, Rust helps you. Rust is your partner in doing that. You can still write code with bugs in it, but Rust makes it harder to do that and easier to write code that’s going to be solid. I think that’s a huge gap that it fills.
Q. What kinds of systems or use cases benefit most from adopting Rust today?
Evan Williams: Systems that are mission critical in some way or other are really key Rust use cases. Yes, Rust has high efficiency and the memory safety is very important. But all of these features combine into a whole that make Rust a really powerful language for doing things that have to work. Things where failure is monetarily or in human cost even a terrible problem. That’s one of the powers of the language and I think it’s great for those situations.
Q. One of the recurring themes in your book is that developers need to think differently in Rust. What does that mindset shift actually involve?
Evan Williams: There are a few things associated with that. One is that Rust is not an object-oriented language. It kind of looks like an object-oriented language in some ways, but it’s not. And if you carry with you an object-oriented language mindset, then you’re going to have nothing but trouble. It’s also a language that requires you to think carefully about the design of your code before you start writing it. It’s very easy to get yourself into trouble in Rust if you don’t plan what you’re doing. You have to start thinking about data and how it’s handled in a different way. You have to think ahead of time about where the data is flowing through your program, what it is, where it is going, how long is it going to live, who is responsible for it. These are things that you don’t really have to think about when you’re writing a Python program or a Java program. Those are good design principles in all of those languages, but Rust requires it of you.
Q. Many developers struggle with the borrow checker early on. How should they reframe it as a design tool rather than a limitation?
Evan Williams: The thing about the borrow checker is it’s there to help you, and it is very easy to get into a mode where you’re fighting with it and you feel like it’s your enemy. But in fact, what it is doing is encouraging you to build your code in a solid manner. It’s encouraging you to think about not just what data you have, but how it’s going to be used. In something like Java or Python, with some amount of plumbing you can get anything from anywhere. You don’t really have to design your program in a highly organized way where you’ve thought about the data flows. But you’re much better off if you do. I think the principles that the borrow checker forces you to adhere to in Rust are the exact principles that you should be using in every programming language. But you don’t have to. So it’s very easy to not think about those things. The borrow checker is your friend because it prevents you from making a messy design. It prevents you from making a broken design. It prevents you from writing whole classes of bugs that you will then spend many hours trying to find. I have found it to be an incredible partner in writing code that allows me to sleep at night.
Q. What are the most common mistakes developers make when they try to apply patterns from other languages directly in Rust?
Evan Williams: The principal thing is, number one, trying to use Rust as if it’s an object-oriented language. It’s not going to work. Viewing the compiler errors as things that you need to figure out how to work around is something that virtually everybody who starts with Rust does. And I can’t emphasize this enough: one of the things that’s sort of interesting about Rust is the more experienced you are, the more years you have doing something in some other language, the more trouble you’re likely to have, because you have patterns of thought that come from those languages that you don’t even realize are there. That’s one of the things that can really get you into trouble. Treating the compiler errors and the problems with the borrow checker as things to work around as opposed to signals that your program needs some redesign, and thinking about things from an object-oriented perspective, those are the main traps.
Q. Traditional design patterns were created with object-oriented languages in mind. How do they evolve when applied to Rust?
Evan Williams: In a number of different ways. First, some of them evolve into entirely new forms or almost out of existence, because being a modern language, Rust has things like enums and all sorts of very advanced use of generics. These are things that make a lot of these design patterns either less necessary or unnecessary. Another way that they evolve is that since there is no inheritance in the language, you have to rethink a lot of the design patterns where, say, you would have an abstract base class. You can’t have that because there’s no such thing. So you would lean into traits. And every single design pattern that you use is affected by the borrow checker and by memory discipline in a way that is not normal in something like Java or Python.
Q. Are there any patterns that become unnecessary or even counterproductive in Rust?
Evan Williams: The one that immediately springs to mind is Singleton, which is so useful that I was using it before it had a name. But it’s useful in Python or Java. In Rust, it tends to be either completely unnecessary because other features of the language make it unneeded, or it tends to encourage designs that are really not necessary and where a much better approach could be used. There are a few occasions where the singleton pattern, as it stands, is actually useful, but more often than not, it’s getting you into trouble.
Q. What Rust-specific patterns do you think are the most powerful and still underutilized?
Evan Williams: The one that I get so excited about that I have to limit myself so that I don’t spend the rest of this conversation talking about it is the typestate pattern. This is something that was not invented for Rust, but you would think that it had been because it works for Rust so perfectly. It’s a way of developing state machines and systems that have state that evolves where invalid state transitions aren’t just errors, they’re impossible to write. The compiler won’t compile them. It represents a huge advance in the way that such systems are written because now instead of runtime errors, you have a state machine that is guaranteed to work because every transition either is a valid transition or it won’t even compile. That’s an amazing thing and I love that feature. Not invented for Rust, but it fits Rust so perfectly, it’s hard to believe it.
Q. Your book emphasizes clear data flow and system architecture. Why is unidirectional data flow so important in Rust systems?
Evan Williams: Thinking about the way data flows through your system in Rust is crucial because Rust makes it much more difficult to thread things back. As a general rule, because you can’t have many different clients holding on to different things in different places, especially not being able to write to different things in different places, having a clear direction of data flow makes it much clearer and much easier to create a consistent system that’s going to compile and work. By saying, I have a chain of ownership that moves down but never moves back up, you are now much more likely to have a system that is going to work in an environment where the number of references that can be held is very limited and you have to be very careful about memory safety. Data flowing down is something that feels natural and smooth and just works. Data trying to fight the stream back up is going to end up giving you problems because the borrow checker is not going to like you.
Q. How does Rust’s ownership model influence architectural decisions at the system level?
Evan Williams: I think one of the crucial things about it is that because you have to be so thoughtful about how your data moves and who owns it, you have to also think about what that means for your system. You need to think about who controls what, how it is controlled, and you need to start from the very beginning thinking about the boundaries of your program and the system architecture, dividing things up into areas of responsibility. Because unlike Python or Java, you can’t have links going all over the place. The borrow checker is never going to accept that. Rust gives you lots of great tools for creating boundaries of abstraction that make it a lot simpler to write code that doesn’t have hidden or difficult-to-find connections between things. You know who owns things, you know who is able to write things. In order to do that, you have to be able to think about things ahead of time and think about what parts of your program own what, and create clear boundaries and areas of responsibility for each piece of the system that you build.
Q. Can you share an example of how Rust leads to better system design compared to other languages?
Evan Williams: One of the examples in the book, one of the projects that we build in the book, is a miniature publish-and-subscribe system similar to Kafka, but very, very much smaller. It is amazing how easy it becomes to make something that is solid and clean in that circumstance. Because Rust has move semantics, you know that if something leaves here and goes here, it’s now not here anymore. It’s there. There’s no question about things like having references dangling or anything like that. The clarity of things moving through the system, the clarity of being able to have immutable data in a lot of places and knowing who can and can’t modify any piece of data, it just makes the design of the system so clear and it makes it so much harder to make a system that doesn’t work. By doing things that way, you have something where you can have a potentially very complicated system and yet have complete confidence that every piece of it individually is going to work right and that they’re going to work together as a system.
Q. What are the real-world challenges that engineering teams face when adopting Rust in production?
Evan Williams: One thing that often happens is that when a team adopts Rust, because of the challenges of learning to work in the language, velocity can go down at first. The team can find itself actually moving slower. Once the team becomes very well acquainted with Rust, velocity can increase dramatically, but there is a period of time where it seems like things have gotten worse. Another problem that often happens is that, as I said earlier, more experienced developers often have a harder time adapting to it. And the last thing I’ll mention, although it’s a lot better than it used to be, is Rust is still not 100% in terms of the kind of rich libraries that you’d find in, say, Java or C, which are just older languages. There’s more out there that supports those languages, although Rust is certainly catching up and it’s remarkable how far it’s come.
Q. When would you advise against using Rust for a project?
Evan Williams: There are a few things. If you’re doing some kind of prototyping, Rust is harder to prototype in and you really want something more like Python. If you’re working in certain niche environments where the tooling is not there, that’s a place where you don’t want to try to fight with the tooling to try to get Rust to work. It’s good to just work with the native things that are there. There are places where a dynamic language like Python is just a much easier thing to use and will more perfectly fit what you’re trying to do. And I also think there are a few areas where Rust has the potential to do a lot but is still catching up. User interfaces, for example. There are certainly user interface frameworks and libraries, but doing a website in Rust is still kind of a feat. And it’s an awful lot easier to use the tools that everybody else is using to accomplish that goal.
Q. What is the best way to introduce Rust to a team without overwhelming your developers?
Evan Williams: The thing that you need to do to start is find a piece of work that you can work on that has a limited scope and which is not in the critical path, because there are going to be roadblocks and bumps as people learn. What you don’t want to do is jump into saying, we’re just going to rewrite our project in Rust now. Pick a small piece, focus on that, gain confidence and mastery of the language, and then use that to build upon it and start bringing in more things.
Q. How should developers balance performance, safety, and complexity when designing systems in Rust?
Evan Williams: One of the nice things about it is that all of those things sort of come with the language. The thing that developers need to do is focus on letting the language help you. Rust will give you all of those things if you focus on thinking in the language and using patterns and features that are natural to the language, as opposed to trying to retool something from another language that doesn’t really fit. There are so many things that people try to do to get around problems that they have where if they just use the features of the language as they exist and did things in a way that’s more natural to the language, all of those things just fall out.
Q. How do you see Rust evolving over the next couple of years in terms of adoption and ecosystem?
Evan Williams: There are a couple of different ways you can answer that. The ecosystem is going to get richer and people are going to be branching out in the set of use cases, hitting areas that right now Rust has relatively weak support for, but where I think people are building all the time. As larger and larger projects are built, there is going to be more refinement of the language itself, but more importantly, more refinement of the use of the language. And I think this is important. This book that I wrote is not just about the language itself, but the way to use the language. And I think the Rust community and the people who are building in Rust are going to be defining and creating new ways to use it that are unique and leverage its power to do things that right now we haven’t even thought of.
Q. Did any of your own assumptions about Rust change while you were writing the book?
Evan Williams: It’s interesting because one of the things that changed is I came to recognize, through working on the early chapters about what not to do, that in Rust it’s actually more work to do things wrong. And I think that’s one of the things that surprised me. I knew that you had to think in a new way and do things differently, but when I went back and tried to write bad code in Rust, it was much harder than writing the good code. That’s an interesting perspective that just didn’t even occur to me.
Q. For developers picking up your book, what key takeaway do you hope they walk away with?
Evan Williams: The key takeaway is that the thing you want is not to learn a particular set of patterns. My book is full of what the title says, how to deal with design patterns in Rust. But the much more important thing is changing your mindset. If I can help people to recognize that there is a new mindset that they need, that’s the key thing. And I see so many people who become frustrated with Rust because Rust has such an unusual learning curve. In most languages, it’s sort of a steady progress. Maybe you plateau a little bit, but you’re always going up. With Rust, very often it seems like you’re learning and learning and learning and getting better and better. And when you reach a certain level of complexity in your programs, it feels like things are getting worse. And that’s because of the mindset. Helping people understand that will save them so much pain. That’s what I want people to take away from the book.
Q. Is there something most developers underestimate about Rust?
Evan Williams: I think the thing that people perhaps underestimate about Rust is it’s not just about memory safety and all these other things. It’s a really powerful modern programming language. It brings so much to the table that has nothing to do with memory safety or thread safety or any of those other things or high performance. It’s just a very clean, beautiful language to write in because it brings so many modern innovations that other languages are sort of stuck having to drag along historic pieces of syntax alongside. Rust is a pleasure to write in. It’s just sometimes the borrow checker can be a little annoying.
Evan Williams is the author of Design Patterns and Best Practices in Rust, published by Packt. This interview was conducted by Saqib Jan, Editor-in-Chief of Deep Engineering.