What is an async runtime?

Unlike other Rust features, you can't just write await in your code and run it. You need to use an async runtime like Tokio or async-std. But why? And what do these runtimes do?

I'm starting to work on the portability and interoperability initiative, part of the Async Foundations Working Group's effort to improve async programming in Rust. This is a background blog post for that work. I hope to have a couple more blog posts soon about the challenges and goals of the work. If you're interested in this area and want to get involved, let me know!

A lightning-quick refresher on futures

A future represents an asynchronous computation. In other words, it is a value that may still be in the process of being computed. In Rust, a future is an object which implements the Future trait and that only requires implementing a poll method. To use a future, one just has to repeatedly call the poll method until it returns something useful.

An async function is just a convenient way to write a function which returns a future. All that is required is for something to call poll.

Executors

So far, so simple. But it turns out that how one calls the poll function is an important and difficult question. We don't want to call it too often because that would waste CPU cycles and potentially starve other tasks (we'll get to what exactly a task is soon) from making progress. We don't want to call it too infrequently though because that means our future won't make progress as quickly as it should.

Furthermore, there is no best way to solve this problem. Different constraints and environments make different approaches more or less optimal.

Executors typically present two key APIs: block_on and spawn. block_on runs a future to completion on the current thread, blocking any other activity, and returning when the future is done. spawn runs a future without blocking the current thread, returning immediately. Sophisticated executors may support variations on these functions.

IO

When I talked about futures earlier, I said that they represent a value in the process of being computed. But the real motivation for futures is usually not active CPU work but waiting for IO to complete. Futures (and asynchronous programming in general) are most efficient when you have a lot of concurrent tasks which have to do a lot of waiting for IO.

Operating systems provide functionality for this kind of asynchronous IO, but it is very low-level (e.g., the epoll API). A good executor must interact with the OS so that it can wake futures at the optimal time (this is sometimes called a reactor).

Separately, to actually use asynchronous IO, we need a bunch of abstractions - traits to abstract different kinds of IO (e.g., async equivalents of the Read and Write traits from std), abstractions over files, sockets, signals, and other IO interfaces, and utilities for things like buffering or path operations.

So what is a runtime?

A runtime starts with an executor for running futures, including interacting with the OS for driving IO operations. It usually includes a library of traits, types, and functions for async IO. Often, the runtime will also expose some of its building blocks for advanced users, such as the Task abstraction from async-std, as well as utilities such as helpers for pinning or writing an async main function. Finally, a runtime often includes other foundations for async programming (e.g., channels and locks, or timer functionality) which are to some extent tied to the executor.

In other words, the essential part of an async runtime is the executor, but an async runtime also includes libraries which make it practical (not just possible) to write async code. In doing so, an async runtime is like a standard library for async programming and a key part of an async ecosystem.

A short and incomplete survey of runtimes

I'll briefly describe the defining characteristics (IMO) of the major async runtimes. This is not a deep comparison and I won't attempt to choose a best runtime (this depends on your program's particular constraints).

There are other runtimes available (and presumably some closed source ones which are not generally available). Some of these others are not widely known or used (or at least not known by me), others are specialised to a single use case (e.g., Fuschia's runtime), primarily educational rather than meant for production use (e.g., Whorl), or are adaptions of other runtimes (e.g., AIUI Actix uses Tokio internally, but adapts it for its own needs).

Futures

The futures crate is a Rust-official crate. It started as a place to develop the Future trait and associated libraries, on their way to inclusion in Rust's standard library. It is now mostly in maintenance mode. It includes a basic executor and a bunch of useful traits for streams, IO, etc. However, it omits the low-level functionality for async IO, so it is not a complete solution. Thus, it is widely used for its traits and stream support, etc., but usually in conjunction with an executor (and concrete IO types) from another runtime.

Tokio

Tokio is one of the oldest and mostly widely-used runtimes, and probably the most used in production. It has a highly performant, customisable, and flexible executor. Of note is that futures do not have to be passed to a background thread for execution which is great for performance, but requires rather strict rules about being 'in context' of the executor.

Tokio's IO traits are a bit different from most others in the ecosystem and the synchronous versions in the standard library. Specifically, data is read into an abstract buffer type, rather than an &mut [u8]. That makes them more flexible and sometimes more performant, but also a little trickier to use.

Smol and async-std

Async-std and Smol are two runtimes based on smol-rs components. Async-std is designed to be as close to the synchronous standard library as possible, while Smol is designed to be more minimal. They are widely used and production-ready. In contrast to Tokio, the Smol executor always uses background threads for executing futures which sacrifices some potential performance for better usability. Async-std/Smol include some interesting building blocks for runtimes, such as the Task abstraction - one of the goals of Smol is to provide building blocks for runtimes, not just an out-of-the-box experience.

The IO traits and utilities of async-std/Smol are mostly inherited from the futures crate.

Glommio

Glommio is a specialised runtime based on the thread-per-core philosophy and implemented using io_uring. It is primarily designed for disk IO (c.f., network IO which has been the traditional motivator for async programming). Unlike Tokio and async-std, Glommio is not a general purpose async runtime and doesn't include things like an AsyncRead trait. But for its use case, it is a complete solution.

Embassy

Embassy is a runtime designed specifically for embedded development. In particular, it avoids allocation and does not require a heap.

Bastion

Bastion is an actors runtime, and thus a bit higher-level and more opinionated than general purpose runtimes. It is designed to be fault-tolerant and highly available. It doesn't have libraries for general IO (I assume you would use it with the futures crate in practice).

Acknowledgement

I'd like to thank Yoshua Wuyts for feedback on this post.