Portable and interoperable async Rust

A goal of the async foundations working group is for async Rust to be portable and interoperable. I want to dig in to what that means in this blog post. For a little background, see my earlier post on async runtimes.

To run async Rust code, you need an async runtime. Currently however, choosing a runtime locks you into a subset of the ecosystem. Library crates and tools are often restricted to a specific runtime; changing runtime is difficult. Mixing and matching libraries from different runtime ecosystems requires running multiple executors and using imperfect compatibility layers. For programmers getting started with async Rust, finding and choosing a runtime is a source of friction.

I plan to spearhead this initiative. I'll work closely with the authors of the major async runtimes since they are important stakeholders. If you're also interested in improving this aspect of async Rust, come say 'hi!' on Zulip.

The shiny future

The full solution here is unknown. Achieving our goal of portability and interoperability will require a lot of technical and non-technical work, and I expect we'll need to iterate a lot, with feedback from users and runtime authors.

The high-level vision is that choosing a runtime won't commit you to a specific fragment of the ecosystem. Nearly all libraries should be independent of the choice of runtime. Combining different libraries should Just Work. Changing runtime should be relatively easy, and you should be able to get started with async programming without the friction of finding and choosing a runtime.

I expect there will always be different runtimes. I can't imagine there will ever be one ultimate or standard executor. Application's constraints and trade-offs are too diverse and specific. But I hope we can standardise a lot of the abstractions and utilities in the async ecosystem so that async code is much more portable. To achieve this, I think a lot of these things should be in the standard library rather than bundled with an executor.

I expect that writing an async executor will always be difficult. They are extremely performance sensitive, low-level, and intrinsically concurrent. They often need to be highly tunable and configurable. However, it should be easier than it is. I hope that the building-block components of runtimes could be widely available and easier to use. And of course, once you've built your own executor, you should be able to interoperate with the entire ecosystem, rather than need to rebuild it.

The plan

Kidding, I don't have a plan. I do have some ideas for things to investigate though. Before I go through those, I want to discuss the timeline I'm imagining (spoiler: it's long). I think that we'll want to make concurrent progress towards the below goals. I think there are some important inter-dependencies, but I think that all this work is fundamentally slow - it requires changes to std which involves RFCs, stabilisation, and discussion. It will also require changes to multiple, independent runtime projects whose maintainers all have their own constraints and goals, and not very much spare time.

That might sound a bit negative, but I think it is only realistic expectation setting. It is also why I want to start some things early, even though there are big unresolved questions which need answering before they can be 'finished'. A lot of the APIs we want to standardise will be fundamentally affected by such questions, e.g., is async overloading feasible in Rust? Do we have enough experience with completion-style async IO (IOCP, io_uring, etc.) to be sure that our APIs are optimal?

A standard library for async Rust

Currently, many runtimes effectively provide their own standard library for async Rust. This inevitably hinders interoperability because changing executor often means changing usage of unrelated (or partially related) library code.

I think we should end up with one set of core async libraries and that should be part of std. That includes IO traits (including AsyncRead/AsyncWrite) and helper functionality, concurrency primitives such as locks and channels, and other utilities such as timers.

This is more challenging than it sounds because different runtimes do not agree on what the APIs for common types should be (and there are design ideas and design problems beyond anything existing today). Furthermore, on the implementation side, even code which seems like it should be runtime-independent might depend on the runtime to facilitate pre-emption, etc.

I think that working on IO traits is a good place to start the interoperability work because it is a well-defined problem, but an extremely tricky one. I expect it will take a long time to solve (there are difficult technical and social questions, dependencies on other open questions, etc.), but I also think it is necessary to solve, and that solving it will bring large benefits to the ecosystem. I think it's an important area to start early, even though there are good reasons that it might not be possible to solve in the near future (more on this in another post).

An API for executors

A key goal is to be able to easily switch between executors. Currently, there is no standardised interface for executors, so how a program interacts with an executor is unique to each executor. On the bright side, most executors offer similar APIs (e.g., a spawn function). However, there are differences in the details and some constraints which are not reflected simply by function signatures (e.g., Tokio's requirement that all async code is within the context of an executor).

One approach would be to define a Spawn or Executor trait in std. In addition, there must be a way to 'plug in' the executor, since executors tend to be somewhat global, rather than passed around. We can follow follow the global allocator pattern, but this is not a complete solution since some programs require multiple executors.

Ease of use

Using and learning async Rust is higher friction than with purely synchronous Rust. It has certainly got easier since most runtimes include a macro for a main function and tests, and so forth, but I think we can do better. I would like to be able to write async fns for main and #[test] functions without magic macros. Ideally, a new learner shouldn't have to put time and effort into picking a runtime before they can experiment with async code.

I don't think we should include an industrial-strength executor in std - I don't think we can choose an executor that covers all bases well enough to justify standardising on it. However, I think that including a minimal (or 'toy') executor might be possible, which could permit executing futures but which is not feature complete, possibly omitting properly asynchronous IO, spawning tasks, or other 'essential' functionality. If we do go down this path, then I believe that an important constraint is that switching out the minimal executor for an industrial-strength one should be very low friction (the principle that 'doing the right thing is the easy thing').

Open questions

Basically everything in this area is an open question at the moment. But there are a few things which I think are particularly unknown.

First is the question of async overloading (see this blog post for an overview and some thoughts from Yosh). If async overloading is possible, ergonomic, etc., then it presents opportunities for very different approaches to including async support in std compared to what is possible today.

Second, how much of the internals of async runtimes can be shared? One of the goals of the async foundations WG is to make it easier to write your own executor. One thing that could be done for that (and which would probably have other benefits) would be to abstract functionality which is likely to be shared by all runtimes. Async-task is an effort to do that which already exists. I wonder if there is benefit to extending this idea to provide universal building blocks (possibly even adding such a library to std), or whether runtime implementations are so diverse that this is a dead end.

Guiding principles

When undertaking this kind of design work, it is important to have good guiding principles. One of the first jobs to do is to make sure our principles are correct, and to think about their relative priorities. Currently I think these guiding principles are an open question. Some of our principles (on top of the usual (mostly implicit) principles of Rust language and library design) might be:

• Async APIs should be as close as possible to their synchronous equivalents.
• Applications should choose a runtime, libraries should be runtime-agnostic.
• Executors/runtimes must have enough flexibility to optimally satisfy their specific constraints.
• It is a non-goal for async support in std to be minimal (i.e., we don't want something like the futures crate to be a long-term solution).

To conclude

I've laid out (at a very high level) what I think needs to be done to get to an async ecosystem where portability and interoperability are a reality. The next steps are to figure how to get there in more detail, and to start the hard work! We're going to need lots of help, so if you would like to plan, discuss, design, or implement this stuff, come chime in on Zulip, or on our Github repo.