Rust in 2022

by Nick Cameron

In previous years, the core team have asked the community to write new years blog posts about Rust in the coming year. They haven't this year, but I wanted to write one anyway. However, it turned out to be harder than expected. Perhaps that's due to Rust being in an interesting place, or perhaps its because I've been away from the community for a while and then come back, but it seems I have a lot on my mind. Apologies for the long post.

Rust in 2021

Rust has had a great year. In terms of user-adoption, I've never felt as optimistic. Rust has a really solid base of users now, in terms of number of users, diversity of domains, committent of users, size of users, etc. It has a bunch of generous sponsors, both via the Foundation and in terms of companies paying folk to work full- or part-time on Rust. The community is mostly healthy. The Foundation came into being and has been finding its feet (this is probably the biggest organisational change to Rust in its entire history). On the technical front, I've been really happy using Rust - the language, libraries, and tooling all feel really solid now. Even the IDE experience is pretty good now (thanks Rust Analyzer!). The 2021 edition was released smoothly and without too much stress, and felt just right in terms of the level of change.

On the other hand, it has been a difficult year for a lot of people. Never mind the Rust stuff, we're in the middle of a horrible pandemic, travel is limited, and half the world seems to be on fire. These are stressful times. Rust leadership, especially the core team, have had it hard. Setting up the foundation was an absolutely huge amount of work. It's work that was organisational, legal, and financial in nature, i.e., not the technical work that most people signed up for. It was incredibly stressful and it is hard to talk about (thus under-appreciated). I doubt many folk outside the core team appreciate just how hard or broad this work was (I was lucky enough to leave the core team just as the work was ramping up, so I have some idea of the scope, but was spared much of the hardship). There were also inter-personal issues which flared up somewhat publicly in 2021. The Foundation is a new leadership entity in the Rust world: its existence means change, and change is difficult. Put together, this has led to Rust leaders working sub-optimally both as individuals (folk are exhausted) and as a group.

Me in 2021

I left full-time Rust project work in early 2019 and stood down from the core team in 2020 (my last project commitment). I had been working at PingCAP, mostly using Rust, but not involved too much in the community. In 2021, I changed jobs (I'm at Microsoft now) and have come back to working on Rust full-time. I also had some (intentional) time off, moved back to the UK (and had some unfortunate and unintentional time off), and got pretty stressed about Covid (and caught a dose at the start of this year).

It has been great to be getting back into Rust project work with a slightly altered perspective. It's also been surprisingly difficult (a lot has happened!) and it has been somewhat challenging to find my place. I've had to ramp up on working at Microsoft, as well as catching up with the Rust community. I've been working on the annual survey, some async things, some error handling things, and generally helping out where I can, both in the community and at Microsoft.

Rust in 2022

IMO, the important things to address in 2022 are the projects leadership issues and organisational debt (as in technical debt, but organisational, not financial debt), and refocussing to take account of Rust's level of maturity.

Keep going

Before talking about things which should change, I think it is important to mention the things which should stay the same. We are doing a lot of things right and we have to keep doing that work. An incomplete list of that work:

  • Maintenance work, keeping Rust going (bug fixes, performance improvements, ongoing support, etc.) is a huge and mostly unglamorous task.
  • Infrastructure and releases.
  • Development and maintenance of essential nut unsung tools like Rustup and Rustdoc.
  • IDE support, still a weakness for Rust but continuing to improve.
  • Work on ergonomics and polishing rough edges in the language, libs, tools, docs, ...
  • Work in the wider community on library crates.
  • Moderation and other community work.
  • Refactoring, paying off technical debt, and library-fication (e.g., Chalk) of core tooling.
  • Work on formalising Rust, Rust's memory model, better understanding unsafe code, developing miri, etc.
  • Improving error handling libraries.
  • Work on const generics.

Leadership, planning, and community

The Rust leadership structure needs an update, and it needs more energy/bandwidth (i.e., more people). I'm excited about the new additions last year to the core team. I understand there are ongoing discussions with the core team, foundation, and other team leaders to make changes and improve things. I trust these folk to make good decisions and I'm not part of those discussions, so I won't suggest much here, but I will say that I think we need quite radical change (this is not a criticism of the current organisation or people, but a reflection of the large changes which need catching up with).

The Foundation must find their stride, both in terms of their own work and how they interact with the wider organisation. Starting up something like the Foundation is hard to do; they've had a year now (and have done some cool stuff already), and have a new ED (who looks really good!). This year I think they really need to work out what they're doing and solidify their place in the project. In particular, they need to better communicate with the community.

I believe the Rust project needs better product management. At present this is ad hoc, informal, and unfocussed. We need to have a better, more cohesive vision for the project (rather than focussed on each team's remit), to better understand our users, to better prioritise work, and better track and communicate progress. We also need to start being more intentional and active about marketing/evangelism, developer relations, and developer advocacy. This has been part of the core team's responsibility but I think we should have a new product team.

The RFC process (and the MCP processes) need reform (and have needed it for a while now). I think it is essential for Rust to do its design work in the open and with the community, but the project has outgrown its current processes. This includes stabilisation of new work too - IMO this process is too slow and leaves too many features unfinished or perma-unstable. Paradoxically, it also results (IMO) in too many features being stabilised without enough thought given to their pros and cons, or to global issues of coherence or complexity.

Rust needs to have a renewed focus on its community. Rust has a great community and it is crucial to Rust's success. But nurturing the community is a big job, and I feel that it is one which has not had the right level of attention recently.

Language

The core language of Rust is in an interesting place, IMO. It is probably the area of the project seeing the most active development, but is also likely the most developed. One could argue that the language is done, at least with the currently in-flight work completed (I wouldn't quite make that argument, but I don't think it is unreasonable). Given the size and complexity of the language, I think that we are well in to the space of diminished returns on new features.

I would like to see a long-term vision for the development of the language which takes into account these diminishing returns and the maturity of the language. That vision should be used to ruthlessly prioritise development work, and in particular the pace of development should be actively controlled.

As well as doing this for the language as a whole, I think this should be done for sub-areas of the language. I think that the work on async fundamentals is a good example of this done well. I hope a similar approach can be applied to other areas.

I would like to see a renewed attempt at finishing language work. Whether that means addressing open questions and stabilising (e.g., specialization), looking again at rough spots in the language (e.g., DSTs, object safety), or removing/replacing unstable features (e.g., box patterns). I feel like there is a lot of language debt in Rust, and given how solid the fundamentals of the language are now, it is time to prioritise paying that off over working on new features (or extensions to features).

Async programming

I guess this could have been in the 'keep going' section, but it is mostly new work so I thought it's worth calling out specifically. There is a ton of work required to make async programming in Rust a good experience and I think it is an important part of the language for many users. Luckily, it is a space which has momentum and multiple people working on it, so I'm optimistic that we'll see good progress in 2022.

Compile times

Rust has a reputation for slow compile times. I'm honestly not sure how big a problem this is, but it is certainly talked about a lot (a product team will hopefully give us a better understanding of the effect of compile times on users). It is something that we've tried to tackle with a few moonshots (e.g., incremental compilation) and ongoing engineering work, but it is still one of the most common criticisms of Rust. We should do something about it, whether that is specific marketing to tell a better story, or radical changes to the compilation model.

(I'd also like to see more effort go into improving the quality of the compiler in general, such as improving code generation, rather than feature work).

Cargo and build systems

Another long-standing issue with Rust is integrating Cargo with other build systems. Lots of folk have found workarounds and we've added various tweaks to Cargo to support integration, but fundamentally Cargo still does not play well with others. I believe that the reason is the design principal that Cargo is both a package manager and build system, and that these functions are tightly coupled. I think we should move to a model where Cargo is primarily a package manager which can work with external build systems (or with the 'default' Cargo build system). This is a big undertaking, but I hope that this can be resourced this year.

Talking of Cargo as a build system, I would love to see Cargo able to share artefacts between projects and able to use pre-built artefacts as a means to reduce build times. This could lead to integrating most Rustup functionality in Cargo (which would be great, IMO) and ties in to the next section on making std more like any other crate.

The crate model, a versioned std, and versioned tools

I would like to see Cargo support multi-crate packages. This would be a big change and requires changes to rustc, Cargo, and maybe crates.io. The idea is that a Rust project would be published as a single package, but contain multiple compilation units (crates). Coherence rules and versioning would apply to the entire package not individual crates. This would replace the workspace concept (which is not fit for purpose, IMO). I think these changes would improve compile times (since crates could be smaller), make modularisation better and decisions easier, help with reasoning about unsafe (smaller crates permit reasoning about crates rather than modules), make publishing packages easier, and generally improve the state if the ecosystem.

It also seems like a good step towards a better libstd, let me explain! I would like to see std more like a versioned crate to allow more evolution and fine-grained use of std. However, one problem with that is that if different crates are using different major versions, then you get incompatibility issues, such as not being able to pass Vecs from one crate to another. If std had multiple crates in a single package, we could evolve some parts without breaking compatibility of others (note, that this somewhat contradicts the comment above about versioning be per-package rather than per-crate, but I think we can work this out).

Similarly, I'd like to be able to specify tool versions (and maybe features) in Cargo.toml in the same way as dependencies. That would allow tools like Rustfmt to have a better back compat/evolution story and integrate more smoothly with the Cargo experience.

Debugging

Finally, I would love to see some love given to debugging Rust programs. Microsoft (among others) are doing some work on improving traditional debugging (i.e., using existing debug symbol standards with existing debuggers). That's important for programs which include Rust and other languages (which is all of them, but often developers don't need to debug the parts of a program written in different languages together) and there is a lot to do there.

I'd also like to see some radical experimentation with more language-specific ways to debug Rust programs. Can we do better than the dbg macro? Can we integrate better with testing? Could we build other tools like the Tokio console to help debug programs? A language-specific debugger?