Ten challenges for Rust
Rust is in a pretty good place; it is getting more and more popular, has more and more contributors, and is used in some pretty significant places. However, it is a time of flux and change, and transitioning from a research project then a new, rapidly changing language to a popular, established project is a difficult change.
Here, I want to describe what I think are the ten biggest challenges for Rust for now and the next few years. I have some ideas for solutions, but they are all big, difficult questions with no simple answer, so the real solutions will all take iteration, energy, and creativity. My focus is on the core project; there are many challenges for the community and ecosystem (e.g., how to make GUIs with Rust, or how to get more crates to 1.0) which I think must be primarily solved by the community.
Governance challenges
1 How to steer development and keep Rust's open character?
There is a tension in open source work between what would be best for the project and what volunteer contributors want to work on. If you're lucky these two things coincide and things just work, but more often than not that is not exactly the case. There are many things you can do to ease the tension: pay people, have a compelling vision and good skills of persuasion, have a strict policy for PR acceptance, gamify things, incentivize certain work with rewards like exposure, gratitude, authority, or eventual job offers, etc.
Since Rust's community has grown and Mozilla's direct support has ended, this tension seems to have gotten worse for Rust. Though there are many folks doing essential maintenance work which is often under-staffed, there is a lack of resources in some important areas and an absence of strategy work or effort to steer contribution.
In some ways, it might be easy to have a more top-down approach. However, I think this would lose some of the advantages that Rust has from being an open and philosophically open source project. The big challenge here is making sure the important but unattractive work is done without losing the parts of the project's character which has made it awesome.
Some specific things I think we're currently struggling with:
- prioritize finishing things over starting new things,
- prioritize tooling, libraries, and non-technical work as well as the language and compiler,
- prioritize lower impact, lower cost work which in aggregate could have large impact (over big, glamorous work).
Related to this challenge is keeping Rust's essential character in the face of growth. In particular, how does the project grow and accept new contributors and leaders (and the inevitable change that brings) without losing sight of Rust's central mission? How do we continue to be open and transparent in our discussion and decision making as the number of observers (and commenters) grows?
2 Diversity and inclusion
Despite being known for having a welcoming and friendly community, Rust's diversity numbers are terrible. They are even terrible compared to the tech industry, which is terrible to begin with. Though we probably do better at being an inclusive project (our numbers for making people feel welcome/unwelcome are encouraging), the fact that so many contributors have felt they have to leave the project for negative reasons suggest we should be doing much better (yes, avoiding burnout is part of inclusivity too).
An important aspect of inclusivity is being able to accommodate a diversity of opinions. If we can only get along when everyone agrees, then we cannot be diverse or inclusive. While our preference for consensus has served us well in some areas, it has also caused problems. Our culture of avoiding conflict rather than resolving it is unhealthy and has led to dysfunctional governance.
These are really hard things to solve! But we must prioritize solving them even though they are difficult and sometimes painful.
3 Avoid ossification of inefficient processes
Rust has grown incredibly over the last few years, but our processes and organization have not kept pace. There is a huge amount of inertia around change in anything governance or process related. Even when existing processes have a huge amount of friction, it seems impossible to do anything but tweak things. We have accumulated so much organizational debt that radical change is needed, but making that change is going to be very difficult.
I believe that at the heart of the problem is a reluctance for the project to accept 'management' (people management, project management, product management) as an essential part of project leadership. These things can be independent of technical leadership, but that requires real delegation of power (not just delegation of work). The challenge for the project is to accept that delegation, support these activities, and bring in new processes which better support the project.
Ecosystem challenges
4 Navigating the crate ecosystem
Rust has struck a good balance between a minimal standard library and 'batteries included'. This has been possible due to a thriving ecosystem and easy to use package manager. However, navigating the crate ecosystem has always been tricky. There are a lot of crates and finding the right one for the job requires a lot of effort or being very involved with the community. This will become more of a problem as we get more and more users who are not active participants in the community, and as the number of crates grows.
5 The async ecosystem
Async programming is important for a lot of the domains Rust targets and works pretty well with Rust's programming model. However, the ecosystem is somewhat split around different runtimes and our efforts to improve things have been slow. We're working on it, but progress is slow and eventual success is uncertain. The risk is that we end up bringing things in to the standard library, these don't get much uptake with the community and we end up with a worse ecosystem than we started with.
Technical challenges
6 How to make the language more broadly appealing without losing its core focus?
Rust has been pretty successful in its niche, and I think there is still a lot of room for growth. However, Rust could be so much more than that. There is a lot of software which is today written in languages with managed runtimes which are performance sensitive and where Rust's focus on safety, ergonomics, and performance could make better products and improve productivity. However, currently Rust is too difficult to learn and requires too much cognitive overhead compared to GC'ed languages to succeed there. Making Rust meaningfully easier to learn and easier to use could mean Rust having a much bigger impact.
I don't think supporting GC, having sugary syntax for Rc<RefCell<T>> types, or adding a bunch of syntax sugar is the solution. We have to make things easier without losing the advantages that Rust has for its core focus of systems programming. Reducing the need for explicit lifetimes, making the borrow checker more powerful, not overcomplicating the trait system, focusing on user experience, and avoiding becoming a huge language will all help. Perhaps we'll find new abstractions which significantly simplify ownership and lifetimes?
7 Memory model and unsafe code
Safety is one of Rust's major offerings and motivates many people to use it. At the boundary of safety is unsafe code and unfortunately there is no smooth transition from safe to unsafe, just an unsafe cliff into a cold, scary world with little support from tooling and worse ergonomics than C. We need to be able to offer more support and a better experience to programmers doing unsafe work. To do that we need a clearer understanding of Rust's memory model and then to develop language features, libraries, and tools.
Fortunately, there is a lot of energy in this area. There is academic research, great work in the community, MIRI, the unsafe code guidelines, and more. Unfortunately, it's a very involved and difficult area and lots of folk on the periphery have opinions which slows down progress and makes the work more of a slog for those involved. Some of the changes which might be really impactful might be too large to make for both political and technical reasons (see the ossification challenge above and the big compiler changes challenge below).
8 Evolving the standard library
Rust has a strict and strong approach to stability, including well-defined guarantees around backwards compatibility. For the language, editions allow some backwards-incompatible evolution without breakage. For libraries in the ecosystem, Cargo and a Semver culture permit evolution. However, the standard library has no way to evolve other than monotonically growing (things can be deprecated but never removed, and things can't be changed). By itself, this will lead to an increasingly large and messy standard library. But there are also second order effects: we have to be extremely conservative about stabilization, and there is no possible status for APIs other than 'stable forever' and 'only usable on nightly and completely subject to change' (c.f., crates where pre-1.0 versions can be used with the stable compiler and post-1.0 versions can be opted into).
Relatedly, the standard library is an all or nothing deal (technically there is liballoc too). As well as having a more fine-grained solution for versioning, it would be beneficial to have more fine-grained use of the standard library, rather than just core or all of std.
9 Big compiler changes
Rustc is a pretty huge piece of software now. It has a lot of inherent complexity (Rust is a complicated language and compiling it fast while giving good error messages is really hard), a lot of the usual problems of large software, and lots of technical debt. There are some big challenges, in particular around compile time (of which incremental and parallel compilation are two work-in-progress approaches), and these are proving hard work to implement. Work like Chalk and Pelonius are huge (multi-person, multi-year) projects. Making big changes in the future is only going to get harder. Luckily, the compiler team are competent, energetic, and well-resourced. But, it will still be challenging to make big, high-impact changes to rustc.
10 Macros
Macros have a bunch of rough edges and one of the least polished parts of the language. Declarative macros introduce a whole new sub-language. Procedural macros are clunky, require heavy dependencies, and are difficult to master. All macros work poorly with the compiler (compile times, incremental compilation, error messages) and tools (all sorts of problems for IDEs, rustdoc, etc.).
The reason I think this is a big challenge (and not just another language feature which can be worked on) is that the problems are diffuse and difficult. There is (probably) no silver bullet solution, just lots and lots of engineering and design work. Many problems (e.g., macro hygiene) require specialized knowledge which is not present in the community. Macros are also neither high priority enough (they essentially work, after all) nor glamorous enough to attract contributors.
The outlook
It can feel a bit negative to list ten problems and describe them as 'big'. But I think these are realistic challenges. The good news is that (I think) all of them are well-known issues to various people in the project and that people are actively working on many of them. Although any solutions are difficult, I think there are viable and realistic solutions to all these challenges. If we can focus on solving these (and of course all the other day to day challenges), then I think Rust will continue to grow and be successful.