core (roadmap)
This RFC proposes the 2017 Rust Roadmap, in accordance with RFC 1728. The goal of the roadmap is to lay out a vision for where the Rust project should be in a year's time. This year's focus is improving Rust's productivity, while retaining its emphasis on fast, reliable code. At a high level, by the end of 2017:
In addition, we should make significant strides in exploring two areas where we're not quite ready to set out specific goals:
The proposal is based on the 2016 survey, systematic outreach, direct conversations with individual Rust users, and an extensive internals thread. Thanks to everyone who helped with this effort!
There's no end of possible improvements to Rust—so what do we use to guide our thinking?
The core team has tended to view our strategy not in terms of particular features or aesthetic goals, but instead in terms of making Rust successful while staying true to its core values. This basic sentiment underlies much of the proposed roadmap, so let's unpack it a bit.
What does it mean for Rust to be successful? There are a lot of good answers to this question, a lot of different things that draw people to use or contribute to Rust. But regardless of our personal values, there's at least one clear measure for Rust's broad success: people should be using Rust in production and reaping clear benefits from doing so.
Production use matters for the obvious reason: it grows the set of stakeholders with potential to invest in the language and ecosystem. To deliver on that potential, Rust needs to be part of the backbone of some major products.
Production use measures our design success; it's the ultimate reality check. Rust takes a unique stance on a number of tradeoffs, which we believe to position it well for writing fast and reliable software. The real test of those beliefs is people using Rust to build large, production systems, on which they're betting time and money.
The kind of production use matters. For Rust to truly be a success, there should be clear-cut reasons people are employing it rather than another language. Rust needs to provide crisp, standout benefits to the organizations using it.
The idea here is not about "taking over the world" with Rust; it's not about market share for the sake of market share. But if Rust is truly delivering a valuable new way of programming, we should be seeing that benefit in "the real world", in production uses that are significant enough to help sustain Rust's development.
That's not to say we should expect to see this usage immediately; there's a long pipeline for technology adoption, so the effects of our work can take a while to appear. The framing here is about our long-term aims. We should be making investments in Rust today that will position it well for this kind of success in the future.
At this point, we have a fair amount of data about how Rust is reaching its audience, through the 2016 survey, informal conversations, and explicit outreach to (pre-)production shops (writeup coming soon). The data from the survey is generally corroborated by these other venues, so let's focus on that.
We asked both current and potential users what most stands in the way of their using Rust, and got some pretty clear answers:
None of these obstacles is directly about the core language or std; people are
generally happy with what the language offers today. Instead, the connecting
theme is productivity—how quickly can I start writing real code? bring up a
team? prototype and iterate? debug my code? And so on.
In other words, our primary challenge isn't making Rust "better" in the abstract; it's making people productive with Rust. The need is most pronounced in the early stages of Rust learning, where we risk losing a large pool of interested people if we can't get them over the hump. Evidence from the survey and elsewhere suggests that once people do get over the initial learning curve, they tend to stick around.
So how do we pull it off?
Part of what makes Rust so exciting is that it attempts to eliminate some seemingly fundamental tradeoffs. The central such tradeoff is between safety and speed. Rust strives for
and delivers on this goal largely thanks to its fundamental concept of ownership.
But there's a problem: at first glance, "productivity" and "learnability" may seem at odds with Rust's core goals. It's common to hear the refrain that "fighting with the borrow checker" is a rite of passage for Rustaceans. Or that removing papercuts would mean glossing over safety holes or performance cliffs.
To be sure, there are tradeoffs here. But as above, if there's one thing the Rust community knows how to do, it's bending the curve around tradeoffs—memory safety without garbage collection, concurrency without data races, and all the rest. We have many examples in the language where we've managed to make a feature pleasant to use, while also providing maximum performance and safety—closures are a particularly good example, but there are others.
And of course, beyond the core language, "productivity" also depends a lot on tooling and the ecosystem. Cargo is one example where Rust's tooling provides a huge productivity boost, and we've been working hard on other aspects of tooling, like the compiler's error messages, that likewise have a big impact on productivity. There's so much more we can be doing in this space.
In short, productivity should be a core value of Rust. By the end of 2017, let's try to earn the slogan:
In the abstract, reaching the kind of adoption we need means bringing people along a series of distinct steps:
We need to (1) provide "drivers", i.e. strong motivation to continue through the stages and (2) avoid "blockers" that prevent people from progressing.
At the moment, our most immediate adoption obstacles are mostly about blockers, rather than a lack of drivers: there are people who see potential value in Rust, but worry about issues like productivity, tooling, and maturity standing in the way of use at scale. The roadmap proposes a set of goals largely angled at reducing these blockers.
However, for Rust to make sense to use in a significant way in production, it also needs to have a "complete story" for one or more domains of use. The goals call out a specific domain where we are already seeing promising production use, and where we have a relatively clear path toward a more complete story.
Almost all of the goals focus squarely on "productivity" of one kind or another.
Now to the meat of the roadmap: the goals. Each is phrased in terms of a qualitative vision, trying to carve out what the experience of Rust should be in one year's time. The details mention some possible avenues toward a solution, but this shouldn't be taken as prescriptive.
These goals are partly informed from the internals thread about the roadmap. That thread also posed a number of possible additional goals. Of course, part of the work of the roadmap is to allocate our limited resources, which fundamentally means not including some possible goals. Some of the most promising suggestions that didn't make it into the roadmap proposal itself are included in the Alternatives section.
Rust offers a unique value proposition in part because it offers a unique feature: its ownership model. Because the concept is not (yet!) a widespread one in other languages, it is something most people have to learn from scratch before hitting their stride with Rust. And that often comes on top of other aspects of Rust that may be less familiar. A common refrain is "the first couple of weeks are tough, but it's oh so worth it." How many people are bouncing off of Rust in those first couple of weeks? How many team leads are reluctant to introduce Rust because of the training needed? (1 in 4 survey respondents mentioned the learning curve.)
Here are some strategies we might take to lower the learning curve:
Improved docs. While the existing Rust book has been successful, we've learned a lot about teaching Rust, and there's a rewrite in the works. The effort is laser-focused on the key areas that trip people up today (ownership, modules, strings, errors).
Gathering cookbooks, examples, and patterns. One way to quickly get productive in a language is to work from a large set of examples and known-good patterns that can guide your early work. As a community, we could push crates to include more substantial example code snippets, and organize efforts around design patterns and cookbooks. (See the commentary on the RFC thread for much more detail.)
Improved errors. We've already made some big strides here, particularly for ownership-related errors, but there's surely more room for improvement.
Improved language features. There are a couple of ways that the language design itself can be oriented toward learnability. First, we can introduce new features with an explicit eye toward how they will be taught. Second, we can improve existing features to make them easier to understand and use -- things like non-lexical lifetimes being a major example. There's already been some discussion on internals
IDEs and other tooling. IDEs provide a good opportunity for deeper teaching. An IDE can visualize errors, for example showing you the lifetime of a borrow. They can also provide deeper inspection of what's going on with things like method dispatch, type inference, and so on.
The edit-compile-debug cycle in Rust takes too long, and it's one of the complaints we hear most often from production users. We've laid down a good foundation with MIR (now turned on by default) and incremental compilation (which recently hit alpha). But we need to continue pushing hard to actually deliver the improvements. And to fully address the problem, the improvement needs to apply to large Rust projects, not just small or mid-sized benchmarks.
To get this done, we're also going to need further improvements to the performance monitoring infrastructure, including more benchmarks. Note, though, that the goal is stated qualitatively, and we need to be careful with what we measure to ensure we don't lose sight of that goal.
While the most obvious routes are direct improvements like incremental compilation, since the focus here is primarily on development (including debugging), another promising avenue is more usable debug builds. Production users often say "debug binaries are too slow to run, but release binaries are too slow to build". There may be a lot of room in the middle.
Depending on how far we want to take IDE support (see below), pushing incremental compilation up through the earliest stages of the compiler may also be important.
For many people—even whole organizations—IDEs are an essential part of the programming workflow. In the survey, 1 in 4 respondents mentioned requiring IDE support before using Rust seriously. Tools like Racer and the IntelliJ Rust plugin have made great progress this year, but compiler integration in its infancy, which limits the kinds of tools that general IDE plugins can provide.
The problem statement here says "solid, but basic" rather than "world-class" IDE support to set realistic expectations for what we can get done this year. Of course, the precise contours will need to be driven by implementation work, but we can enumerate some basic constraints for such an IDE here:
Note that while some of this functionality is available in existing IDE/plugin efforts, a key part of this initiative is to (1) lay the foundation for plugins based on compiler integration (2) pull together existing tools into a single service that can integrate with multiple IDEs.
Another major message from the survey and elsewhere is that Rust's ecosystem, while growing, is still immature (1 in 9 survey respondents mentioned this). Maturity is not something we can rush. But there are steps we can take across the ecosystem to help improve the quality and discoverability of crates, both of which will help increase the overall sense of maturity.
Some avenues for quality improvement:
cargo new set up Travis/Appveyor.Some avenues for discoverability improvement:
A number of ideas along these lines were discussed in the Rust Platform thread.
The biggest area we've seen with interest in production Rust so far is the server, particularly in cases where high-scale performance, control, and/or reliability are paramount. At the moment, our ecosystem in this space is nascent, and production users are having to build a lot from scratch.
Of the specific domains we might target for having a more complete story, Rust on the server is the place with the clearest direction and momentum. In a year's time, it's within reach to drastically improve Rust's server ecosystem and the overall experience of writing server code. The relevant pieces here include foundations for async IO, language improvements for async code ergonomics, shared infrastructure for writing services (including abstractions for implementing protocols and middleware), and endless interfaces to existing services/protocols.
There are two reasons to focus on the robust, high-scale case. Most importantly, it's the place where Rust has the clearest value proposition relative to other languages, and hence the place where we're likeliest to achieve significant, quality production usage (as discussed earlier in the RFC). More generally, the overall server space is huge, so choosing a particular niche provides essential focus for our efforts.
Rust has taken a decidedly lean approach to its standard library, preferring for
much of the typical "batteries included" functionality to live externally in the
crates.io ecosystem. While there are a lot of benefits to that approach, it's
important that we do in fact provide the batteries somewhere: we need 1.0-level
functionality for essential tasks. To pick just one example, the rand crate
has suffered from a lack of vision and has effectively stalled before reaching
1.0 maturity, despite its central importance for a non-trivial part of the
ecosystem.
There are two basic strategies we might take to close these gaps.
The first is to identify a broad set of "essential tasks" by, for example, finding the commonalities between large "batteries included" standard libraries, and focus community efforts on bolstering crates in these areas. With sustained and systematic effort, we can probably help push a number of these crates to 1.0 maturity this year.
A second strategy is to focus specifically on tasks that play to Rust's strengths. For example, Rust's potential for fearless concurrency across a range of paradigms is one of the most unique and exciting aspects of the language. But we aren't fully delivering on this potential, due to the immaturity of libraries in the space. The response to work in this space, like the recent futures library announcement, suggests that there is a lot of pent-up demand and excitement, and that this kind of work can open a lot of doors for Rust. So concurrency/asynchrony/parallelism is one segment of the ecosystem that likely deserves particular focus (and feeds into the high-scale server goal as well); there are likely others.
When working with larger organizations interested in using Rust, one of the
first hurdles we tend to run into is fitting into an existing build
system. We've been exploring a number of different approaches, each of which
ends up using Cargo (and sometimes rustc) in different ways, with different
stories about how to incorporate crates from the broader crates.io ecosystem.
Part of the issue seems to be a perceived overlap between functionality in Cargo
(and its notion of compilation unit) and in ambient build systems, but we have
yet to truly get to the bottom of the issues—and it may be that the problem is
one of communication, rather than of some technical gap.
By the end of 2017, this kind of integration should be easy: as a community, we should have a strong understanding of best practices, and potentially build tooling in support of those practices. And of course, we want to approach this goal with Rust's values in mind, ensuring that first-class access to the crates.io ecosystem is a cornerstone of our eventual story.
The Rust community is awesome, in large part because of how welcoming it is. But we could do a lot more to help grow people into roles in the project, including pulling together important work items at all level of expertise to direct people to, providing mentoring, and having a clearer on-ramp to the various official Rust teams. Outreach and mentoring is also one of the best avenues for increasing diversity in the project, which, as the survey demonstrates, has a lot of room for improvement.
While there's work here for all the teams, the community team in particular will continue to focus on early-stage outreach, while other teams will focus on leadership onboarding.
The goals above represent the steps we think are most essential to Rust's success in 2017, and where we are in a position to lay out a fairly concrete vision.
Beyond those goals, however, there are a number of areas with strong potential for Rust that are in a more exploratory phase, with subcommunities already exploring the frontiers. Some of these areas are important enough that we want to call them out explicitly, and will expect ongoing progress over the course of the year. In particular, the subteams are expected to proactively help organize and/or carry out explorations in these areas, and by the end of the year we expect to have greater clarity around Rust's story for these areas, putting us in a position to give more concrete goals in subsequent roadmaps.
Here are the two proposed Areas of Exploration.
Other languages here includes "low-level" cases like C/C++, and "high-level" cases like JavaScript, Ruby, Python, Java and C#. Rust adoption often depends on being able to start using it incrementally, and language integration is often a key to doing so -- an intuition substantiated by data from the survey and commercial outreach.
Rust's core support for interfacing with C is fairly strong, but wrapping a C library still involves tedious work mirroring declarations and writing C shims or other glue code. Moreover, many projects that are ripe for Rust integration are currently using C++, and interfacing with those effectively requires maintaining an alternative C wrapper for the C++ APIs. This is a problem both for Rust code that wants to employ existing libraries and for those who want to integrate Rust into existing C/C++ codebases.
For interfacing with "high-level" languages, there is the additional barrier of working with a runtime system, which often involves integration with a garbage collector and object system. There are ongoing projects on these fronts, but it's early days and there are still a lot of open questions.
Some potential avenues of exploration include:
repr).Rust is a natural fit for programming resource-constrained devices, and there are some ongoing efforts to better organize work in this area, as well as a thread on the current significant problems in the domain. Embedded devices likewise came up repeatedly in the internals thread. It's also a potentially huge market. At the moment, though, it's far from clear what it will take to achieve significant production use in the embedded space. It would behoove us to try to get a clearer picture of this space in 2017.
Some potential avenues of exploration include:
std in various ways, depending on
platform capabilities.std (i.e., without aborting when
out of memory).Finally, it's important that the roadmap "have teeth": we should be focusing on the goals, and avoid getting distracted by other improvements that, whatever their appeal, could sap bandwidth and our ability to ship what we believe is most important in 2017.
To that end, it's worth making some explicit non-goals, to set expectations and short-circuit discussions:
No major new language features, except in service of one of the goals. Cases that have a very strong impact on the "areas of support" may be considered case-by-case.
No major expansions to std, except in service of one of the goals. Cases
that have a very strong impact on the "areas of support" may be considered
case-by-case.
No Rust 2.0. In particular, no changes to the language or std that could be
perceived as "major breaking changes". We need to be doing everything we can
to foster maturity in Rust, both in reality and in perception, and ongoing
stability is an important part of that story.
It's a bit difficult to enumerate the full design space here, given how much there is we could potentially be doing. Instead, we'll take a look at some alternative high-level strategies, and some additional goals from the internals thread.
At a high level, though, the biggest alternatives (and potential for drawbacks) are probably at the strategic level. This roadmap proposal takes the approach of (1) focusing on reducing clear blockers to Rust adoption, particularly connected with productivity and (2) choosing one particular "driver" for adoption to invest in, namely high-scale servers. The balance between blocker/driver focus could be shifted—it might be the case that by providing more incentive to use Rust in a particular domain, people are willing to overlook some of its shortcomings.
Another possible blind spot is the conservative take on language expansion, particularly when it comes to productivity. For example, we could put much greater emphasis on "metaprogramming", and try to complete Plugins 2.0 in 2017. That kind of investment could pay dividends, since libraries can do amazing things with plugins that could draw people to Rust. But, as above, the overall strategy of reducing blockers assumes that what's most needed isn't more flashy examples of Rust's power, but rather more bread-and-butter work on reducing friction, improving tooling, and just making Rust easier to use across the board.
The roadmap is informed by the survey, systematic outreach, numerous direct conversations, and general strategic thinking. But there could certainly be blind spots and biases. It's worth double-checking our inputs.
Finally, there were several strong contenders for additional goals from the internals thread that we might consider. To be clear, these are not currently part of the proposed goals, but we may want to consider elevating them:
A goal explicitly for systematic expansion of commercial use; this proposal takes that as a kind of overarching idea for all of the goals.
A goal for Rust infrastructure, which came up several times. While this goal seems quite worthwhile in terms of paying dividends across the project, in terms of our current subteam makeup it's hard to see how to allocate resources toward this goal without dropping other important goals. We might consider forming a dedicated infrastructure team, or somehow organizing and growing our bandwidth in this area.
A goal for progress in areas like scientific computing, HPC.
After an exhaustive look at the thread, the remaining proposals are in one way or another covered somewhere in the discussion above.
The main unresolved question is how to break the given goals into more deliverable pieces of work, but that's a process that will happen after the overall roadmap is approved.
Are there other "areas of support" we should consider? Should any of these areas be elevated to a top-level goal (which would likely involve cutting back on some other goal)?
Should we consider some loose way of organizing "special interest groups" to focus on some of the priorities not part of the official goal set, but where greater coordination would be helpful? This was suggested multiple times.
Finally, there were several strong contenders for additional goals from the internals thread that we might consider, which are listed at the end of the goals section.