Cargo in 2019

I recently published a post detailing a vision for the next few years (hah! Not so recently now, this took a lot longer than expected). Here I'll get into more detail about 2019.

There will be three high-priority focus areas: cross-compilation, plugins, and compile times. There are also two buckets of small items: addressing technical debt, and finishing work which is almost complete. See the multiyear post for why I think the timing is right for these items within the bigger picture. These are ambitious goals! We probably don't have the resources to get all this done, but I think it is possible, especially if people are keen to help; if you are, let us know!

Cross-compilation

Cross compilation includes a number of different scenarios which are well-suited to Rust development, and used by a large number of users (multi-platform development, embedded and mobile development, and WASM). There are already several tools available to help users (e.g., Xargo, cross, Rustup, and WASM-Pack).

We believe that by better supporting these tools and bringing some of their features into Cargo, we can bring many users and projects into a more standard, lower-friction Cargo workflow. Since there has been much iteration on these tools already, there are some low risk initial directions. However, this is a large and hard to scope area, so there is a risk we take on too much or end up with an ill-suited design. We plan to pursue an iterative approach and primarily work on features that have already had a proven impact.

Plugins

As we laid out in the long-term post, plugins (aka custom commands, aka tasks, aka workflows) are a key strategy to bring new and flexible workflows to users sooner rather than later, and let us experiment with features which might one day live in Cargo. Example features which might be provided as plugins are cargo add, cargo upgrade, cargo vendor, or cargo clean. By making plugins more discoverable (which will require collaboration with the crates.io team), and providing library crates for plugin authors which abstract parts of Cargo's API (for example, processing Cargo.tomls, improving the output of cargo metadata, or querying the project structure or crate graph), we hope to make plugins more powerful, reliable, and usable. If we do this well, then we can bring a lot of value to Cargo users and buy time to properly design some complex features.

There are risks with this strategy: we might end up with a low-quality or fragmented ecosystem, or users might not be satisfied with plugins (vs native sub-commands). Mitigating that is probably a matter of keeping an eye on how the plugin ecosystem evolves, and trying to keep the cost of development low (e.g., by starting with low risk, low cost libraries before investing more development effort in difficult libraries). There are also some open questions about how to deal with versioning and other technical issues.

Compile times

This area is in service of the Rust project's goals, rather than specifically working toward Cargo's long term goals. Compile times are one of the big negatives about Rust which we hear about again and again in our surveys and anecdotally.

Although this is primarily a compiler issue, there are some things Cargo could help with. One promising idea is to support the distribution of compiled crates, as well as source code. That would allow users to build their main crate without waiting for dependencies to build first. That could make a massive impact on initial build times for crates (and whenever a user changes or updates their toolchain).

Longer term, this could be the foundation for distributing and using incremental compiler artifacts, or some kind of distributed or shared build facility in Cargo.

Another potential avenue is pipelining builds. Consider two crates with a dependency relationship: currently we have to completely finish building the first crate before we can start to build the second. However, in theory Cargo could start building the second as soon as the first's metadata was generated, before running LLVM (and potentially borrow checking). I think using compiled crates will be a bigger win and lower risk, so this idea is definitely a nice-to-have for this year rather than a top priority; it's also possible that the ongoing work on incrementalising the compiler will make this work unnecessary or impossible.

Technical debt

All projects should spend time paying off technical debt. Technical debt slows down development, makes bugs more likely, and puts off potential contributors. A different kind of technical debt is unfinished or incomplete work - here we pay the maintenance cost for the code, but don't bring a benefit to users.

Particularly at this point in the edition cycle, we think it is a good idea to tackle some of Cargo's technical debt.

Some specific items:

  • Cargo features
  • documentation
  • stabilisation backlog
  • refactoring and architectural issues
  • Cargo's use of disk space
  • how Cargo and crates.io verify crates for publication
  • unit tests

Features

Cargo features are almost an item to themselves. Although they are nominally finished, they are one of the roughest edges in Cargo. Issues include controlling dependency granularity (e.g., build vs regular dependencies; this is particularly painful for no_std development), workspace-wide feature selection, edge cases with platform- or target-specific dependencies, constraints between features, features for (transitive) dependencies, and more.

The Cargo team will work towards a long-term plan for Cargo features and conditional resolution. With an understanding of how all the different issues fit together, it should be easier to tackle individual pieces incrementally. There are significant challenges with backwards compatibility and implementation details. Once we have a long-term plan and better understanding of how the pieces interact, we should be able to choose some of the high priority parts to implement this year.

Almost complete work

There is a bunch of work which is almost complete, we should try to get it across the line. Even though these don't directly contribute to our long-term goals, they are good things to have and since they're almost done, they are low cost, high impact items. These include:

Non-goals

Some things that I don't think should be (or can be) prioritised (but might get done if somebody is very keen and volunteers to do the work, and there is bandwidth to review). There's obviously a lot that could be on this list (Cargo shouldn't bake you a cake, for example). Some of the items that I see mentioned a lot include:

  • toolchain and tool versioning (we need to think about the design of this feature a lot more)
  • sysroot handling (a quality of life improvement for the 2018 edition, but no rush to implement before the next edition)
  • replace Rustup (I would love for Cargo to absorb all of Rustup and reduce the number of Rust's tools, but it would be a lot of work for relatively little gain)
  • a plumbing and porcelain architecture (this is high cost and high risk. We believe that extracting parts of Cargo into their own library crates is lower risk and less effort)