libs (platform)
line_endings
Change all functions dealing with reading "lines" to treat both '\n' and '\r\n' as a valid line-ending.
The current behavior of these functions is to treat only '\n' as line-ending. This is surprising for programmers experienced in other languages. Many languages open files in a "text-mode" per default, which means when they iterate over the lines, they don't have to worry about the two kinds of line-endings. Such programmers will be surprised to learn that they have to take care of such details themselves in Rust. Some may not even have heard of the distinction between two styles of line-endings.
The current design also violates the "do what I mean" principle. Both '\r\n' and '\n' are widely used as line-separators. By talking about the concept of "lines", it is clear that the current file (or buffer, really) is considered to be in text format. It is thus very reasonable to expect "lines" to apply to both kinds of encoding lines in binary format.
In particular, if the crate is developed on Linux or Mac, the programmer will probably have most of his input encoded with only '\n' for the line-endings. He may use the functions talking about "lines", and they will work all right. It is only when someone runs this crate on input that contains '\r\n' that the bug will be uncovered. The editor has personally run into this issue when reading line-by-line from stdin, with the program suddenly failing on Windows.
The following functions will have to be changed: BufRead::lines
and
str::lines
. They both should treat '\r\n' as marking the end of a line. This
can be implemented, for example, by first splitting at '\n' like now and then
removing a trailing '\r' right before returning data to the caller.
Furthermore, str::lines_any
(the only function currently dealing with both
kinds of line-endings) is deprecated, as it is then functionally equivalent with
str::lines
.
This is a semantics-breaking change, changing the behavior of released, stable
API. However, as argued above, the new behavior is much less surprising than the
old one - so one could consider this fixing a bug in the original
implementation. There are alternatives available for the case that one really
wants to split at '\n' only, namely BufRead::split
and str::split
. However,
BufRead:split
does not iterate over String
, but rather over Vec<u8>
, so
users have to insert an additional explicit call to String::from_utf8
.
There's the obvious alternative of not doing anything. This leaves a gap in the features Rust provides to deal with text files, making it hard to treat both kinds of line-endings uniformly.
The second alternative is to add BufRead::lines_any
which works similar to
str::lines_any
in that it deals with both '\n' and '\r\n'. This provides all
the necessary functionality, but it still leaves people with the need to choose
one of the two functions - and potentially choosing the wrong one. In
particular, the functions with the shorter, nicer name (the existing ones) will
almost always not be the right choice.
None I can think of.