8
Github Suggest character encoding is incorrect when encountering random null byt...
source link: https://github.com/rust-lang/rust/pull/81856
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
This adds a note whenever null bytes are seen at the start of a token unexpectedly, since those tend to come from UTF-16 encoded files without a BOM (if a UTF-16 BOM appears it won't be valid UTF-8, but if there is no BOM it be both valid UTF-16 and valid but garbled UTF-8). This approach was suggested in #73979 (comment).
Closes #73979.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK