7/29/2026 at 8:34:45 AM
Ugh, why are BOMs even still a thing in the 21st century? E.g. when will Windows finally arrive in the late 1990s and switch to UTF-8 for everything?The UTF-8 BOM is especially bizarre because UTF-8 is completely endian-agnostic (so this "UTF-8 Byte Order Mark" is at most an indicator that this file is UTF-8 encoded, but guess what? Outside the Windows bubble, all text files are UTF-8 anyway).
by flohofwoe
7/29/2026 at 9:58:05 AM
Windows bought into Unicode hard before Unicode expanded beyond 16 bits. The consequence is that its native strings are theoretically UTF-16, but often behave like UCS-2, allowing random halves of surrogate pairs. Windows APIs reflect that, Windows filenames reflect that, and translation into UTF-8 has to reflect that -- somehow. It's not great.Personally I'd like a world where Unicode gave up all the compromises that come with supporting UTF-16. But presumably burning all your early adopters is not a winning strategy.
by gdwatson
7/29/2026 at 12:38:29 PM
> But presumably burning all your early adopters is not a winning strategy.This topic is so old that most early adaptors probably have retired by now ;)
by flohofwoe
7/29/2026 at 5:13:30 PM
Fair. If only they'd retired those APIs, too.I remember when Windows gained experimental support for a UTF-8 locale usable with its 8-bit APIs. I used to run it at work; it had some issues at first, but most got shaken out pretty quickly. I don't know its current state, but that's probably the best route to UTF-8 on Windows if MS would lean into it.
by gdwatson