Björn JACKE <bjacke@xxxxxxxxx> writes: > It turns out though that the case insensitive feature is not usable because it > does not match the case mapping tables that other operating systems use. More > specifically, the german letter "ß" is treated as a case equivanten of "ss". > > There is an equivalent of "ß" and "ss in some other scopes, also AD LDAP treats > them as an equivante. For systems that requires "lossless" case conversion > however should not treat ß and ss as equivalent. This is also why a filesystem > should never ever do that Well, filesystems should never ever have filename encoding. Once they do, we have to make semantics decisions and they are all apparently stupid to someone. And any kind of Casefolding is an inherently lossy operation in this sense, as multiple byte sequences will map to the same file. The big problem is that each of the big OS vendors chose specific semantics of what to casefold. APFS does NFD + full casefolding[1], right? except for "some code-points". I'm not sure what they do with ß, tbh. I could never find any documentation on the specific code-points they add/ignore. In ext4, we decided to have no exceptions. Just do plain NFD + CF. That means we do C+F from the table below: https://www.unicode.org/Public/12.1.0/ucd/CaseFolding.txt Which includes ß->SS. We could argue forever whether that doesn't make sense for language X, such as German. I'm not a German speaker but friends said it would be common to see straße uppercased to STRASSE there, even though the 2017 agreement abolished it in favor of ẞ. So what is the right way? My point is we can't rely fully on languages to argue the right semantics. There are no right semantics. And Languages are also alive and changing. There are many other examples where full casefold will look stupid; for instance, one would argue we should also translate the T column (i.e non-Turkish languages). We can produce all sorts of stupid examples with combining characters in Portuguese/Spanish too. Linux is not broken beyond the fact the whole idea is broken. These are just the semantics we agreed were slightly less insane back in 2017 (considering we don't want to have locales). And, apart from the ignorable code points issue, I still think our implementation is relatively sane. > Since 2017 there is a well-defined uppercase version of the codepoint (U+00DF) > of the "ß" letter in Unicode: U+1E9E, this could eventually be used but I > haven't seen any filesystem using that so far. This would be a possible and > lossless case equivalent, but well, that's actually another thing to > discuss. > > The important point is to _not_ use the ß/ss case equicalent. The casefold > feature is mainly useless otherwise. It is not useless. Android and Wine emulators have been using it just fine for years. We also cannot break compatibility for them. > Can this be changed without causing too much hassle? We attempted to do a much smaller change recently in commit 5c26d2f1d3f5, because we assumed no one would be trying to create files with silly stuff like ZWSP (U+200B). Turns out there is a reasonable use-case for that with Variation Selectors, and we had to revert it. So we need to be very careful with any changes here, so people don't lose access to their files on a kernel update. Even with that, more casefolding flavor will cause all sorts of compatibility issues when moving data across volumes, so I'd be very wary of having more than one flavor. What are the exact requirements for samba? Do you only fold the C column? Do you need stuff like compatibility normalization? [1] https://developer.apple.com/support/downloads/Apple-File-System-Reference.pdf -- Gabriel Krisman Bertazi
