Feeding the rule "NFD; [:Nonspacing Mark:] Remove; NFC"
into the ICU Transliterator demo, the character Ø
(\u00d8
== LATIN CAPITAL LETTER O WITH STROKE
) remains as-is (i.e. the STROKE is not stripped).
In the list of non-marking spaces (Category Mn
), I cannot find anything named COMBINING DIAGONAL STROKE
akin to the COMBINING SHORT STROKE OVERLAY
(\u0335
) or COMBINING LONG STROKE OVERLAY
(\u0336
).
However, I do find COMBINING SHORT SOLIDUS OVERLAY
(\u0337
) and COMBINING LONG SOLIDUS OVERLAY
(\u0338
). They appear similar, but render as much thicker lines in my browser when combined with o
and O
.
The Unicode data I accessed for \u00d8
does not provide a decomposition for that character.
At the same time, the ICU Collator Demo will collate each of ø
, o
, Ø
, O
, o\u0337
and O\u0338
to the same code point using a Primary (Level = 1 = Base Letter) Collator.
Does this mean that the locale of Collator used in the Demo has been set up to identify the base character in a way where the Unicode spec is silent?
If so, do I need to a 开发者_Python百科custom Rule Based Transliterator if I want to strip the STROKE from LATIN [CAPITAL, SMALL] LETTER *
characters on transliteration?
See the following. The Latin-ASCII transliterator went into ICU 4.6. As you noted, the collation demo uses UCA / CLDR tailorings which have O versus slashed-O as base letter differences, this is not the same question as whether there's a decomposition. "w" doesn't decompose into "v + v" either. The decompositions have to do with whether there were existing encodings which represent characters in two different ways.
Java library for text normalization
UTF-8 to ASCII using ICU Library
http://unicode.org/repos/cldr/trunk/common/transforms/Latin-ASCII.xml
Yes. For some reason, the letter Ø
does not have a decomposition, so you have to handle it manually.
This transform along with replaceAll
works even for removing the Ø and other characters.
String id = "Accents-Any;NFD;[:Nonspacing Mark:] Remove; NFC";
System.out.println(latin.replaceAll("[^\\w]",""));
精彩评论