RGBCube/serenity
1
Fork 0
mirror of https://github.com/RGBCube/serenity synced 2025-05-31 08:18:11 +00:00
Code Issues Activity Actions

LibUnicode: Substitute Unicode locale aliases during canonicalization

Browse source 
Unicode TR35 defines how locale subtag aliases should be emplaced when
converting a locale to canonical form. For most subtags, it is a simple
substitution. Language subtags depend on context; for example, the
language "sh" should become "sr-Latn", but if the original locale has a
script subtag already ("sh-Cyrl"), then only the language subtag of the
alias should be taken ("sr-Latn").

To facilitate this, we now make two passes when canonicalizing a locale.
In the first pass, we convert the LocaleID structure to canonical syntax
(where the conversions all happen in-place). In the second pass, we form
the canonical string based on the canonical syntax.
This commit is contained in:
Timothy Flynn 2021-08-30 15:34:31 -04:00 • committed by Linus Groh
parent 9b118f1f06
commit 556374a904
2 changed files with 178 additions and 51 deletions
Download patch file Download diff file Expand all files Collapse all files

36
Tests/LibUnicode/TestUnicodeLocale.cpp
View file

@ -334,4 +334,40 @@ TEST_CASE(canonicalize_unicode_locale_id)
test("EN-Z-BBB-U-AA-T-EN-0-AAA"sv, "en-0-aaa-t-en-u-aa-z-bbb"sv);
test("en-z-bbb-u-aa-t-en-0-aaa-x-ccc"sv, "en-0-aaa-t-en-u-aa-z-bbb-x-ccc"sv);
test("EN-Z-BBB-U-AA-T-EN-0-AAA-X-CCC"sv, "en-0-aaa-t-en-u-aa-z-bbb-x-ccc"sv);
// Language subtag aliases.
test("sh"sv, "sr-Latn"sv);
test("SH"sv, "sr-Latn"sv);
test("sh-cyrl"sv, "sr-Cyrl"sv);
test("SH-CYRL"sv, "sr-Cyrl"sv);
test("cnr"sv, "sr-ME"sv);
test("CNR"sv, "sr-ME"sv);
test("cnr-ba"sv, "sr-BA"sv);
test("CNR-BA"sv, "sr-BA"sv);
// Territory subtag aliases.
test("ru-su"sv, "ru-RU"sv);
test("RU-SU"sv, "ru-RU"sv);
test("ru-810"sv, "ru-RU"sv);
test("RU-810"sv, "ru-RU"sv);
test("en-su"sv, "en-RU"sv);
test("EN-SU"sv, "en-RU"sv);
test("en-810"sv, "en-RU"sv);
test("EN-810"sv, "en-RU"sv);
// Script subtag aliases.
test("en-qaai"sv, "en-Zinh"sv);
test("EN-QAAI"sv, "en-Zinh"sv);
// Variant subtag aliases.
test("en-polytoni"sv, "en-polyton"sv);
test("EN-POLYTONI"sv, "en-polyton"sv);
// Subdivision subtag aliases.
test("en-u-sd-cn11"sv, "en-u-sd-cnbj"sv);
test("EN-U-SD-CN11"sv, "en-u-sd-cnbj"sv);
test("en-u-rg-cn12"sv, "en-u-rg-cntj"sv);
test("EN-U-RG-CN12"sv, "en-u-rg-cntj"sv);
test("en-u-aa-cn11"sv, "en-u-aa-cn11"sv);
test("EN-U-AA-CN11"sv, "en-u-aa-cn11"sv);
}