Before LibUnicode generated methods were weakly linked, we had a public
method (get_locale_currency_mapping) for retrieving currency mappings.
That method invoked one of several style-specific methods that only
existed in the generated UnicodeLocale.
One caveat of weakly linked functions is that every such function must
have a public declaration. The result is that each of those styled
methods are declared publicly, which makes the wrapper redundant
because it is just as easy to invoke the method for the desired style.
Intl.DisplayNames v2 adds "calendar" and "dateTimeField" types, as well
as a "languageDisplay" option for the "language" type. This just adds
these options to the constructor.
There are a few algorithms in TR-35 that need to replace digits before
returning any results to callers. For example, when formatting time zone
offsets, a string like "GMT+12:34" must have its digits replaced with
the default numbering system for the desired locale.
This is a normative change to the Intl spec:
20e5c26
Note that this doesn't actually affect us. Its purpose is to provide the
hour-cycle to BestFitFormatMatcher. This AO is implementation defined,
and ours just invokes BasicFormatMatcher, which doesn't use this field.
We could now have LibUnicode generate this field and use it to find a
better format pattern, though.
Instead of using plain objects as Iterator records, causes confusion
about the object itself actually being its [[Iterator]] slot, and
requires non-standard type conversion shenanigans fpr the [[NextValue]]
and [[Done]] internal slots, implement a proper Iterator record struct
and use it throughout.
Also annotate the remaining Iterator AOs with spec comments while we're
here.
This is a normative change in the Intl spec:
f0f66cf
There are two main changes here:
1. Converting BigInt/Number objects to mathematical values.
2. A change in how ToRawPrecision computes its exponent and significant
digits.
For (1), we do not yet support BigInt number formatting, thus already
have coerced Number objects to a double. When BigInt is supported, the
number passed into these methods will likely still be a Value, thus can
be coereced then.
For (2), our implementation already returns the expected edge-case
results pointed out on the spec PR.
This is a normative change in the Intl spec:
f0f66cf
Our implementation is unaffected by this change. LibUnicode pre-computes
positive, negative, and signless format patterns, so we already format
negative infinity correctly. Also, the CLDR does not contain specific
locale-dependent strings for negative infinity anyways.
The general idea when ENABLE_UNICODE_DATABASE_DOWNLOAD is OFF has been
that the Intl APIs will provide obviously incorrect results, but should
not crash. This regressed a bit with NumberFormat and DateTimeFormat.
There are 443 number system objects generated, each of which held an
array of number system symbols. Of those 443 arrays, only 39 are unique.
To uniquely store these, this change moves the generated NumericSymbol
enumeration to the public LibUnicode/NumberFormat.h header with a pre-
defined set of symbols that we need. This is to ensure the generated,
unique arrays are created in a known order with known symbols. While it
is unfortunate to no longer discover these symbols at generation time,
it does allow us to ignore unwanted symbols and perform less string-to-
enumeration conversions at lookup time.
For the test cases changed here, we now recognize "morning2" and
"afternoon2" from the CLDR, so the expected results now match the specs
and other engines.
In the CLDR, there aren't "night" values, there are "night1" & "night2"
values. This is for locales which use a different name for nighttime
depending on the hour. For example, the ja locale uses "夜" between the
hours of 19:00 and 23:00, and "夜中" between the hours of 23:00 and
04:00. Our CLDR parser is currently ignoring "night2", so this rename
is to prepare for that.
We could probably come up with better names, but in the end, the API in
LibUnicode will be such that outside callers won't even see Night1, etc.
In a future commit, the "part" view returned from FormatDateTimePattern
may be a view into a string that goes out of scope. Ensure the AO only
returns valid views. A similar approach is used in Intl.NumberFormat.
There are a few FIXMEs that will need to be addressed, but this
implements most of the prototype method. The FIXMEs are mostly related
to range formatting, which has been entirely ignored so far. But other
than that, the following will need to be addressed:
* Determining flexible day periods must be made locale-aware.
* DST will need to be determined and acted upon.
* Time zones other than UTC and calendars other than Gregorian are
ignored.
* Some of our results differ from other engines as they have some
format patterns we do not. For example, they seem to have a lonely
{dayPeriod} pattern, whereas our closest pattern is
"{hour} {dayPeriod}".
Unlike the locale, the data locale has Unicode locale extensions removed
(e.g. the data locale for "en-US-u-ca-gregory" is just "en-US"). Cache
the data locale for LibUnicode lookups during formatting.
This was an oversight in e42d954743.
These fields should always follow the locale preference in the CLDR.
Overriding these fields would permit formats like "h:mm:ss" to result in
strings like "1:2:3" instead of "1:02:03".
TR-35's Matching Skeleton algorithm dictates how user requests including
fractional second digits should be handled when the CLDR format pattern
does not include that field. When the format pattern contains {second},
but does not contain {fractionalSecondDigits}, generate a second pattern
which appends "{decimal}{fractionalSecondDigits}" to the {second} field.
ECMA-402 doesn't explicitly handle a note in the TR-35 spec related to
expanding field lengths based on user-provided options. Instead, it
assumes the "implementation defined" locale data includes the possible
values.
LibUnicode does not generate every possible combination of field lengths
in its implementation of TR-35's "Missing Skeleton Fields", because the
number of generated patterns would grow out of control. Instead, it's
much simpler to handle this difference at runtime.
Other implementations unconditionally initialize [[pattern12]] from
[[pattern]] regardless of whether [[pattern]] has an hour pattern of h11
or h12. LibUnicode does not do this. So when InitializeDateTimeFormat
defaults the hour cycle to the locale's preferred hour cycle, if the
best format didn't have an equivalent hour pattern, [[pattern12]] will
be empty.
This adds plumbing for the Intl.DateTimeFormat object, constructor, and
prototype.
Note that unlike other Intl objects, the Intl.DateTimeFormat object has
a LibUnicode structure as a base. This is to prevent wild amounts of
code duplication between LibUnicode, Intl.DateTimeFormat, and other
not-yet-defined Intl structures, because there's 12 fields shared
between them.
Currently, we generate separate data files for locale and number format
related tables/methods, but provide public accessors for all of the data
in one Locale.h file. Rather than continuing this trend for date-time,
relative time, etc. formatting, it's a bit easier to reason about if the
public accessors are also in separate files.
Allocating a Vector for each of these invocations is a bit silly when
the values are basically all compile-time arrays. This AO is used even
more heavily by Intl.DateTimeFormat, so change it to accept a Span to
reduce its cost.
This also adds an overload to accept a fixed-size C-array so callers do
not have to be prefixed with AK::Array, i.e. this:
get_option(..., AK::Array { "a"sv, "b"sv }, ...);
Reduces to:
get_option(..., { "a"sv, "b"sv }, ...);
(Which is how all call sites were already written to construct a Vector
in place).
This wasn't the case for compact patterns, but unit patterns can contain
multiple (up to 2, really) identifiers that must each be recognized by
LibJS.
Each generated NumberFormat object now stores an array of identifiers
parsed. The format pattern itself is encoded with the index into this
array for that identifier, e.g. the compact format string "0K" will
become "{number}{compactIdentifier:0}".
This field is currently used to store the StringView into the compact
name/symbol in the format string. Units will need to store a similar
field, so rename the field to be more generic, and extract the parser
for it.
Instead of currency pattern lookups within select_currency_unit_pattern,
rename the method to select_pattern_with_plurality and accept any list
of patterns. This method will be needed for units.
Finding the best number format to use for compact notation involves
creating a Vector of all compact formats for the locale and looking for
the one that best matches the number's magnitude. ECMA-402 wants this
number format to be found multiple times, so cache the result for future
use.
The compact scale of each formatting rule was precomputed in commit:
be69eae651
Using the formula: compact scale = magnitude - pattern scale
This computation was off-by-one.
For example, consider the format key "10000-count-one", which maps to
"00 thousand" in en-US. What we are really after is the exponent that
best represents the string "thousand" for values greater than 10000
and less than 100000 (the next format key). We were previously doing:
log10(10000) - "00 thousand".count("0") = 2
Which clearly isn't what we want. Instead, if we do:
log10(10000) + 1 - "00 thousand".count("0") = 3
We get the correct exponent for each format key for each locale.
This commit also renames the generated variable from "compact_scale" to
"exponent" to match the terminology used in ECMA-402.
