1
Fork 0
mirror of https://github.com/RGBCube/serenity synced 2025-07-27 09:37:34 +00:00

LibJS: Improve JS::modulo precision for large floating values

JS::modulo was yielding a result of '0' for the input:
```
modulo(1., 18446744073709551616.)
```

Instead of the expected '1'.

As far as I can tell the reason for this is that the repeated calls to
fmod is losing precision in the calculation, leading to the wrong
result. Fix this by only calling fmod once, and preserving the negative
value behaviour by an 'if' check.

Without this, the LibWeb text test:
`/Streams/ReadableByteStream-enqueue-respond.html`

Would hang forever after using this function in the IDL conversion of a
u64 in ConvertToInt.

This should also be more efficient :^)
This commit is contained in:
Shannon Booth 2024-01-02 19:24:36 +13:00 committed by Andreas Kling
parent 986abe7047
commit f589bedb0d

View file

@ -300,7 +300,8 @@ auto modulo(T x, U y)
if constexpr (IsFloatingPoint<T> || IsFloatingPoint<U>) {
if constexpr (IsFloatingPoint<U>)
VERIFY(isfinite(y));
return fmod(fmod(x, y) + y, y);
auto r = fmod(x, y);
return r < 0 ? r + y : r;
} else {
return ((x % y) + y) % y;
}