RGBCube/serenity
1
Fork 0
mirror of https://github.com/RGBCube/serenity synced 2025-07-27 12:17:44 +00:00
Code Issues Activity Actions

LibJS: Improve JS::modulo precision for large floating values

Browse source 
JS::modulo was yielding a result of '0' for the input:
```
modulo(1., 18446744073709551616.)
```

Instead of the expected '1'.

As far as I can tell the reason for this is that the repeated calls to
fmod is losing precision in the calculation, leading to the wrong
result. Fix this by only calling fmod once, and preserving the negative
value behaviour by an 'if' check.

Without this, the LibWeb text test:
`/Streams/ReadableByteStream-enqueue-respond.html`

Would hang forever after using this function in the IDL conversion of a
u64 in ConvertToInt.

This should also be more efficient :^)
This commit is contained in:
Shannon Booth 2024-01-02 19:24:36 +13:00 • committed by Andreas Kling
parent 986abe7047
commit f589bedb0d
1 changed files with 2 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

3
Userland/Libraries/LibJS/Runtime/AbstractOperations.h
View file

@ -300,7 +300,8 @@ auto modulo(T x, U y)
if constexpr (IsFloatingPoint<T> || IsFloatingPoint<U>) { if constexpr (IsFloatingPoint<T> || IsFloatingPoint<U>) {
if constexpr (IsFloatingPoint<U>) if constexpr (IsFloatingPoint<U>)
VERIFY(isfinite(y)); VERIFY(isfinite(y));
return fmod(fmod(x, y) + y, y); auto r = fmod(x, y);
return r < 0 ? r + y : r;
} else { } else {
return ((x % y) + y) % y; return ((x % y) + y) % y;
} }