RGBCube/serenity
1
Fork 0
mirror of https://github.com/RGBCube/serenity synced 2025-07-25 13:57:35 +00:00
Code Issues Activity Actions

LibCrypto: Add a rounding mode to UnsignedBigInteger::to_double

Browse source 
This allows using different options for rounding, like IEEE
roundTiesToEven, which is the mode that JS requires.

Also fix that the last word read from the bigint for the mantissa could
be shifted incorrectly leading to incorrect results.
This commit is contained in:
davidot 2022-08-25 17:56:34 +02:00 • committed by Linus Groh
parent 8ba6e96d05
commit 77d71a5ffd
5 changed files with 144 additions and 24 deletions
Download patch file Download diff file Expand all files Collapse all files

4
Userland/Libraries/LibCrypto/BigInt/SignedBigInteger.cpp
View file

@ -65,9 +65,9 @@ u64 SignedBigInteger::to_u64() const
return ~(unsigned_value - 1); // equivalent to `-unsigned_value`, but doesn't trigger UBSAN
}
double SignedBigInteger::to_double() const
double SignedBigInteger::to_double(UnsignedBigInteger::RoundingMode rounding_mode) const
{
double unsigned_value = m_unsigned_data.to_double();
double unsigned_value = m_unsigned_data.to_double(rounding_mode);
if (!m_sign)
return unsigned_value;