This implements Intel's slicing-by-8 algorithm for CRC checksums (only
little endian CPUs for now, as I don't have a way to test big endian).
The original paper for this algorithm seems to have disappeared, but
Intel's source code is still available as a reference:
https://sourceforge.net/projects/slicing-by-8/
As well as other implementations for reference:
https://docs.rs/slice-by-8/latest/src/slice_by_8/algorithm.rs.html
Using the "enwik8" file as a test (100MB uncompressed, commonly used in
benchmarks: https://www.mattmahoney.net/dc/enwik8.zip), decompression
time decreases from:
4.89s to 3.52s on Serenity (cold)
1.72s to 1.32s on Serenity (warm)
1.06s to 0.92s on Linux
CRC32 table is generated at compile-time and put into a static
variable in the header file. This can be moved to be a function
instead of a class, be moved to the `.cpp` file` and generated as an
array instead of a class which only implements `operator[]`.
We had some inconsistencies before:
- Sometimes "The", sometimes "the"
- Sometimes trailing ".", sometimes no trailing "."
I picked the most common one (lowecase "the", trailing ".") and applied
it to all copyright headers.
By using the exact same string everywhere we can ensure nothing gets
missed during a global search (and replace), and that these
inconsistencies are not spread any further (as copyright headers are
commonly copied to new files).
SPDX License Identifiers are a more compact / standardized
way of representing file license information.
See: https://spdx.dev/resources/use/#identifiers
This was done with the `ambr` search and replace tool.
ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
