Instead of returning HeapBlock memory to the kernel (or a non-type
specific shared cache), we now keep a BlockAllocator per CellAllocator
and implement "deallocation" by basically informing the kernel that we
don't need the physical memory right now.
This is done with MADV_FREE or MADV_DONTNEED if available, but for other
platforms (including SerenityOS) we munmap and then re-mmap the memory
to achieve the same effect. It's definitely clunky, so I've added a
FIXME about implementing the madvise options on SerenityOS too.
The important outcome of this change is that GC types that use a
type-specific allocator become immune to use-after-free type confusion
attacks, since their virtual addresses will only ever be re-used for
the same exact type again and again.
Fixes#22274
Due to a `requires` mistake, we were always using the fallback
size-based cell allocators.
Also, now that we start using them, make them NeverDestroyed so
we don't try to deallocate them on program exit.
Instead of allocating these in a mixture of ways, we now always put
them on the malloc heap, and keep an intrusive linked list of them
that we can iterate for GC marking purposes.
This patch adds two macros to declare per-type allocators:
- JS_DECLARE_ALLOCATOR(TypeName)
- JS_DEFINE_ALLOCATOR(TypeName)
When used, they add a type-specific CellAllocator that the Heap will
delegate allocation requests to.
The result of this is that GC objects of the same type always end up
within the same HeapBlock, drastically reducing the ability to perform
type confusion attacks.
It also improves HeapBlock utilization, since each block now has cells
sized exactly to the type used within that block. (Previously we only
had a handful of block sizes available, and most GC allocations ended
up with a large amount of slack in their tails.)
There is a small performance hit from this, but I'm sure we can make
up for it elsewhere.
Note that the old size-based allocators still exist, and we fall back
to them for any type that doesn't have its own CellAllocator.
The functions for registering and unregistering MarkedVector, Handle,
etc. were quite prominent in benchmark profiles.
4% speed-up on the entire Kraken benchmark :^)
(including: 7% speed-up on Kraken/imaging-gaussian-blur.js, the current
slowest subtest)
This change adds a check to discard pointers that are lower than the
minimum address of all allocated blocks or higher than the maximum
address of all blocks. By doing this we avoid executing plenty of set()
operations on the HashMap in the add_possible_value().
With this change gather_conservative_roots() run 10x times faster in
Speedometer React-Redux-TodoMVC test.
This change introduces HeapFunction, which is intended to be used as a
replacement for SafeFunction. The new type behaves like a regular
GC-allocated object, which means it needs to be visited from
visit_edges, and unlike SafeFunction, it does not create new roots for
captured parameters.
Co-Authored-By: Andreas Kling <kling@serenityos.org>
This change introduces a very basic GC graph dumper. The `dump_graph()`
function outputs JSON data that contains information about all nodes in
the graph, including their class types and edges.
Root nodes will have a property indicating their root type or source
location if the root is captured by a SafeFunction. It would be useful
to add source location for other types of roots in the future.
Output JSON dump have following format:
```json
"4908721208": {
"class_name": "Accessor",
"edges": [
"4909298232",
"4909297976"
]
},
"4907520440": {
"root": "SafeFunction Optional Optional.h:137",
"class_name": "Realm",
"edges": [
"4908269624",
"4924821560",
"4908409240",
"4908483960",
"4924527672"
]
},
"4908251320": {
"class_name": "CSSStyleRule",
"edges": [
"4908302648",
"4925101656",
"4908251192"
]
},
```
Stop worrying about tiny OOMs. Work towards #20449.
While going through these, I also changed the function signature in many
places where returning ThrowCompletionOr<T> is no longer necessary.
This patch triggers the collector when allocated memory doubles instead
of every 100k allocations. Which can almost half (reduce by ~48%) the
time spent on collection when loading google-maps.
This dynamic approach is inspired by some other GCs like Golang's and
Lua's and improves performance in memory heavy applications because
marking must visit old objects which will dominate the marking phase if
the GC is invoked too often.
This commit also improves the Octane Splay benchmark and almost
doubles it :^)
The Heap::uproot_cell() API was used to implement markAsGarbage() which
was used in 3 tests to forcibly destroy a value, even if it had
references on the stack or elsewhere.
This patch rewrites the 3 tests that used this mechanism to be
structured in a way that allows garbage collection to collect the values
as intended without hacks. And now that the uprooting mechanism is no
longer needed, it's uprooted as well.
This fixes 3 test-js tests in bytecode mode. :^)
Cell::heap() and Cell::vm() needed to access member functions from
HeapBlock, and wanted to be inline, so they were moved to VM.h.
That approach will no longer work with VM.h not being included in every
file (starting from the next commit), so this commit fixes that circular
import issue by introducing secondary base classes to host the
references to Heap and VM, respectively.
This is a similar strategy to what v8 does. Use the ASAN API function
__asan_addr_is_in_fake_stack to check any fake stack frames associated
with each stack address we scan. This fully allows running test-js -g
with the option detect_stack_use_after_return turned on.
Note that as of this commit, there aren't any such throwers, and the
call site in Heap::allocate will drop exceptions on the floor. This
commit only serves to change the declaration of the overrides, make sure
they return an empty value, and to propagate OOM errors frm their base
initialize invocations.
Doing things in the destructor of a GC-allocated object isn't always
safe, in case it involves accessing other GC-allocated objects.
If they were already swept by GC, we'd be poking into freed memory.
This patch adds a separate finalization pass where GC calls finalize()
on every unmarked cell that's about to be deleted.
It's safe to access other GC objects in finalize(), even if they're
also unmarked.
This is a monster patch that turns all EventTargets into GC-allocated
PlatformObjects. Their C++ wrapper classes are removed, and the LibJS
garbage collector is now responsible for their lifetimes.
There's a fair amount of hacks and band-aids in this patch, and we'll
have a lot of cleanup to do after this.
This is a continuation of the previous five commits.
A first big step into the direction of no longer having to pass a realm
(or currently, a global object) trough layers upon layers of AOs!
Unlike the create() APIs we can safely assume that this is only ever
called when a running execution context and therefore current realm
exists. If not, you can always manually allocate the Error and put it in
a Completion :^)
In the spec, throw exceptions implicitly use the current realm's
intrinsics as well: https://tc39.es/ecma262/#sec-throw-an-exception
This is a continuation of the previous three commits.
Now that create() receives the allocating realm, we can simply forward
that to allocate(), which accounts for the majority of these changes.
Additionally, we can get rid of the realm_from_global_object() in one
place, with one more remaining in VM::throw_completion().
This is a continuation of the previous commit.
Calling initialize() is the first thing that's done after allocating a
cell on the JS heap - and in the common case of allocating an object,
that's where properties are assigned and intrinsics occasionally
accessed.
Since those are supposed to live on the realm eventually, this is
another step into that direction.
This feature had bitrotted somewhat and would trigger errors because
PrimitiveStrings were "destroyed" but because of this mode they were not
removed from the string cache. Even fixing that case running test-js
with the options still failed in more places.
This abstracts a vector of Cell* with a strongly typed span() accessor
that gives you Span<T*> instead of Span<Cell*>.
It is intended to replace MarkedValueList in situations where you only
need to store pointers to Cell (or an even more specific type of Cell).
The API can definitely be improved, it's just the bare basics for now.
The way that transition avoidance (foo_without_transition) was
implemented led to shapes being unshareable and caused shape explosion
instead, precisely what we were trying to avoid.
This patch removes all the attempts to avoid transitioning shapes, and
instead *adds* transitions when changing an object's prototype.
This makes transitions flow naturally, and as a result we end up with
way fewer shape objects in real-world situations.
When we run out of big problems, we can get back to avoiding transitions
as an optimization, but for now, let's avoid ballooning our processes
with a unique shape for every object.
This patch ups the max number of heap allocations between each GC
from 10'000 to 100'000. This is still relatively aggressive but already
does a good job of cutting down on time spent in GC.
This patch adds a `-z` option to js and test-js. When run in this mode,
garbage cells are never actually destroyed. We instead keep them around
in a special zombie state.
This allows us to validate that zombies don't get marked in future GC
scans (since there were not supposed to be any more references!) :^)
Cells get notified when they become a zombie (via did_become_zombie())
and this is used by WeakContainer cells to deregister themselves from
the heap.
This should fix the flaky tests of test-js.
It also fixes the tests when running with the -g flag since the values
will not be garbage collected too soon.
- Replace the misleading abuse of the m_transitions_enabled flag for the
fast path without lookup with a new m_initialized boolean that's set
either by Heap::allocate() after calling the Object's initialize(), or
by the GlobalObject in its special initialize_global_object(). This
makes it work regardless of the shape's uniqueness.
- When we're adding a new property past the initialization phase,
there's no need to do a second metadata lookup to retrieve the storage
value offset - it's known to always be the shape's property count
minus one. Also, instead of doing manual storage resizing and
assignment via indexing, just use Vector::append().
- When we didn't add a new property but are overwriting an existing one,
the property count and therefore storage value offset doesn't change,
so we don't have to retrieve it either.
As a result, Object::set_shape() is now solely responsible for updating
the m_shape pointer and is not resizing storage anymore, so I moved it
into the header.
This patch adds a BlockAllocator to the GC heap where we now cache up to
64 HeapBlock-sized mmap's that get recycled when allocating HeapBlocks.
This improves test-js runtime performance by ~35%, pretty cool! :^)
