Pages such as https://html5test.com are testing all sorts of weird,
incomplete, and wrong HTML but can be useful or at least interesting for
development - let's try to avoid crashing the process.
This is needed to access the 'adjusted current node' in the 'Markup
declaration open state'. We don't want to create a full parser for
something like syntax highlighting, so it's optional (null) by default.
Emitting tokens on EOF caused an infinite loop, freezing the app, which
could be a bit annoying when writing an HTML comment at the end of
the file in Text Editor. :^)
Commit b193351a99 caused the HTML comments to flash when changing
the text cursor. Also, when double-clicking on a comment, the selection
started from the beginning of the file instead.
The following message was displaying when `TOKENIZER_TRACE_DEBUG`
was enabled:
(Tokenizer::nth_last_position) Invalid position requested: 4th-last
of 4. Returning (0-0).
Changing the `nth_last_position` to 3 fixes this. I'm guessing that's
because the parser is at that moment on the second hyphen of the `<!--`
string, so it has to go back only by three characters.
The difference should be between m_utf8_iterator and the
the new position, if m_prev_utf8_iterator is used one fewer
source position is popped than required.
This issue was not apparent on most pages since restore_to
used for tokens such <!doctype> that are normally
followed by a newline that resets the column to zero,
but it can be seen on pages with minified HTML.
The environment settings object is effectively the context a piece of
script is running under, for example, it contains the origin,
responsible document, realm, global object and event loop for the
current context. This effectively replaces ScriptExecutionContext, but
it cannot be removed in this commit as EventTarget still depends on it.
https://html.spec.whatwg.org/multipage/webappapis.html#environment-settings-object
Note our Attribute class is what the spec refers to as just "Attr". The
main differences between the existing implementation and the spec are
just that the spec defines more fields.
Attributes can contain namespace URIs and prefixes. However, note that
these are not parsed in HTML documents unless the document content-type
is XML. So for now, these are initialized to null. Web pages are able to
set the namespace via JavaScript (setAttributeNS), so these fields may
be filled in when the corresponding APIs are implemented.
The main change to be aware of is that an attribute is a node. This has
implications on how attributes are stored in the Element class. Nodes
are non-copyable and non-movable because these constructors are deleted
by the EventTarget base class. This means attributes cannot be stored in
a Vector or HashMap as these containers assume copyability / movability.
So for now, the Vector holding attributes is changed to hold RefPtrs to
attributes instead. This might change when attribute storage is
implemented according to the spec (by way of NamedNodeMap).
This particularly implements these two points:
- "If the adjusted current node is an HTML integration point and the
token is a start tag"
- "If the adjusted current node is an HTML integration point and the
token is a character token"
This also adds spec comments to the tree constructor.
We now fire "pageshow" events at the appropriate time during document
loading (done by the parser.)
Note that there are no corresponding "pagehide" events yet.
This will be used to determine whether "pageshow" and "pagehide" events
are appropriate. We won't actually make use of it until we implement
more of history traversal and document unloading.
The only difference from what we were already doing is that setting the
same ready state twice no longer fires a "readystatechange" event.
I don't think that could happen in practice though.
We will now spin in "the end" until there are no more "things delaying
the load event". Of course, nothing actually uses this yet, and there
are a lot of things that need to.
Instead of firing up a network request and synchronously blocking for it
to finish via a nested event loop, we now start an asynchronous request
when encountering <script src>.
Once the script load finishes (or fails), it gets executed at one of the
synchronization points in the HTML parser.
This solves some long-standing issues with random unexpected events
getting dispatched in the middle of parsing.
The previous implementation was about a half implementation and was
tied to Element::innerHTML. This separates it and puts it into
HTMLDocumentParser, as this is in the parsing section of the spec.
This provides a near finished HTML fragment serialisation algorithm,
bar namespaces in attributes and the `is` value.
This namespace will be used for all interfaces defined in the URL
specification, like URL and URLSearchParams.
This has the unfortunate side-effect of requiring us to use the fully
qualified AK::URL name whenever we want to refer to the AK class, so
this commit also fixes all such references.
Call HTML::EventLoop::spin_until() from the HTML parser when deciding
whether we can run a script yet.
Note that spin_until() actually doesn't do any work yet.
This completely changes how HTMLTokens store their data. Previously,
space was allocated for all token types separately. Now, the HTMLToken's
data is stored in just a String, two booleans and a Variant.
This change reduces sizeof(HTMLToken) from 68 to 32. Also, this reduces
raw tokenization time by around 20 to 50 percent, depending on the page.
Full document parsing time (with HTMLDocumentParser, on a local HTML
page without any dependency files) is reduced by between 4 and 20
percent, depending on the page.
Since tokenizing HTML pages can easily generated 50'000 tokens and more,
the storage has been designed in a way that avoids heap allocations
where possible, while trying to reduce the size of the tokens. The only
tokens which need to allocate on the heap are thus DOCTYPE tokens (max.
1 per document), and tag tokens (but only if they have attributes). This
way, only around 5 percent of all tokens generated need to allocate on
the heap (except for StringImpl allocations).
Since all interaction with the HTMLToken class now happens over getters
and setters, there is no more need for HTMLTokenizer and
HTMLDocumentParser to have direct access to the members.
This is in preparation for an upcoming storage change of HTMLToken. In
contrast to the other token types, the accessor can hand out a mutable
reference to allow users to change parts of the DoctypeData easily.
Previously, HTMLToken would expose the Vector<Attribute> directly to
its users. In preparation for a future change, all users now use
implementation-agnostic APIs which do not expose the Vector directly.
