mirror of
https://github.com/RGBCube/serenity
synced 2025-07-25 15:47:44 +00:00
AK: Do not VERIFY on invalid code point bytes in UTF8View
The previous behavior was to always VERIFY that the UTF-8 bytes were valid when iterating over the code points of an UTF8View. This change makes it so we instead output the 0xFFFD 'REPLACEMENT CHARACTER' code point when encountering invalid bytes, and keep iterating the view after skipping one byte. Leaving the decision to the consumer would break symmetry with the UTF32View API, which would in turn require heavy refactoring and/or code duplication in generic code such as the one found in Gfx::Painter and the Shell. To make it easier for the consumers to detect the original bytes, we provide a new method on the iterator that returns a Span over the data that has been decoded. This method is immediately used in the TextNode::compute_text_for_rendering method, which previously did this in a ad-hoc waay. This also add tests for the new behavior in TestUtf8.cpp, as well as reinforcements to the existing tests to check if the underlying bytes match up with their expected values.
This commit is contained in:
parent
32ee195d62
commit
e01f1c949f
6 changed files with 135 additions and 20 deletions
|
@ -26,7 +26,7 @@ void EditEventHandler::handle_delete_character_after(const DOM::Position& cursor
|
|||
|
||||
auto& node = *static_cast<DOM::Text*>(const_cast<DOM::Node*>(cursor_position.node()));
|
||||
auto& text = node.data();
|
||||
auto code_point_length = Utf8View(text).iterator_at_byte_offset(cursor_position.offset()).code_point_length_in_bytes();
|
||||
auto code_point_length = Utf8View(text).iterator_at_byte_offset(cursor_position.offset()).underlying_code_point_length_in_bytes();
|
||||
|
||||
StringBuilder builder;
|
||||
builder.append(text.substring_view(0, cursor_position.offset()));
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue