1
Fork 0
mirror of https://github.com/RGBCube/serenity synced 2025-07-25 09:57:35 +00:00

LibJS: Unify syntax highlighting

So far we have three different syntax highlighters for LibJS:

- js's Line::Editor stylization
- JS::MarkupGenerator
- GUI::JSSyntaxHighlighter

This not only caused repetition of most token types in each highlighter
but also a lot of inconsistency regarding the styling of certain tokens:

- JSSyntaxHighlighter was considering TokenType::Period to be an
  operator whereas MarkupGenerator categorized it as punctuation.
- MarkupGenerator was considering TokenType::{Break,Case,Continue,
  Default,Switch,With} control keywords whereas JSSyntaxHighlighter just
  disregarded them
- MarkupGenerator considered some future reserved keywords invalid and
  others not. JSSyntaxHighlighter and js disregarded most

Adding a new token type meant adding it to ENUMERATE_JS_TOKENS as well
as each individual highlighter's switch/case construct.

I added a TokenCategory enum, and each TokenType is now associated to a
certain category, which the syntax highlighters then can use for styling
rather than operating on the token type directly. This also makes
changing a token's category everywhere easier, should we need to do that
(e.g. I decided to make TokenType::{Period,QuestionMarkPeriod}
TokenCategory::Operator for now, but we might want to change them to
Punctuation.
This commit is contained in:
Linus Groh 2020-10-04 22:28:59 +01:00 committed by Andreas Kling
parent 6b55b007dd
commit e80217a746
5 changed files with 195 additions and 437 deletions

View file

@ -204,131 +204,28 @@ String MarkupGenerator::style_from_style_type(StyleType type)
MarkupGenerator::StyleType MarkupGenerator::style_type_for_token(Token token)
{
switch (token.type()) {
case TokenType::Enum:
case TokenType::Eof:
case TokenType::Implements:
case TokenType::Invalid:
case TokenType::Package:
case TokenType::Private:
case TokenType::Protected:
case TokenType::Public:
case TokenType::Static:
case TokenType::UnterminatedTemplateLiteral:
switch (token.category()) {
case TokenCategory::Invalid:
return StyleType::Invalid;
case TokenType::NumericLiteral:
case TokenType::BigIntLiteral:
case TokenCategory::Number:
return StyleType::Number;
case TokenType::StringLiteral:
case TokenType::TemplateLiteralStart:
case TokenType::TemplateLiteralEnd:
case TokenType::TemplateLiteralString:
case TokenType::RegexLiteral:
case TokenType::RegexFlags:
case TokenType::UnterminatedStringLiteral:
case TokenType::UnterminatedRegexLiteral:
case TokenCategory::String:
return StyleType::String;
case TokenType::BracketClose:
case TokenType::BracketOpen:
case TokenType::Comma:
case TokenType::CurlyClose:
case TokenType::CurlyOpen:
case TokenType::ParenClose:
case TokenType::ParenOpen:
case TokenType::Semicolon:
case TokenType::Colon:
case TokenType::Period:
case TokenCategory::Punctuation:
return StyleType::Punctuation;
case TokenType::Ampersand:
case TokenType::AmpersandEquals:
case TokenType::Arrow:
case TokenType::Asterisk:
case TokenType::AsteriskEquals:
case TokenType::Caret:
case TokenType::CaretEquals:
case TokenType::DoubleAmpersand:
case TokenType::DoubleAsterisk:
case TokenType::DoubleAsteriskEquals:
case TokenType::DoublePipe:
case TokenType::DoubleQuestionMark:
case TokenType::Equals:
case TokenType::EqualsEquals:
case TokenType::EqualsEqualsEquals:
case TokenType::ExclamationMark:
case TokenType::ExclamationMarkEquals:
case TokenType::ExclamationMarkEqualsEquals:
case TokenType::GreaterThan:
case TokenType::GreaterThanEquals:
case TokenType::LessThan:
case TokenType::LessThanEquals:
case TokenType::Minus:
case TokenType::MinusEquals:
case TokenType::MinusMinus:
case TokenType::Percent:
case TokenType::PercentEquals:
case TokenType::Pipe:
case TokenType::PipeEquals:
case TokenType::Plus:
case TokenType::PlusEquals:
case TokenType::PlusPlus:
case TokenType::QuestionMark:
case TokenType::QuestionMarkPeriod:
case TokenType::ShiftLeft:
case TokenType::ShiftLeftEquals:
case TokenType::ShiftRight:
case TokenType::ShiftRightEquals:
case TokenType::Slash:
case TokenType::SlashEquals:
case TokenType::Tilde:
case TokenType::TripleDot:
case TokenType::UnsignedShiftRight:
case TokenType::UnsignedShiftRightEquals:
case TokenCategory::Operator:
return StyleType::Operator;
case TokenType::BoolLiteral:
case TokenType::NullLiteral:
return StyleType::KeywordBold;
case TokenType::Async:
case TokenType::Class:
case TokenType::Const:
case TokenType::Debugger:
case TokenType::Delete:
case TokenType::Export:
case TokenType::Extends:
case TokenType::Function:
case TokenType::Import:
case TokenType::In:
case TokenType::Instanceof:
case TokenType::Interface:
case TokenType::Let:
case TokenType::New:
case TokenType::Super:
case TokenType::TemplateLiteralExprStart:
case TokenType::TemplateLiteralExprEnd:
case TokenType::This:
case TokenType::Throw:
case TokenType::Typeof:
case TokenType::Var:
case TokenType::Void:
return StyleType::Keyword;
case TokenType::Await:
case TokenType::Break:
case TokenType::Case:
case TokenType::Catch:
case TokenType::Continue:
case TokenType::Default:
case TokenType::Do:
case TokenType::Else:
case TokenType::Finally:
case TokenType::For:
case TokenType::If:
case TokenType::Return:
case TokenType::Switch:
case TokenType::Try:
case TokenType::While:
case TokenType::With:
case TokenType::Yield:
case TokenCategory::Keyword:
switch (token.type()) {
case TokenType::BoolLiteral:
case TokenType::NullLiteral:
return StyleType::KeywordBold;
default:
return StyleType::Keyword;
}
case TokenCategory::ControlKeyword:
return StyleType::ControlKeyword;
case TokenType::Identifier:
case TokenCategory::Identifier:
return StyleType::Identifier;
default:
dbgln("Unknown style type for token {}", token.name());