1
Fork 0
mirror of https://github.com/RGBCube/serenity synced 2025-07-25 16:37:35 +00:00

LibJS: Unify syntax highlighting

So far we have three different syntax highlighters for LibJS:

- js's Line::Editor stylization
- JS::MarkupGenerator
- GUI::JSSyntaxHighlighter

This not only caused repetition of most token types in each highlighter
but also a lot of inconsistency regarding the styling of certain tokens:

- JSSyntaxHighlighter was considering TokenType::Period to be an
  operator whereas MarkupGenerator categorized it as punctuation.
- MarkupGenerator was considering TokenType::{Break,Case,Continue,
  Default,Switch,With} control keywords whereas JSSyntaxHighlighter just
  disregarded them
- MarkupGenerator considered some future reserved keywords invalid and
  others not. JSSyntaxHighlighter and js disregarded most

Adding a new token type meant adding it to ENUMERATE_JS_TOKENS as well
as each individual highlighter's switch/case construct.

I added a TokenCategory enum, and each TokenType is now associated to a
certain category, which the syntax highlighters then can use for styling
rather than operating on the token type directly. This also makes
changing a token's category everywhere easier, should we need to do that
(e.g. I decided to make TokenType::{Period,QuestionMarkPeriod}
TokenCategory::Operator for now, but we might want to change them to
Punctuation.
This commit is contained in:
Linus Groh 2020-10-04 22:28:59 +01:00 committed by Andreas Kling
parent 6b55b007dd
commit e80217a746
5 changed files with 195 additions and 437 deletions

View file

@ -35,9 +35,9 @@ namespace JS {
const char* Token::name(TokenType type)
{
switch (type) {
#define __ENUMERATE_JS_TOKEN(x) \
case TokenType::x: \
return #x;
#define __ENUMERATE_JS_TOKEN(type, category) \
case TokenType::type: \
return #type;
ENUMERATE_JS_TOKENS
#undef __ENUMERATE_JS_TOKEN
default:
@ -51,6 +51,24 @@ const char* Token::name() const
return name(m_type);
}
TokenCategory Token::category(TokenType type)
{
switch (type) {
#define __ENUMERATE_JS_TOKEN(type, category) \
case TokenType::type: \
return TokenCategory::category;
ENUMERATE_JS_TOKENS
#undef __ENUMERATE_JS_TOKEN
default:
ASSERT_NOT_REACHED();
}
}
TokenCategory Token::category() const
{
return category(m_type);
}
double Token::double_value() const
{
ASSERT(type() == TokenType::NumericLiteral);