Bug 192294 – [WHLSL] Add a handwritten lexer

• Component: New Bugs
• Status: RESOLVED FIXED

[Myles C. Maxfield]

[WHLSL] Add a handwritten lexer

[Myles C. Maxfield]

Created attachment 356353 [details]
WIP

[EWS Watchlist]

Attachment 356353 [details] did not pass style-queue:


ERROR: Source/WebCore/ChangeLog:8:  You should remove the 'No new tests' and either add and list tests, or explain why no new tests were possible.  [changelog/nonewtests] [5]
Total errors found: 1 in 5 files


If any of these errors are false positives, please file a bug against check-webkit-style.

[Myles C. Maxfield]

Created attachment 357617 [details]
Patch

[Jon Lee]

Comment on attachment 357617 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=357617&action=review

> Source/WebCore/Modules/webgpu/WHLSL/WHLSLLexer.cpp:556
> +    for ( ; m_offset < m_stringView.length() && !isNewline(m_stringView[m_offset]); ++m_offset) { }

Nit: consistent style of empty block.

[Myles C. Maxfield]

Comment on attachment 357617 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=357617&action=review

>> Source/WebCore/Modules/webgpu/WHLSL/WHLSLLexer.cpp:556
>> +    for ( ; m_offset < m_stringView.length() && !isNewline(m_stringView[m_offset]); ++m_offset) { }
> 
> Nit: consistent style of empty block.

ERROR: Source/WebCore/Modules/webgpu/WHLSL/WHLSLLexer.cpp:558:  One line control clauses should not use braces.  [whitespace/braces] [4]

and if i use a semicolon

ERROR: Source/WebCore/Modules/webgpu/WHLSL/WHLSLLexer.cpp:556:  Semicolon defining empty statement for this loop. Use { } instead.  [whitespace/semicolon] [5]

[WebKit Commit Bot]

Comment on attachment 357617 [details]
Patch

Clearing flags on attachment: 357617

Committed r239398: <https://trac.webkit.org/changeset/239398>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/46854789>

[Saam Barati]

Comment on attachment 357617 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=357617&action=review

> Source/WebCore/Modules/webgpu/WHLSL/WHLSLLexer.cpp:255
> +auto Lexer::recognizeKeyword(unsigned end) -> std::optional<Token::Type>

Is this slow? Do you know what code LLVM emits for this? Is it actually a bunch of string compares? It might be worth doing what JSC’s lexer does. IIRC, we have a compile step that essentially compiles a function that does this.

[Myles C. Maxfield]

Yeah I thought about making a trie at compiler time but I didn’t want to add the complexity in the first patch. I’ll measure how bad the current code is to see if it’s worth it.