Bug 34144

Summary: Web Inspector: Implement simple highlighter for HTML.
Product: WebKit Reporter: Pavel Feldman <pfeldman>
Component: Web Inspector (Deprecated)Assignee: Pavel Feldman <pfeldman>
Status: RESOLVED FIXED    
Severity: Normal CC: apavlov, bweinstein, joepeck, rik, timothy, webkit.review.bot
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: All   
OS: All   
Attachments:
Description Flags
[PATCH] Proposed change.
none
[PATCH] Proposed change (style fix).
none
[PATCH] Same with better naming and CSS tokenizer bonus. timothy: review+

Description Pavel Feldman 2010-01-25 11:42:48 PST 
This change also introduces parse state that is necessary to keep generated lexer code compact.
Comment 1 Pavel Feldman 2010-01-25 11:44:48 PST 
Created attachment 47361 [details]
[PATCH] Proposed change.
Comment 2 WebKit Review Bot 2010-01-25 11:48:20 PST 
Attachment 47361 [details] did not pass style-queue:

Failed to run "WebKitTools/Scripts/check-webkit-style" exit_code: 1
Skipping input 'WebCore/inspector/front-end/JavaScriptTokenizer.js': Can't open for reading
WebCore/inspector/front-end/TokenizerForHTML.js:78:  Line contains tab character.  [whitespace/tab] [5]
WebCore/inspector/front-end/TokenizerForHTML.js:79:  Line contains tab character.  [whitespace/tab] [5]
Total errors found: 2


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 3 Pavel Feldman 2010-01-25 11:56:56 PST 
Created attachment 47364 [details]
[PATCH] Proposed change (style fix).

Please ignore style failures in the removed file.
Comment 4 Timothy Hatcher 2010-01-25 12:09:29 PST 
Comment on attachment 47361 [details]
[PATCH] Proposed change.

I prefer HTMLTokenizer over TokenizerForHTML, like you had JavaScriptTokenizer. It is a better class name. (Even though the new name keeps the files together.)

Otherwise r+.
Comment 5 Pavel Feldman 2010-01-25 14:23:32 PST 
Created attachment 47368 [details]
[PATCH] Same with better naming and CSS tokenizer bonus.
Comment 6 Timothy Hatcher 2010-01-25 17:29:13 PST 
Comment on attachment 47368 [details]
[PATCH] Same with better naming and CSS tokenizer bonus.

> -    this._tokenizer = new WebInspector.JavaScriptTokenizer();
> +    this._tokenizer = new WebInspector.SourceCSSTokenizer();

What tokenizer should this be set to?
Comment 7 Pavel Feldman 2010-01-25 21:41:33 PST 
(In reply to comment #6)
> (From update of attachment 47368 [details])
> > -    this._tokenizer = new WebInspector.JavaScriptTokenizer();
> > +    this._tokenizer = new WebInspector.SourceCSSTokenizer();
> 
> What tokenizer should this be set to?

This is going to be determined by the mimeType passed to the editor (simple mapping). Currently just hardcoded for testing.

I think I am one patch away form the editor as a viewer.
Comment 8 Pavel Feldman 2010-01-26 02:19:11 PST 
Committing to http://svn.webkit.org/repository/webkit/trunk ...
	R	WebCore/inspector/front-end/JavaScriptTokenizer.js => WebCore/inspector/front-end/SourceJavaScriptTokenizer.js
	R	WebCore/inspector/front-end/JavaScriptTokenizer.re2js => WebCore/inspector/front-end/SourceJavaScriptTokenizer.re2js
	M	WebCore/ChangeLog
	M	WebCore/WebCore.gypi
	M	WebCore/WebCore.vcproj/WebCore.vcproj
	A	WebCore/inspector/front-end/SourceCSSTokenizer.js
	A	WebCore/inspector/front-end/SourceCSSTokenizer.re2js
	A	WebCore/inspector/front-end/SourceHTMLTokenizer.js
	A	WebCore/inspector/front-end/SourceHTMLTokenizer.re2js
	M	WebCore/inspector/front-end/TextEditorHighlighter.js
	M	WebCore/inspector/front-end/WebKit.qrc
	M	WebCore/inspector/front-end/inspector.html
Committed r53846