Bug 63940 – Web Inspector: draft implementation of compiler source mappings support in debugger.

• Component: Web Inspector (Deprecated)
• Status: RESOLVED FIXED

[Pavel Podivilov]

Web Inspector: draft implementation of compiler source mappings support in debugger.

When compiled script has source mapping associated with it, all debugger events in that script are converted to source code coordinates according to mapping. Also, it is possible to set breakpoint on source code rather then in compiled script - breakpoint location would be automatically converted to compiled script coordinates.

[Pavel Podivilov]

Created attachment 99713 [details]
Patch.

[Patrick Mueller]

Is there any description of how the source mappings is intended to work?  There's been interest in being able to debug CoffeeScript source, and it seems like the work going on in this area can be used to make that happen.  

Or is there some other use case for source mappings, besides dealing with unobfuscating min-ized scripts?

[Pavel Podivilov]

(In reply to comment #2)
> Is there any description of how the source mappings is intended to work?  

Here it is:
1. User provides a source mapping url associated with a script.
  1a. If script has //@sourceMap=<sourceMapURL> comment, there will be an info bar asking for permission to enable the mapping.
  2a. It should be possible to explicitly specify source mapping url for a any script no matter it has "//@sourceMap" or not.
  3a. It should be possible to specify a root url for all source mappings.

2. Inspector downloads source mapping using JSONP (http://en.wikipedia.org/wiki/JSONP). We want to be able to download the mapping from arbitrary location (local disk, app domain, any other domain), and JSONP is the only way to do it right from front-end page.

3. Inspector adds an entry to scripts panel's file select for each source files listed in source mapping.

4. Source file content is downloaded only when it is actually needed (user selects corresponding entry in files select or debugger is paused on some line in that file). Source files are downloaded using JSONP as well.

Source code may be an unobfuscated javascript of any other language (Java, CoffeeScript, etc) as long as compiler generates source maps that we understand. It shouldn't be hard to support as many formats as needed.

Similar effort from Mozilla - https://wiki.mozilla.org/DevTools/Features/SourceMap.
Existing FireBug extension that uses source maps generated by closure compiler - http://code.google.com/closure/compiler/docs/inspector.html.


>There's been interest in being able to debug CoffeeScript source, and it seems like the work going on in this area can be used to make that happen.  
> 

Great!

> Or is there some other use case for source mappings, besides dealing with unobfuscating min-ized scripts?

It is the only case I'm aware of.

[johnjbarton]

(In reply to comment #2)
> Or is there some other use case for source mappings, besides dealing with unobfuscating min-ized scripts?

I just want to put out the suggestion to think about source mappings very broadly. Ultimately I think we want the entire development API to work through source mappings. Each part of the browser maintains a crude source map in the form of file/line number tables. Often these maps are lame and ad-hoc, making the tools that have to use them lame. Now imagine all parts of the browser having a uniformly strong, multiple-layer source map capability. Then we can deal with -mapping compressed-CoffeeScript -> CoffeeScript -> EcmaScript 7 -> EcmaScript 3 -> JIT intermediate representation -> etc.  More important in the big picture, we can use source maps to explain runtime results. For example, a source map can be used to show developers that "class='foo'" was added to div#bar by the JavaScript code on line 445 in baz.js.  For scalability, these maps may be generated on demand, but if the infrastructure all shifts from file/line to source maps we are in a great position to leverage the infrastructure in future.

[johnjbarton]

Any info on the API afforded by these proposed source maps? Sample files?

[Pavel Feldman]

Yeah, a high level design description of how this fits into the overall picture of source code processing would be great. We have a Web Inspector wiki for things like that.

[johnjbarton]

(In reply to comment #6)
> Yeah, a high level design description of how this fits into the overall picture of source code processing would be great. We have a Web Inspector wiki for things like that.

I assume you mean:
   http://trac.webkit.org/wiki/WebInspector
Is there a better place to discuss (rather than document) this work?

Also related discussions:
"I would appreciate some feedback on the Source Map developer tool (planning stage)"
http://groups.google.com/group/mozilla.dev.platform/browse_thread/thread/d870fa0d2a625f7d#

[Pavel Podivilov]

(In reply to comment #7)
> (In reply to comment #6)
> > Yeah, a high level design description of how this fits into the overall picture of source code processing would be great. We have a Web Inspector wiki for things like that.
> 
> I assume you mean:
>    http://trac.webkit.org/wiki/WebInspector
> Is there a better place to discuss (rather than document) this work?

Created a document with a draft description of source maps https://docs.google.com/document/d/14AWiLDDxEuLaWuyG0X6deLRufrvxRu8HBP0LNJwvRZw/edit?hl=en_US&pli=1.

[Patrick Mueller]

(In reply to comment #8)
> Created a document with a draft description of source maps https://docs.google.com/document/d/14AWiLDDxEuLaWuyG0X6deLRufrvxRu8HBP0LNJwvRZw/edit?hl=en_US&pli=1.

As my jazz guitar teacher used to say, "good enough for rock and roll".  Meaning, looks good enough for me, for now.  I'm sure we'll want to tweak some of this stuff once it's actually kinda working, but we won't know what to tweak until we have something kinda working.  :-)

The only thing that stands out to me is 1a)

"1a. If script has //@sourceMap=<sourceMapURL> comment, there will be an info bar asking for permission to enable the mapping."

That seems like it's going to be a pain in the ass.  We already have the de-obfuscate/obfuscate (tiny) button; just use that instead of the infobar permission.  Or if you MUST have an infobar, only ask the first time "do you want to enable source mappings" with an option to turn it off, globally, in the new settings panel.

[Nick Fitzgerald :fitzgen]

Hello Pavel,

I want to invite you to join with Mozilla and the Google Closure Compiler team and help us make a standard source map format: https://docs.google.com/document/d/1U1RGAehQwRypUTovF1KRlpiOFze0b-_2gc6fAH0KY0k/edit?pli=1

I think it would be best if we all agreed upon one format and standardized it.

Great to see that other people are also working on this problem!

Nick (Lead Developer on Mozilla's Source Map feature)

[Yehuda Katz]

(In reply to comment #3)
> (In reply to comment #2)
> > Is there any description of how the source mappings is intended to work?  
> 
> Here it is:
> 1. User provides a source mapping url associated with a script.
>   1a. If script has //@sourceMap=<sourceMapURL> comment, there will be an info bar asking for permission to enable the mapping.
>   2a. It should be possible to explicitly specify source mapping url for a any script no matter it has "//@sourceMap" or not.
>   3a. It should be possible to specify a root url for all source mappings.
> 
> 2. Inspector downloads source mapping using JSONP (http://en.wikipedia.org/wiki/JSONP). We want to be able to download the mapping from arbitrary location (local disk, app domain, any other domain), and JSONP is the only way to do it right from front-end page.
> 
> 3. Inspector adds an entry to scripts panel's file select for each source files listed in source mapping.
> 
> 4. Source file content is downloaded only when it is actually needed (user selects corresponding entry in files select or debugger is paused on some line in that file). Source files are downloaded using JSONP as well.
> 
> Source code may be an unobfuscated javascript of any other language (Java, CoffeeScript, etc) as long as compiler generates source maps that we understand. It shouldn't be hard to support as many formats as needed.
> 
> Similar effort from Mozilla - https://wiki.mozilla.org/DevTools/Features/SourceMap.
> Existing FireBug extension that uses source maps generated by closure compiler - http://code.google.com/closure/compiler/docs/inspector.html.
> 
> 
> >There's been interest in being able to debug CoffeeScript source, and it seems like the work going on in this area can be used to make that happen.  
> > 
> 
> Great!
> 
> > Or is there some other use case for source mappings, besides dealing with unobfuscating min-ized scripts?
> 
> It is the only case I'm aware of.

Another case is compiled templates. I compile Handlebars templates to JavaScript, but Handlebars itself is a quasi-scripting language. It would be great if I could annotate the compiled JavaScript with a map back to the template source. It would be decently easy, since I use Jison to parse my source, which would make it trivial for me to annotate my AST with line/column numbers, which I could then use during compilation to generate the source map (as a secondary build artifact).

[Ray Cromwell]

I've added SourceMap support to GWT (Java -> JS), but one interesting question I've got is whether we can support CSS source-mapping. GWT contains a Css Parser/Css Optimizer/Css Minifier. We might want to consider a way to specify that <style> elements or class="" attributes link to a separate sourcemap. This would help in particular for obfuscated class attribute names which some CSS tools produce.

[Guy Bedford]

I'm interested in seeing how source maps can work for both <style> elements and evalled script. This is in order to bring support to the RequireJS CoffeeScript parser and LESS parser. One way I've been experimenting with this is through data-uris in the @ sourceMapperURL property.

When testing this with eval code, it seems that this method is working correctly. The only issue I have is that the @ sourceURL property is loading the js file not the coffeescript file in my case, even though the debug line points correspond to the coffeescript file. If this could be fixed so that when a @ sourceURL and @ sourceMapperURL are both present in evalled code, the @ sourceURL code is displayed in full, that would be great.

Example repo here - https://github.com/guybedford/cs-source-maps  (test-eval.html is the sample page)