Bug 110493 – Web Inspector: Segment built files by module boundaries as alternative to sourceURL

• Status: RESOLVED
• Resolution: INVALID
• Component: Web Inspector (Deprecated)

[James Burke]

It is common for performance to combine multiple files into one file. However those combined files are hard to debug. One mechanism to do so is to use //@ sourceURL, but this requires the combined code to be stringified and then evaled.

This can be awkward to use because:

* Some Content Security Policy (CSP) uses forbid eval. This is comes up directly for certain types of Web Apps.
* It causes an awkward interaction with source maps, makes it harder to get minified code.

With the coming use of modules in ECMAScript, and the use of AMD modules today, it is possible to use those module boundaries instead. This has a few benefits:

* No eval in play.
* as ES modules get adopted, there will be modules that are made up of tinier, scoped interior modules. Being able to report to the library author the module ID that had the problem will give better targeting of issues.
* //@ sourceURL relies on a path. With modules that use module IDs, paths become less important, and in some cases confusing in the case of bundled modules distributed and used in other projects: those paths will likely not match the application paths.

While ES modules are not done yet, they seem to have settled on using string IDs, like AMD and CommonJS/node use. In the ES case, a file containing many modules will have boundaries like this:

module 'some/thing' {
  // module code in here
}

For AMD, it looks like this:

define('some/thing', [optional array], function () {
  // module code in here
});

The developer tool debugger could look for these boundaries then use the module ID in the list of scripts ("some/thing" in the above example). Error line numbers would be reported relative to the module wrapper, with the first line number being the first line inside the module wrapper.

While ES modules are not complete yet, this can be put into use today, and work out any wrinkles by targeting AMD modules, as they are in widespread use.

I will likely need some pointers, but I am willing to write test cases if it helps.

(Also filed for Firefox at: https://bugzilla.mozilla.org/show_bug.cgi?id=843720

[johnjbarton]

It would be helpful to explain why the combined files are hard to debug. That would help clarify what solution would really help. 

Just to note, the bigger problem with debugging performance code is compression. I'm not sure this feature would be worth the cost/benefit given that ES modules will have native support for the names.

Parsing every source for these strings just in case they may be present is a significant overhead.  //@ sourceURL is always at the end. Maybe we could have a //@ amdSource tag at the end to engage parsing for names.

[James Burke]

(In reply to comment #1)
> It would be helpful to explain why the combined files are hard to debug. That would help clarify what solution would really help. 

* associating line numbers back to the unbundled files. This is mainly useful for any errors that are generated. Although, for longer modules, matching up line numbers to the unbundled code in the debugger would informative.

* easier access in the debugger to select segments of the bundled file. So, if I know I want to do debugging in "some/thing", being able to select that module as an item in the script list.

> Just to note, the bigger problem with debugging performance code is compression. I'm not sure this feature would be worth the cost/benefit given that ES modules will have native support for the names.

I am not sure I follow this. If you mean debugging minified code, I expect source maps to be used for that.

On treating module names like function names: that would likely allow the separate selection via the debugger interface, but would not have the line numbers relative to the module container, so not as useful for correlation back to the source files.

> Parsing every source for these strings just in case they may be present is a significant overhead.  //@ sourceURL is always at the end. Maybe we could have a //@ amdSource tag at the end to engage parsing for names.

An opt-in comment to get the behavior and incur the extra parsing cost seems reasonable.

[johnjbarton]

How about generating a source-map from the bundle to the original source?

[James Burke]

(In reply to comment #3)
> How about generating a source-map from the bundle to the original source?

For me, source maps are a bit unwieldy to use, and it should be possible to get these benefits without needing to resort to source maps, particularly when minification is not involved.

Source maps seem to deal with file paths, where modules with IDs are not directly tied to specific file paths. In the case of a reusable library that is made up of some internal modules, it does not seem to align with an approach that is file path-based.

[johnjbarton]

(In reply to comment #4)
> (In reply to comment #3)
> > How about generating a source-map from the bundle to the original source?
> 
> For me, source maps are a bit unwieldy to use,

Right, that should be fixed!

> and it should be possible to get these benefits without needing to resort to source maps, particularly when minification is not involved.

But if we fixed source maps UX, we'd get a lot more bang for the cost.

> 
> Source maps seem to deal with file paths, where modules with IDs are not directly tied to specific file paths. In the case of a reusable library that is made up of some internal modules, it does not seem to align with an approach that is file path-based.

But your goals include mapping line numbers to unbundled files and navigation by file name.  That sounds exactly like what source maps should do.

I think your concatenator could build the source map using 
https://github.com/mozilla/source-map/ 
pretty easy then we could find out exactly what the unwieldy bits are.
We might conclude that there is no hope or we might conclude that source
maps can be a lot better.

[James Burke]

(In reply to comment #5)
> But your goals include mapping line numbers to unbundled files and navigation by file name.  That sounds exactly like what source maps should do.

The goals include that but are not only that use case. I consider Module IDs the new units of functionality, which can be split up, loaded from different places, or grouped together. Even when wanting to map back to a separate file location, the module ID is enough without having the actual file location, and it also works in situations where a path does not make sense.

> I think your concatenator could build the source map using 
> https://github.com/mozilla/source-map/ 
> pretty easy then we could find out exactly what the unwieldy bits are.
> We might conclude that there is no hope or we might conclude that source
> maps can be a lot better.

I do not expect source maps will be usable in all cases. Example use cases where they seem awkward:

* appcache-enabled web apps.
* packaged web apps that are delivered via zip files.

I do not want to deliver the source map baggage for those cases. In those cases, I am more likely to deliver the source as-is or maybe just comments stripped, but still have some concatenated layers for performance. Working with a mobile device, it is common for them to have slower local file I/O where bundling can still help app load time. This has been found in practice with Firefox OS devices.

In those cases, this feature request would still allow better error reports, and if debugging, finer grained debug labels and code unit selection.

[Brian Burg]

Closing as invalid, as this bug pertains to the old inspector UI and/or its tests.
Please file a new bug (https://www.webkit.org/new-inspector-bug) if the bug/feature/issue is still relevant to WebKit trunk.