Bug 211867 - [IWYU] Remove unnecessary includes from JSC implementation files
Summary: [IWYU] Remove unnecessary includes from JSC implementation files
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: JavaScriptCore (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Ross Kirsling
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2020-05-13 16:08 PDT by Ross Kirsling
Modified: 2020-05-15 13:50 PDT (History)
10 users (show)

See Also:


Attachments
WIP Patch (directories L and up) (214.27 KB, patch)
2020-05-13 16:17 PDT, Ross Kirsling
no flags Details | Formatted Diff | Diff
WIP Patch (directories L and up) (214.69 KB, patch)
2020-05-13 16:24 PDT, Ross Kirsling
no flags Details | Formatted Diff | Diff
WIP Patch (directories L and up) (213.08 KB, patch)
2020-05-13 18:34 PDT, Ross Kirsling
no flags Details | Formatted Diff | Diff
WIP Patch (directories L and up) (212.87 KB, patch)
2020-05-13 19:05 PDT, Ross Kirsling
no flags Details | Formatted Diff | Diff
WIP Patch (directories L and up) (213.08 KB, patch)
2020-05-13 20:21 PDT, Ross Kirsling
no flags Details | Formatted Diff | Diff
WIP Patch (directories L and up) (212.58 KB, patch)
2020-05-13 21:58 PDT, Ross Kirsling
no flags Details | Formatted Diff | Diff
Patch (448.54 KB, patch)
2020-05-14 18:27 PDT, Ross Kirsling
no flags Details | Formatted Diff | Diff
Patch (447.96 KB, patch)
2020-05-14 18:46 PDT, Ross Kirsling
no flags Details | Formatted Diff | Diff
Patch (447.39 KB, patch)
2020-05-14 19:03 PDT, Ross Kirsling
no flags Details | Formatted Diff | Diff
Patch (446.92 KB, patch)
2020-05-14 19:47 PDT, Ross Kirsling
no flags Details | Formatted Diff | Diff
Patch (446.92 KB, patch)
2020-05-14 19:50 PDT, Ross Kirsling
no flags Details | Formatted Diff | Diff
Patch (446.92 KB, patch)
2020-05-14 20:02 PDT, Ross Kirsling
no flags Details | Formatted Diff | Diff
Patch (447.00 KB, patch)
2020-05-14 20:35 PDT, Ross Kirsling
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ross Kirsling 2020-05-13 16:08:04 PDT
Continuation of bug 211821 -- let's go through include-what-you-use's recommended #include removals for Source/JavaScriptCore/**/*.cpp.
Comment 1 Ross Kirsling 2020-05-13 16:17:07 PDT Comment hidden (obsolete)
Comment 2 Ross Kirsling 2020-05-13 16:24:38 PDT Comment hidden (obsolete)
Comment 3 Ross Kirsling 2020-05-13 18:34:36 PDT Comment hidden (obsolete)
Comment 4 Mark Lam 2020-05-13 18:41:17 PDT
What is your methodology for doing this change?  (1) Just trial and error to see if removing a #include causes a compilation error?  Or (2)do you have another means of doing this?
Comment 5 Keith Miller 2020-05-13 18:44:33 PDT
(In reply to Mark Lam from comment #4)
> What is your methodology for doing this change?  (1) Just trial and error to
> see if removing a #include causes a compilation error?  Or (2)do you have
> another means of doing this?

[IWYU] is "include what you use" it's a clang tool. Although, it seems to have broken things here...
Comment 6 Ross Kirsling 2020-05-13 18:51:23 PDT
(In reply to Mark Lam from comment #4)
> What is your methodology for doing this change?  (1) Just trial and error to
> see if removing a #include causes a compilation error?  Or (2)do you have
> another means of doing this?

I described it in bug 211821 comment 2 -- IWYU spits out a list of recommended insertions/deletions and I'm going through them manually.

It's tedious, but more sustainable than blind trial-and-error...although given how error-prone it is, the cost/benefit might be questionable. I think it gets pretty confused by our *Inlines.h approach; evidently there are lots of linking errors that simply won't happen with JSCOnly.
Comment 7 Ross Kirsling 2020-05-13 19:05:30 PDT Comment hidden (obsolete)
Comment 8 Ross Kirsling 2020-05-13 20:21:07 PDT Comment hidden (obsolete)
Comment 9 Ross Kirsling 2020-05-13 21:58:40 PDT
Created attachment 399332 [details]
WIP Patch (directories L and up)
Comment 10 Ross Kirsling 2020-05-14 11:03:46 PDT
(Also, sorry for the noise; EWS auto-CCs even without r? but I'm planning to finish the other half of this before asking for review...)
Comment 11 Ross Kirsling 2020-05-14 18:27:04 PDT Comment hidden (obsolete)
Comment 12 Ross Kirsling 2020-05-14 18:46:19 PDT Comment hidden (obsolete)
Comment 13 Ross Kirsling 2020-05-14 19:03:30 PDT Comment hidden (obsolete)
Comment 14 Ross Kirsling 2020-05-14 19:47:01 PDT Comment hidden (obsolete)
Comment 15 Ross Kirsling 2020-05-14 19:50:03 PDT Comment hidden (obsolete)
Comment 16 Ross Kirsling 2020-05-14 20:02:16 PDT Comment hidden (obsolete)
Comment 17 Ross Kirsling 2020-05-14 20:35:14 PDT
Created attachment 399450 [details]
Patch
Comment 18 Ross Kirsling 2020-05-14 20:44:34 PDT
Comment on attachment 399450 [details]
Patch

Phew, I think all bots should be happy now.
Comment 19 Ross Kirsling 2020-05-14 22:16:06 PDT
To be slightly more explicit about how I "went through the removals manually", for the curious:

1. Open a batch of files and prune includes from each as recommended

2. Build and see what compile/link errors occur
   a. First do the non-unified JSCOnly build as usual.
   b. Then do a normal build-jsc run, since it encounters WAY more link errors (in spite of being a unified build, which ideally shouldn't be used here at all).

3. Put some includes back
   a. Compile errors are usually as simple as "you said we didn't need that line but clearly we do".
   b. Link errors are always because of an *Inlines.h include. In most of these cases, it's possible to replace that include with a single smaller *Inlines.h (but if you need multiple then you may as well restore the original).

4. Profit?

I wouldn't necessarily recommend others to spend multiple days of their life doing such a thing, but as you can see, the manual effort does produce concrete results, whereas I probably would've just given up outright before ever figuring out how to make IWYU play nicely with WK in an automated fashion (as KeithR said he did a number of years ago).
Comment 20 Keith Miller 2020-05-15 01:17:43 PDT
(In reply to Ross Kirsling from comment #19)
> To be slightly more explicit about how I "went through the removals
> manually", for the curious:
> 
> 1. Open a batch of files and prune includes from each as recommended
> 
> 2. Build and see what compile/link errors occur
>    a. First do the non-unified JSCOnly build as usual.
>    b. Then do a normal build-jsc run, since it encounters WAY more link
> errors (in spite of being a unified build, which ideally shouldn't be used
> here at all).
> 
> 3. Put some includes back
>    a. Compile errors are usually as simple as "you said we didn't need that
> line but clearly we do".
>    b. Link errors are always because of an *Inlines.h include. In most of
> these cases, it's possible to replace that include with a single smaller
> *Inlines.h (but if you need multiple then you may as well restore the
> original).
> 
> 4. Profit?
> 
> I wouldn't necessarily recommend others to spend multiple days of their life
> doing such a thing, but as you can see, the manual effort does produce
> concrete results, whereas I probably would've just given up outright before
> ever figuring out how to make IWYU play nicely with WK in an automated
> fashion (as KeithR said he did a number of years ago).

I'm kinda surprised that IWYU can't figure out how to sink the includes. I'm guessing it's because it doesn't look at every translation unit at once? Seems like it could export some graph to be processed offline though. Not saying that's a trivial project...
Comment 21 Joseph Pecoraro 2020-05-15 11:18:22 PDT
(In reply to Ross Kirsling from comment #18)
> Comment on attachment 399450 [details]
> Patch
> 
> Phew, I think all bots should be happy now.

Wow, nice! Did this produce any noticeable build time improvement?
Comment 22 Keith Miller 2020-05-15 11:33:35 PDT
Comment on attachment 399450 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=399450&action=review

rs=me

> Source/JavaScriptCore/ChangeLog:9
> +        * API/:
> +        * assembler/:

Lol, I love the blank lines.
Comment 23 Ross Kirsling 2020-05-15 12:23:51 PDT
(In reply to Joseph Pecoraro from comment #21)
> (In reply to Ross Kirsling from comment #18)
> > Comment on attachment 399450 [details]
> > Patch
> > 
> > Phew, I think all bots should be happy now.
> 
> Wow, nice! Did this produce any noticeable build time improvement?

I'm hoping so! Gonna do a before-and-after now and see what time/size differences I can detect.
Comment 24 EWS 2020-05-15 12:39:47 PDT
Committed r261755: <https://trac.webkit.org/changeset/261755>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 399450 [details].
Comment 25 Radar WebKit Bug Importer 2020-05-15 12:40:20 PDT
<rdar://problem/63284954>
Comment 26 Ross Kirsling 2020-05-15 13:50:53 PDT
Well, it doesn't seem like there's any difference in the size of the build products with a normal unified `build-jsc` run (which I guess shouldn't surprise me). My Debug build did complete 45s faster, although I don't have enough data points to ensure this wasn't a fluke.