Bug 108512 – cr-linux debug should use clang and maybe be a components build

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Tony Chang]

In bug 98957, Alan added cr-linux-debug EWS bots.  That's great!

I suggest that we convert these bots to use clang and the components build so we can increase coverage.  E.g., Chromium Mac uses clang, so this will catch some errors that trip up Chromium Mac.  Use the components build should be faster for debug too.

[Nico Weber]

As far as I know, webkit doesn't have a components build, it's all one single big component.

Is that no longer true?

Using clang sounds great :-)

[Alan Cutter]

I'm happy to set up clang for the debug bots. I'm not familiar with what you mean by a components build though, any hints?

[Tony Chang]

http://www.chromium.org/developers/how-tos/component-build describes the components build.

I don't think there's a way to force this via build-webkit, but if you set the variable in ~/.gyp/include.gypi, then run update-webkit-chromium, it should generate gyp files for .so files.

[Tony Chang]

Alternately, we could add a command line switch to update-webkit-chromium.

[Nico Weber]

(In reply to comment #4)
> Alternately, we could add a command line switch to update-webkit-chromium.

[Nico Weber]

(In reply to comment #4)
> Alternately, we could add a command line switch to update-webkit-chromium.

(In reply to comment #3)
> http://www.chromium.org/developers/how-tos/component-build describes the components build.
> 
> I don't think there's a way to force this via build-webkit, but if you set the variable in ~/.gyp/include.gypi, then run update-webkit-chromium, it should generate gyp files for .so files.

Right, but that builds  a single monolithic .so for all of the code in webkit (it even pulls in the tests), right? Is there any point in using this in a standalone webkit build?

[Tony Chang]

(In reply to comment #6)
> Right, but that builds  a single monolithic .so for all of the code in webkit (it even pulls in the tests), right? Is there any point in using this in a standalone webkit build?

I was just suggesting it as a way to get more compile configuration diversity (stuff that might break on build.chromium.org), but it should still link DRT and webkit_unit_tests faster than having a single binary including all webkit, base, net, webkit_glue, third_party, etc.

[Nico Weber]

(In reply to comment #7)
> (In reply to comment #6)
> > Right, but that builds  a single monolithic .so for all of the code in webkit (it even pulls in the tests), right? Is there any point in using this in a standalone webkit build?
> 
> I was just suggesting it as a way to get more compile configuration diversity (stuff that might break on build.chromium.org),

Right, but does is that actually the case? The dependent components that come in via deps are built on the chromium waterfall first before they're rolled into chromium so that doesn't catch anything.

And the only thing with _EXPORT annotations in webkit is the chromium webkit api. Would anything link against that on the webkit waterfall alone? DRT I suppose, so I guess this does make sense.

Carry on then :-)

> but it should still link DRT and webkit_unit_tests faster than having a single binary including all webkit, base, net, webkit_glue, third_party, etc.

[Tony Chang]

I was actually thinking of the case where you add a new file to webkit_unit_tests and it doesn't work properly in the components build because of the weird "compile tests into libwebkit.so" behavior you mentioned above.

[Alan Cutter]

Created attachment 185982 [details]
Patch

[Eric Seidel (no email)]

Comment on attachment 185982 [details]
Patch

OK.

[Eric Seidel (no email)]

I thought 2-clause bsd was the new hotness?  (also, nico is really your best reviewer... but he's not a reviewer quite yet!)

[Alan Cutter]

(In reply to comment #10)
> Created an attachment (id=185982) [details]
> Patch

The current GCE Chromium Linux debug build bots have already been set up to use clang. This patch just ensures new bots for the queue will similarly be running clang over gcc.

Based on the discussion in the comments I felt the components build configuration was best left for another bug if it's still a good idea.

[Alan Cutter]

(In reply to comment #12)
> I thought 2-clause bsd was the new hotness?  (also, nico is really your best reviewer... but he's not a reviewer quite yet!)

I wasn't sure about our licence policy, I'll use 2-clause BSD next time. (:

[Nico Weber]

Comment on attachment 185982 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=185982&action=review

> Tools/EWSTools/GoogleComputeEngine/build-cr-linux-debug-ews.sh:59
> +    bash configure-clang-linux.sh $QUEUE_TYPE &&

How often does this run? All our waterfall bots run update.sh after every update as the compiler changes every 1-2 weeks usually. If this script runs only at bot startup, that's not often enough.

> Tools/EWSTools/configure-clang-linux.sh:38
> +export CXX=clang++

Does this bot use goma? If not, you don't need to set PATH, CC and CXX: clang=1 in your GYP_DEFINES will do this automatically.

[Alan Cutter]

(In reply to comment #15)
> (From update of attachment 185982 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=185982&action=review
> 
> > Tools/EWSTools/GoogleComputeEngine/build-cr-linux-debug-ews.sh:59
> > +    bash configure-clang-linux.sh $QUEUE_TYPE &&
> 
> How often does this run? All our waterfall bots run update.sh after every update as the compiler changes every 1-2 weeks usually. If this script runs only at bot startup, that's not often enough.
> 
> > Tools/EWSTools/configure-clang-linux.sh:38
> > +export CXX=clang++
> 
> Does this bot use goma? If not, you don't need to set PATH, CC and CXX: clang=1 in your GYP_DEFINES will do this automatically.

Updating the scripts to address these concerns, running into problems with GCE at the moment unfortunately. Will repatch once I can test the new scripts properly.

[Alan Cutter]

Created attachment 186821 [details]
Patch

[Nico Weber]

Comment on attachment 186821 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=186821&action=review

> Tools/EWSTools/GoogleComputeEngine/build-cr-linux-debug-ews.sh:60
> +screen -t kr ./start-queue.sh -r \\\"configure-clang-linux.sh $QUEUE_TYPE\\\" $QUEUE_TYPE $BOT_ID 10\" &&

Is configure-clang-linux.sh a new file? Do you need to add it to this patch?

[Alan Cutter]

Created attachment 186920 [details]
Patch

[Alan Cutter]

(In reply to comment #18)
> (From update of attachment 186821 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=186821&action=review
> 
> > Tools/EWSTools/GoogleComputeEngine/build-cr-linux-debug-ews.sh:60
> > +screen -t kr ./start-queue.sh -r \\\"configure-clang-linux.sh $QUEUE_TYPE\\\" $QUEUE_TYPE $BOT_ID 10\" &&
> 
> Is configure-clang-linux.sh a new file? Do you need to add it to this patch?

Thanks for spotting that! It somehow got dropped out between patches.

[Tony Chang]

Bump for eric or abarth to review.

FWIW, I've seen two component build rollouts in the past week, but we can tackle that in a different bug.
https://bugs.webkit.org/show_bug.cgi?id=109746

[WebKit Review Bot]

Comment on attachment 186920 [details]
Patch

Clearing flags on attachment: 186920

Committed r142802: <http://trac.webkit.org/changeset/142802>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.