Bug 108512 - cr-linux debug should use clang and maybe be a components build
Summary: cr-linux debug should use clang and maybe be a components build
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Alan Cutter
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-01-31 12:29 PST by Tony Chang
Modified: 2013-02-13 14:41 PST (History)
5 users (show)

See Also:


Attachments
Patch (3.73 KB, patch)
2013-02-01 01:48 PST, Alan Cutter
no flags Details | Formatted Diff | Diff
Patch (6.56 KB, patch)
2013-02-06 04:16 PST, Alan Cutter
no flags Details | Formatted Diff | Diff
Patch (8.54 KB, patch)
2013-02-06 14:31 PST, Alan Cutter
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Tony Chang 2013-01-31 12:29:29 PST
In bug 98957, Alan added cr-linux-debug EWS bots.  That's great!

I suggest that we convert these bots to use clang and the components build so we can increase coverage.  E.g., Chromium Mac uses clang, so this will catch some errors that trip up Chromium Mac.  Use the components build should be faster for debug too.
Comment 1 Nico Weber 2013-01-31 12:30:30 PST
As far as I know, webkit doesn't have a components build, it's all one single big component.

Is that no longer true?

Using clang sounds great :-)
Comment 2 Alan Cutter 2013-01-31 15:19:57 PST
I'm happy to set up clang for the debug bots. I'm not familiar with what you mean by a components build though, any hints?
Comment 3 Tony Chang 2013-01-31 16:44:18 PST
http://www.chromium.org/developers/how-tos/component-build describes the components build.

I don't think there's a way to force this via build-webkit, but if you set the variable in ~/.gyp/include.gypi, then run update-webkit-chromium, it should generate gyp files for .so files.
Comment 4 Tony Chang 2013-01-31 16:44:41 PST
Alternately, we could add a command line switch to update-webkit-chromium.
Comment 5 Nico Weber 2013-01-31 16:45:18 PST
(In reply to comment #4)
> Alternately, we could add a command line switch to update-webkit-chromium.
Comment 6 Nico Weber 2013-01-31 16:46:34 PST
(In reply to comment #4)
> Alternately, we could add a command line switch to update-webkit-chromium.

(In reply to comment #3)
> http://www.chromium.org/developers/how-tos/component-build describes the components build.
> 
> I don't think there's a way to force this via build-webkit, but if you set the variable in ~/.gyp/include.gypi, then run update-webkit-chromium, it should generate gyp files for .so files.

Right, but that builds  a single monolithic .so for all of the code in webkit (it even pulls in the tests), right? Is there any point in using this in a standalone webkit build?
Comment 7 Tony Chang 2013-01-31 16:58:38 PST
(In reply to comment #6)
> Right, but that builds  a single monolithic .so for all of the code in webkit (it even pulls in the tests), right? Is there any point in using this in a standalone webkit build?

I was just suggesting it as a way to get more compile configuration diversity (stuff that might break on build.chromium.org), but it should still link DRT and webkit_unit_tests faster than having a single binary including all webkit, base, net, webkit_glue, third_party, etc.
Comment 8 Nico Weber 2013-01-31 17:04:14 PST
(In reply to comment #7)
> (In reply to comment #6)
> > Right, but that builds  a single monolithic .so for all of the code in webkit (it even pulls in the tests), right? Is there any point in using this in a standalone webkit build?
> 
> I was just suggesting it as a way to get more compile configuration diversity (stuff that might break on build.chromium.org),

Right, but does is that actually the case? The dependent components that come in via deps are built on the chromium waterfall first before they're rolled into chromium so that doesn't catch anything.

And the only thing with _EXPORT annotations in webkit is the chromium webkit api. Would anything link against that on the webkit waterfall alone? DRT I suppose, so I guess this does make sense.

Carry on then :-)

> but it should still link DRT and webkit_unit_tests faster than having a single binary including all webkit, base, net, webkit_glue, third_party, etc.
Comment 9 Tony Chang 2013-01-31 17:08:22 PST
I was actually thinking of the case where you add a new file to webkit_unit_tests and it doesn't work properly in the components build because of the weird "compile tests into libwebkit.so" behavior you mentioned above.
Comment 10 Alan Cutter 2013-02-01 01:48:26 PST
Created attachment 185982 [details]
Patch
Comment 11 Eric Seidel (no email) 2013-02-01 01:49:58 PST
Comment on attachment 185982 [details]
Patch

OK.
Comment 12 Eric Seidel (no email) 2013-02-01 01:50:38 PST
I thought 2-clause bsd was the new hotness?  (also, nico is really your best reviewer... but he's not a reviewer quite yet!)
Comment 13 Alan Cutter 2013-02-01 01:53:30 PST
(In reply to comment #10)
> Created an attachment (id=185982) [details]
> Patch

The current GCE Chromium Linux debug build bots have already been set up to use clang. This patch just ensures new bots for the queue will similarly be running clang over gcc.

Based on the discussion in the comments I felt the components build configuration was best left for another bug if it's still a good idea.
Comment 14 Alan Cutter 2013-02-01 02:04:37 PST
(In reply to comment #12)
> I thought 2-clause bsd was the new hotness?  (also, nico is really your best reviewer... but he's not a reviewer quite yet!)

I wasn't sure about our licence policy, I'll use 2-clause BSD next time. (:
Comment 15 Nico Weber 2013-02-01 07:49:21 PST
Comment on attachment 185982 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=185982&action=review

> Tools/EWSTools/GoogleComputeEngine/build-cr-linux-debug-ews.sh:59
> +    bash configure-clang-linux.sh $QUEUE_TYPE &&

How often does this run? All our waterfall bots run update.sh after every update as the compiler changes every 1-2 weeks usually. If this script runs only at bot startup, that's not often enough.

> Tools/EWSTools/configure-clang-linux.sh:38
> +export CXX=clang++

Does this bot use goma? If not, you don't need to set PATH, CC and CXX: clang=1 in your GYP_DEFINES will do this automatically.
Comment 16 Alan Cutter 2013-02-03 20:38:26 PST
(In reply to comment #15)
> (From update of attachment 185982 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=185982&action=review
> 
> > Tools/EWSTools/GoogleComputeEngine/build-cr-linux-debug-ews.sh:59
> > +    bash configure-clang-linux.sh $QUEUE_TYPE &&
> 
> How often does this run? All our waterfall bots run update.sh after every update as the compiler changes every 1-2 weeks usually. If this script runs only at bot startup, that's not often enough.
> 
> > Tools/EWSTools/configure-clang-linux.sh:38
> > +export CXX=clang++
> 
> Does this bot use goma? If not, you don't need to set PATH, CC and CXX: clang=1 in your GYP_DEFINES will do this automatically.

Updating the scripts to address these concerns, running into problems with GCE at the moment unfortunately. Will repatch once I can test the new scripts properly.
Comment 17 Alan Cutter 2013-02-06 04:16:29 PST
Created attachment 186821 [details]
Patch
Comment 18 Nico Weber 2013-02-06 08:12:17 PST
Comment on attachment 186821 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=186821&action=review

> Tools/EWSTools/GoogleComputeEngine/build-cr-linux-debug-ews.sh:60
> +screen -t kr ./start-queue.sh -r \\\"configure-clang-linux.sh $QUEUE_TYPE\\\" $QUEUE_TYPE $BOT_ID 10\" &&

Is configure-clang-linux.sh a new file? Do you need to add it to this patch?
Comment 19 Alan Cutter 2013-02-06 14:31:31 PST
Created attachment 186920 [details]
Patch
Comment 20 Alan Cutter 2013-02-06 14:32:09 PST
(In reply to comment #18)
> (From update of attachment 186821 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=186821&action=review
> 
> > Tools/EWSTools/GoogleComputeEngine/build-cr-linux-debug-ews.sh:60
> > +screen -t kr ./start-queue.sh -r \\\"configure-clang-linux.sh $QUEUE_TYPE\\\" $QUEUE_TYPE $BOT_ID 10\" &&
> 
> Is configure-clang-linux.sh a new file? Do you need to add it to this patch?

Thanks for spotting that! It somehow got dropped out between patches.
Comment 21 Tony Chang 2013-02-13 14:08:44 PST
Bump for eric or abarth to review.

FWIW, I've seen two component build rollouts in the past week, but we can tackle that in a different bug.
https://bugs.webkit.org/show_bug.cgi?id=109746
Comment 22 WebKit Review Bot 2013-02-13 14:41:24 PST
Comment on attachment 186920 [details]
Patch

Clearing flags on attachment: 186920

Committed r142802: <http://trac.webkit.org/changeset/142802>
Comment 23 WebKit Review Bot 2013-02-13 14:41:29 PST
All reviewed patches have been landed.  Closing bug.