Bug 93472 – Introduce cr-android bots in the EWS system

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Peter Beverloo]

Since Android is slightly different than Linux Chromium (which already has a bot in the EWS), we'd like to add a cr-android builder as well. Two VMs are available for this, though we can increase capacity if necessary.

[Peter Beverloo]

Created attachment 157207 [details]
Patch

[Peter Beverloo]

I've got a few remaining questions about attaching the bots themselves:

- How noisy is the Bugzilla account necessary for the bot? I assume it'll report the build failures? I've set up "[e-mail]" as "Peter Beverloo (cr-android EWS)" for now.
- Can you give EditBugs to "[e-mail]", please?
- There is a configure-svn-auth.sh script in the EWSTools folder. I presume this is only for the commit queue?

Is this correct?: After initializing the checkout in /mnt/git/webkit (optionally though the create-webkit-git script), running "start-queue.sh chromium-android-ews 1" will attach the first bot, and "..ews 2" will start the second bot.

[Adam Barth]

> - How noisy is the Bugzilla account necessary for the bot? I assume it'll report the build failures? I've set up "[e-mail]" as "Peter Beverloo (cr-android EWS)" for now.

I would recommend using a throw-away account instead.  They tend to get CCed on a lot of bugs.  You can then "watch" the account using Bugzilla to get email when something fails.

> - Can you give EditBugs to "[e-mail]", please?

Done.

> - There is a configure-svn-auth.sh script in the EWSTools folder. I presume this is only for the commit queue?

Yes.

> Is this correct?: After initializing the checkout in /mnt/git/webkit (optionally though the create-webkit-git script), running "start-queue.sh chromium-android-ews 1" will attach the first bot, and "..ews 2" will start the second bot.

Yes, but I would use a more descriptive bot ID than just "1".  The bot ID is just a label.  Perhaps cr-android-01 and cr-android-02?  You can see the bot IDs that folks are using today in the first column of http://queues.webkit.org/active-bots

[Adam Barth]

Comment on attachment 157207 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=157207&action=review

> Tools/QueueStatusServer/model/queues.py:43
> +        "cr-android-ews",

You probably also want to add it to http://trac.webkit.org/browser/trunk/Tools/QueueStatusServer/model/queues.py#L38

[Adam Barth]

> > Is this correct?: After initializing the checkout in /mnt/git/webkit (optionally though the create-webkit-git script), running "start-queue.sh chromium-android-ews 1" will attach the first bot, and "..ews 2" will start the second bot.
> 
> Yes, but I would use a more descriptive bot ID than just "1".  The bot ID is just a label.  Perhaps cr-android-01 and cr-android-02?  You can see the bot IDs that folks are using today in the first column of http://queues.webkit.org/active-bots

It looks like you named the bot cr-android-ews, which is better, so the command to start the queue would be:

start-queue.sh cr-android-ews cr-android-01

or whatever you like for the second argument.

The only other tricky thing is configuring your bugzilla credentials in git.  You can see how that's done in the http://trac.webkit.org/browser/trunk/Tools/EWSTools/cold-boot.sh script we use on the cr-linux-ews.  Basically, you add a section to the end of the .git/config file that looks like this:

[bugzilla]
	        username = $3
	        password = $4

[Peter Beverloo]

(In reply to comment #3)
> > - How noisy is the Bugzilla account necessary for the bot? I assume it'll report the build failures? I've set up "[e-mail]" as "Peter Beverloo (cr-android EWS)" for now.
> 
> I would recommend using a throw-away account instead.  They tend to get CCed on a lot of bugs.  You can then "watch" the account using Bugzilla to get email when something fails.

I already set up Gmail filters and adjusted the account's e-mail preferences on Bugzilla. It should be fine, I'm already CC'ed on the majority of bugs and the "+ews" bit is set up to only forward real e-mail. Benefit of using this is that people have an idea about who to contact if they're puzzled about the failure.

> 
> > - Can you give EditBugs to "[e-mail]", please?
> 
> Done.
> 
> > - There is a configure-svn-auth.sh script in the EWSTools folder. I presume this is only for the commit queue?
> 
> Yes.
> 
> > Is this correct?: After initializing the checkout in /mnt/git/webkit (optionally though the create-webkit-git script), running "start-queue.sh chromium-android-ews 1" will attach the first bot, and "..ews 2" will start the second bot.
> 
> Yes, but I would use a more descriptive bot ID than just "1".  The bot ID is just a label.  Perhaps cr-android-01 and cr-android-02?  You can see the bot IDs that folks are using today in the first column of http://queues.webkit.org/active-bots

Right. Thanks!

[Peter Beverloo]

(In reply to comment #4)
> (From update of attachment 157207 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=157207&action=review
> 
> > Tools/QueueStatusServer/model/queues.py:43
> > +        "cr-android-ews",
> 
> You probably also want to add it to http://trac.webkit.org/browser/trunk/Tools/QueueStatusServer/model/queues.py#L38

Huh? I did..

[Peter Beverloo]

(In reply to comment #5)
> It looks like you named the bot cr-android-ews, which is better, so the command to start the queue would be:
> 
> start-queue.sh cr-android-ews cr-android-01
> 
> or whatever you like for the second argument.
> 
> The only other tricky thing is configuring your bugzilla credentials in git.  You can see how that's done in the http://trac.webkit.org/browser/trunk/Tools/EWSTools/cold-boot.sh script we use on the cr-linux-ews.  Basically, you add a section to the end of the .git/config file that looks like this:
> 
> [bugzilla]
>             username = $3
>             password = $4

Yes, I read about that on the EWS wiki page. Thanks!

[Adam Barth]

Comment on attachment 157207 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=157207&action=review

>>> Tools/QueueStatusServer/model/queues.py:43
>>> +        "cr-android-ews",
>> 
>> You probably also want to add it to http://trac.webkit.org/browser/trunk/Tools/QueueStatusServer/model/queues.py#L38
> 
> Huh? I did..

You're totally right!

[Adam Barth]

Let me know when you want me to deploy the QueueStatusServer change.  Once we do that, the cr-android-ews will start to queue up patches to work on, so we should wait on that until you're ready to start processing them.

[WebKit Review Bot]

Comment on attachment 157207 [details]
Patch

Clearing flags on attachment: 157207

Committed r125038: <http://trac.webkit.org/changeset/125038>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.

[Peter Beverloo]

(In reply to comment #10)
> Let me know when you want me to deploy the QueueStatusServer change.  Once we do that, the cr-android-ews will start to queue up patches to work on, so we should wait on that until you're ready to start processing them.

After bug 93489 lands, we should be good to go. It'd be great if you could enable it just before going to bed (i.e. late PM PST -- I don't have to be around yet). That way, I can pick it up tomorrow morning GMT, and it can "test drive" during GMT hours.

If things really go terribly wrong, it would also give us the ability to disable it without causing too much disturbance in PST times.

[Adam Barth]

Will do.

[Wei James (wistoch)]

(In reply to comment #13)
> (In reply to comment #10)
> > Let me know when you want me to deploy the QueueStatusServer change.  Once we do that, the cr-android-ews will start to queue up patches to work on, so we should wait on that until you're ready to start processing them.
> 
> After bug 93489 lands, we should be good to go. It'd be great if you could enable it just before going to bed (i.e. late PM PST -- I don't have to be around yet). That way, I can pick it up tomorrow morning GMT, and it can "test drive" during GMT hours.
> 
> If things really go terribly wrong, it would also give us the ability to disable it without causing too much disturbance in PST times.

peter, glad to see that cr-android bots landed EWS. 

do you guys have any plan to enable IA android bots in future? thanks

[Adam Barth]

It's alive!  http://queues.webkit.org/queue-status/cr-android-ews

[Peter Beverloo]

(In reply to comment #15)
> (In reply to comment #13)
> > (In reply to comment #10)
> > > Let me know when you want me to deploy the QueueStatusServer change.  Once we do that, the cr-android-ews will start to queue up patches to work on, so we should wait on that until you're ready to start processing them.
> > 
> > After bug 93489 lands, we should be good to go. It'd be great if you could enable it just before going to bed (i.e. late PM PST -- I don't have to be around yet). That way, I can pick it up tomorrow morning GMT, and it can "test drive" during GMT hours.
> > 
> > If things really go terribly wrong, it would also give us the ability to disable it without causing too much disturbance in PST times.
> 
> peter, glad to see that cr-android bots landed EWS. 
> 
> do you guys have any plan to enable IA android bots in future? thanks

As far as I'm aware, the only x86 bot we have right now is still downstream (i.e. not on Chromium or WebKit). I'll inquire about the general plans to have an x86 bot for the Chromium side of things. If there are any plans, it makes sense to make the same move at Android.

The worry I have is that it won't necessarily be a priority for us, and that it may go outdated, or that it may take a while for people to look at it when it's broken. Furthermore, setting up both a builder and tester will make the WebKit waterfall significantly more crowded.

Hypothetically speaking, do you think that a builder and tester on the same machine (thus slower) would work? Are there significant differences in compile warnings and errors/test results? Maybe you (Intel) could help out in maintaining it as well, when build or test errors occur?

[Wei James (wistoch)]

(In reply to comment #17)
> (In reply to comment #15)
> > (In reply to comment #13)
> > > (In reply to comment #10)
> > > > Let me know when you want me to deploy the QueueStatusServer change.  Once we do that, the cr-android-ews will start to queue up patches to work on, so we should wait on that until you're ready to start processing them.
> > > 
> > > After bug 93489 lands, we should be good to go. It'd be great if you could enable it just before going to bed (i.e. late PM PST -- I don't have to be around yet). That way, I can pick it up tomorrow morning GMT, and it can "test drive" during GMT hours.
> > > 
> > > If things really go terribly wrong, it would also give us the ability to disable it without causing too much disturbance in PST times.
> > 
> > peter, glad to see that cr-android bots landed EWS. 
> > 
> > do you guys have any plan to enable IA android bots in future? thanks
> 
> As far as I'm aware, the only x86 bot we have right now is still downstream (i.e. not on Chromium or WebKit). I'll inquire about the general plans to have an x86 bot for the Chromium side of things. If there are any plans, it makes sense to make the same move at Android.
> 
> The worry I have is that it won't necessarily be a priority for us, and that it may go outdated, or that it may take a while for people to look at it when it's broken. Furthermore, setting up both a builder and tester will make the WebKit waterfall significantly more crowded.
> 
> Hypothetically speaking, do you think that a builder and tester on the same machine (thus slower) would work? Are there significant differences in compile warnings and errors/test results? Maybe you (Intel) could help out in maintaining it as well, when build or test errors occur?

peter, thanks for sharing the information and your concern with us. 
Very appreciate your help to inquire the plan for x86 bot. 

surely we(Intel has a team working on Chromium Android Browser on IA) can help to maintain the x86 bots when build/test errors occur. In fact, we have an internal buildbot for chromium android x86 and will do daily unit tests to catch failure. 

until now, we caught three failures for chromium android x86 specifically, which will not break ARM chromium android build/test but will break x86 build/test. 
 
http://chromiumcodereview.appspot.com/10384025/
http://chromiumcodereview.appspot.com/10332192/
http://chromiumcodereview.appspot.com/10827176/

The difference in compile warning and errors and test results are not so significant and so I believe builder and tester on the same machine is ok.