Bug 142850 – [Win] Activate tests on Windows EWS

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Brent Fulgham]

We have a working EWS system for Windows. Let's turn on tests so we encounter fewer test failures!

[Brent Fulgham]

Created attachment 248993 [details]
Patch

[Csaba Osztrogonác]

What is the testing time of the Win EWS? 

If the tree is red, it will build and run tests twice: build and run 
tests with patch + build and run tests without patch to make sure if
the test failures comes from the patch or the trunk. If there is a 
flakey test, it will build three times.

Nowadays everybody (including Apple employees) ignores test results 
on Apple Windows and Apple employees regularly break the Windows
build against the red EWS bubble and don't fix the broken build after
landing the patch.

I think after slowing down the Windows EWS drastically, folks won't wait for 
the EWS bubble and will land patch before it finished. Now at least they 
can see the red bubble in time if they take notice of Windows port.

[Csaba Osztrogonác]

Comment on attachment 248993 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=248993&action=review

> Tools/Scripts/webkitpy/common/config/ews.json:17
> +        "watchers": [
> +            "[e-mail]"

Is watchers still used for anything? I think it is unused since EWS can't comment bugs and set cq- on failure.
( because folks don't want to get "SPAM", they want to be able break the world without any notification :) )

[Csaba Osztrogonác]

(In reply to comment #3)
> Comment on attachment 248993 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=248993&action=review
> 
> > Tools/Scripts/webkitpy/common/config/ews.json:17
> > +        "watchers": [
> > +            "[e-mail]"
> 
> Is watchers still used for anything? I think it is unused since EWS can't
> comment bugs and set cq- on failure.
> ( because folks don't want to get "SPAM", they want to be able break the
> world without any notification :) )

I was wrong, it still comments bug on test failures and cc the watchers,
but not on build and style failures.  ( I can imagine the angry webkit-dev
mails: "Stop bugzilla SPAMs in case of Windows test failures" :)) )

[Alexey Proskuryakov]

Comment on attachment 248993 [details]
Patch

Ossy, an exciting new development is that Windows tests are all green, and have been maintained for the last couple of weeks. So, it seems reasonable to try and make that stick.

[Csaba Osztrogonác]

(In reply to comment #5)
> Comment on attachment 248993 [details]
> Patch
> 
> Ossy, an exciting new development is that Windows tests are all green, and
> have been maintained for the last couple of weeks. So, it seems reasonable
> to try and make that stick.

Yes, I see that Brent is actively working on keeping the bot green. I just
noticed that I'm very skeptic about it, because the majority of the folks 
don't care about the Windows bot at al. If somebody ignores the red bubble 
which signals build failure, I don't think if the same person will care about
test failures in the future, except the boss emphasizes it is mandatory and 
not optional policy.

I chekced, the testing time is ~37 mins on the Win buildbot now, only one
failing test on the tree would increase the cycle time with ~74 mins, only one
flakey (after retry or timeout or crash) with ~111 mins plus the build times.

But let's see, I always supported the full green tree policy in the past.

[Brent Fulgham]

Committed r181748: <http://trac.webkit.org/changeset/181748>

[Csaba Osztrogonác]

(In reply to comment #7)
> Committed r181748: <http://trac.webkit.org/changeset/181748>

Windows EWS doesn't work, because apache isn't installed:
http://webkit-queues.appspot.com/results/6371421504667648

[Brent Fulgham]

Rats! I'm running down to the lab right now to fix that. Thanks for catching that!

[Alexey Proskuryakov]

httpd seems to still be broken, should we roll out for now?

[Brent Fulgham]

(In reply to comment #10)
> httpd seems to still be broken, should we roll out for now?

No. It's working now, but may take a bit for all the bits to reflect this.

-Brent

[Csaba Osztrogonác]

http tests work, but others are still flakey

[Brent Fulgham]

I'm going to roll this out because tests seem to be failing, even though they work fine on the main test bots. Something must be misconfigured.

[Brent Fulgham]

Temporarily rolled out in r181801 <http://trac.webkit.org/changeset/181801>.

[Csaba Osztrogonác]

reopen

[Fujii Hironori]

bug 269676 resolves this.

[Radar WebKit Bug Importer]

<rdar://problem/131498099>