Bug 57678 - slow or flaky workers test under new-run-webkit-tests
Summary: slow or flaky workers test under new-run-webkit-tests
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: 528+ (Nightly build)
Hardware: PC OS X 10.5
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-04-01 16:01 PDT by Dirk Pranke
Modified: 2011-04-19 20:35 PDT (History)
3 users (show)

See Also:


Attachments
Patch (1.76 KB, patch)
2011-04-19 16:10 PDT, Dmitry Titov
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Dirk Pranke 2011-04-01 16:01:14 PDT
This test seems to take more than six seconds to complete; this is probably not a good thing.
Comment 1 Dmitry Titov 2011-04-04 11:16:57 PDT
I see this test is actually SKIP now in platform/mac/test_expectations.txt.

Is it possible to add more info as for why it is SKIP (a link to failed results would be useful).
Comment 2 Dirk Pranke 2011-04-04 11:33:20 PDT
I only marked this as a SKIP on friday because I was trying to get the mac NRWT bot to run cleanly. In my experience that test was taking anywhere from a few seconds to more than 35 to complete (timeouts).

Unfortunately, there is no mac NRWT bot yet, so I can't point you to logs.

It's easy enough for me to reproduce. Is there something I can get for you that would be helpful?
Comment 3 Dmitry Titov 2011-04-04 11:46:42 PDT
(In reply to comment #2)
> It's easy enough for me to reproduce. Is there something I can get for you that would be helpful?

I run it numerous times (in RWT and NRWT) and it takes <1s with no timeouts. So I'd like to figure out what's different. Do you run debug/release? what OS? is it only repros under NRWT or RWT also? What is exact command you run? etc.
Comment 4 Dirk Pranke 2011-04-04 11:53:30 PDT
(In reply to comment #3)
> (In reply to comment #2)
> > It's easy enough for me to reproduce. Is there something I can get for you that would be helpful?
> 
> I run it numerous times (in RWT and NRWT) and it takes <1s with no timeouts. So I'd like to figure out what's different. Do you run debug/release? what OS? is it only repros under NRWT or RWT also? What is exact command you run? etc.

Mac SL release, upstream mac DumpRenderTree (not chromium), just run as "new-run-webkit-tests --worker-model=inline fast", for example.

I am rebuilding against tip-of-tree right now and will confirm that it still happens.
Comment 5 Dirk Pranke 2011-04-04 12:30:04 PDT
(In reply to comment #4)
> (In reply to comment #3)
> > (In reply to comment #2)
> > > It's easy enough for me to reproduce. Is there something I can get for you that would be helpful?
> > 
> > I run it numerous times (in RWT and NRWT) and it takes <1s with no timeouts. So I'd like to figure out what's different. Do you run debug/release? what OS? is it only repros under NRWT or RWT also? What is exact command you run? etc.
> 
> Mac SL release, upstream mac DumpRenderTree (not chromium), just run as "new-run-webkit-tests --worker-model=inline fast", for example.
> 
> I am rebuilding against tip-of-tree right now and will confirm that it still happens.

Okay, it is still timing out, but only when I run the entire set of tests. Exactly command line is:

% new-run-webkit-tests --print default,config --worker-model=inline
Comment 6 Dirk Pranke 2011-04-04 12:32:12 PDT
Note that the test is definitely flaky, because when NRWT retries the test, it passes.

This suggests to me that some previous test is messing with the state.
Comment 7 Dirk Pranke 2011-04-04 12:52:25 PDT
Hmm. Probably better to mark this as a flaky PASS/TIMEOUT test, since it looks like it doesn't always timeout and that way we'll eventually be able to get better data on it from the flakiness dashboard.
Comment 8 Dirk Pranke 2011-04-04 15:10:57 PDT
interesting .. if you run with --time-out-ms=35000 (to match the ORWT default), several tests are reliably failing with a timeout on notifyDone() inside the test.
Comment 9 Dirk Pranke 2011-04-04 16:04:59 PDT
Okay, after I adjusted the default test timeout on NRWT to match ORWT, I'm seeing the following tests time out occasionally inside DRT themselves, when run as part of the whole test suite:

fast/workers/dedicated-worker-lifecycle.html
fast/workers/shared-worker-frame-lifecycle.html
fast/workers/shared-worker-lifecycle.html
fast/workers/storage/interrupt-database.html
fast/workers/worker-close-more.html
fast/workers/worker-lifecycle.html

To reproduce, run the command line as before. For now I will check in some test expectation suppressions so that things will run cleanly while we hunt this down.

I've updated the subject accordingly as well.
Comment 10 David Levin 2011-04-04 16:47:50 PDT
(In reply to comment #9)
> Okay, after I adjusted the default test timeout on NRWT to match ORWT, I'm seeing the following tests time out occasionally inside DRT themselves, when run as part of the whole test suite:
> 
> fast/workers/dedicated-worker-lifecycle.html
> fast/workers/shared-worker-frame-lifecycle.html
> fast/workers/shared-worker-lifecycle.html
> fast/workers/storage/interrupt-database.html
> fast/workers/worker-close-more.html
> fast/workers/worker-lifecycle.html
> 
> To reproduce, run the command line as before. For now I will check in some test expectation suppressions so that things will run cleanly while we hunt this down.
> 
> I've updated the subject accordingly as well.

So they timeout with NRWT but not with ORWT?
Comment 11 Dmitry Titov 2011-04-13 14:59:49 PDT
Might be related to bug 57090. One of the tests listed in this bug as flaky actually had a race condition in it.
Comment 12 Dmitry Titov 2011-04-19 16:10:19 PDT
Created attachment 90271 [details]
Patch
Comment 13 Dmitry Titov 2011-04-19 16:13:03 PDT
I run:

new-run-webkit-tests --print default,config --worker-model=inline

multiple times and didn't see a single flake on those worker tests after landing fix for bug 67090. I think the better termination sequence and removing the document.write() usage form utility script fixed that.

I did see failures on my machine before that fix.
Comment 14 David Levin 2011-04-19 17:28:58 PDT
Comment on attachment 90271 [details]
Patch

If it has problems, it can easily be reverted.
Comment 15 WebKit Commit Bot 2011-04-19 20:33:04 PDT
The commit-queue encountered the following flaky tests while processing attachment 90271 [details]:

http/tests/misc/favicon-loads-with-icon-loading-override.html bug 58412 (author: alice.liu@apple.com)
http/tests/media/video-load-twice.html bug 51138 (authors: eric.carlson@apple.com and jamesr@chromium.org)
The commit-queue is continuing to process your patch.
Comment 16 WebKit Commit Bot 2011-04-19 20:35:41 PDT
Comment on attachment 90271 [details]
Patch

Clearing flags on attachment: 90271

Committed r84335: <http://trac.webkit.org/changeset/84335>
Comment 17 WebKit Commit Bot 2011-04-19 20:35:47 PDT
All reviewed patches have been landed.  Closing bug.