Bug 57678 – slow or flaky workers test under new-run-webkit-tests

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Dirk Pranke]

This test seems to take more than six seconds to complete; this is probably not a good thing.

[Dmitry Titov]

I see this test is actually SKIP now in platform/mac/test_expectations.txt.

Is it possible to add more info as for why it is SKIP (a link to failed results would be useful).

[Dirk Pranke]

I only marked this as a SKIP on friday because I was trying to get the mac NRWT bot to run cleanly. In my experience that test was taking anywhere from a few seconds to more than 35 to complete (timeouts).

Unfortunately, there is no mac NRWT bot yet, so I can't point you to logs.

It's easy enough for me to reproduce. Is there something I can get for you that would be helpful?

[Dmitry Titov]

(In reply to comment #2)
> It's easy enough for me to reproduce. Is there something I can get for you that would be helpful?

I run it numerous times (in RWT and NRWT) and it takes <1s with no timeouts. So I'd like to figure out what's different. Do you run debug/release? what OS? is it only repros under NRWT or RWT also? What is exact command you run? etc.

[Dirk Pranke]

(In reply to comment #3)
> (In reply to comment #2)
> > It's easy enough for me to reproduce. Is there something I can get for you that would be helpful?
> 
> I run it numerous times (in RWT and NRWT) and it takes <1s with no timeouts. So I'd like to figure out what's different. Do you run debug/release? what OS? is it only repros under NRWT or RWT also? What is exact command you run? etc.

Mac SL release, upstream mac DumpRenderTree (not chromium), just run as "new-run-webkit-tests --worker-model=inline fast", for example.

I am rebuilding against tip-of-tree right now and will confirm that it still happens.

[Dirk Pranke]

(In reply to comment #4)
> (In reply to comment #3)
> > (In reply to comment #2)
> > > It's easy enough for me to reproduce. Is there something I can get for you that would be helpful?
> > 
> > I run it numerous times (in RWT and NRWT) and it takes <1s with no timeouts. So I'd like to figure out what's different. Do you run debug/release? what OS? is it only repros under NRWT or RWT also? What is exact command you run? etc.
> 
> Mac SL release, upstream mac DumpRenderTree (not chromium), just run as "new-run-webkit-tests --worker-model=inline fast", for example.
> 
> I am rebuilding against tip-of-tree right now and will confirm that it still happens.

Okay, it is still timing out, but only when I run the entire set of tests. Exactly command line is:

% new-run-webkit-tests --print default,config --worker-model=inline

[Dirk Pranke]

Note that the test is definitely flaky, because when NRWT retries the test, it passes.

This suggests to me that some previous test is messing with the state.

[Dirk Pranke]

Hmm. Probably better to mark this as a flaky PASS/TIMEOUT test, since it looks like it doesn't always timeout and that way we'll eventually be able to get better data on it from the flakiness dashboard.

[Dirk Pranke]

interesting .. if you run with --time-out-ms=35000 (to match the ORWT default), several tests are reliably failing with a timeout on notifyDone() inside the test.

[Dirk Pranke]

Okay, after I adjusted the default test timeout on NRWT to match ORWT, I'm seeing the following tests time out occasionally inside DRT themselves, when run as part of the whole test suite:

fast/workers/dedicated-worker-lifecycle.html
fast/workers/shared-worker-frame-lifecycle.html
fast/workers/shared-worker-lifecycle.html
fast/workers/storage/interrupt-database.html
fast/workers/worker-close-more.html
fast/workers/worker-lifecycle.html

To reproduce, run the command line as before. For now I will check in some test expectation suppressions so that things will run cleanly while we hunt this down.

I've updated the subject accordingly as well.

[David Levin]

(In reply to comment #9)
> Okay, after I adjusted the default test timeout on NRWT to match ORWT, I'm seeing the following tests time out occasionally inside DRT themselves, when run as part of the whole test suite:
> 
> fast/workers/dedicated-worker-lifecycle.html
> fast/workers/shared-worker-frame-lifecycle.html
> fast/workers/shared-worker-lifecycle.html
> fast/workers/storage/interrupt-database.html
> fast/workers/worker-close-more.html
> fast/workers/worker-lifecycle.html
> 
> To reproduce, run the command line as before. For now I will check in some test expectation suppressions so that things will run cleanly while we hunt this down.
> 
> I've updated the subject accordingly as well.

So they timeout with NRWT but not with ORWT?

[Dmitry Titov]

Might be related to bug 57090. One of the tests listed in this bug as flaky actually had a race condition in it.

[Dmitry Titov]

Created attachment 90271 [details]
Patch

[Dmitry Titov]

I run:

new-run-webkit-tests --print default,config --worker-model=inline

multiple times and didn't see a single flake on those worker tests after landing fix for bug 67090. I think the better termination sequence and removing the document.write() usage form utility script fixed that.

I did see failures on my machine before that fix.

[David Levin]

Comment on attachment 90271 [details]
Patch

If it has problems, it can easily be reverted.

[WebKit Commit Bot]

The commit-queue encountered the following flaky tests while processing attachment 90271 [details]:

http/tests/misc/favicon-loads-with-icon-loading-override.html bug 58412 (author: [e-mail])
http/tests/media/video-load-twice.html bug 51138 (authors: [e-mail] and [e-mail])
The commit-queue is continuing to process your patch.

[WebKit Commit Bot]

Comment on attachment 90271 [details]
Patch

Clearing flags on attachment: 90271

Committed r84335: <http://trac.webkit.org/changeset/84335>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.