Bug 28624 – REGRESSION: media/video-pause-empty-events.html is occasionally timing out on bots

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Eric Seidel (no email)]

REGRESSION: media/video-pause-empty-events.html is occasionally timing out on bots

http://build.webkit.org/results/Leopard%20Intel%20Release%20(Tests)/r47639%20(4313)/results.html

Seems to have been for just the last few hours.  Keeps popping up every other build or so.  I've not yet determined which revision started the problem.

[Eric Seidel (no email)]

http://build.webkit.org/builders/Leopard%20Intel%20Release%20%28Tests%29/builds/4306

is the first build which seems to have seen the timeout.

[Eric Seidel (no email)]

Actually:
http://build.webkit.org/builders/Leopard%20Intel%20Release%20%28Tests%29/builds/4295
is an earlier build from Thu Aug 20 21:27:41 2009 which saw the timeout.

[Eric Carlson]

That build was triggered by ollie's JSC variable count change: http://trac.webkit.org/changeset/47620. Don't know how that would make this test timeout.

[Eric Seidel (no email)]

I don't think it's related to Oliver's change at all.  I think this test is just flakey.  What caused the flake, I don't know, but something before Oliver's change.

[Eric Seidel (no email)]

This could have the same root cause as bug 28845.

[Eric Seidel (no email)]

Just saw media/video-source-error.html timeout.
http://build.webkit.org/results/Leopard%20Intel%20Release%20(Tests)/r48163%20(4792)/results.html
Possibly the same cause.

[Eric Seidel (no email)]

Seen again landing bug 29038.media/video-source-error.html seems to be the most common timeout seen by the commit-bot.

[Eric Seidel (no email)]

Another timeout of media/video-source-error.html seen while landing bug 26700.  I'll attach the timeout file.

[Eric Seidel (no email)]

Created attachment 39306 [details]
spin report from most recent hang from landing bug 26700.  This looks like these hangs are caused by bug 28624.

[Eric Seidel (no email)]

Yet another timeout when landing bug 29001. :(

[Eric Seidel (no email)]

Yet another timeout landing bug 29065.

[Simon Fraser (smfr)]

I'm running the media and compositing tests hundreds of times (via the patch in bug 29263), and haven't get crashed, on 10.5.8.

[Eric Seidel (no email)]

media/video-source-type.html is a new one, probably the same root cause:
http://build.webkit.org/results/Leopard%20Intel%20Release%20(Tests)/r48683%20(5277)/results.html

[Eric Seidel (no email)]

video-loop and video-source again just now:
http://build.webkit.org/results/Leopard%20Intel%20Release%20(Tests)/r48687%20(5281)/results.html

Looks like we have a few more media tests to skip. :(

[Eric Seidel (no email)]

Simon suggests that we disable hardware compositiing until this bug can be fixed in QuickTime:

defaults write .... WebKitAcceleratedCompositingEnabled -bool NO

We will need to make a patch to DRT to set that default on leopard builds (at least until a new version of quicktime is available).

Eric, perhaps you could help me with the quicktime version check?  I'd like to make it only turn it off for bad versions of QuickTime.

[Eric Seidel (no email)]

By eric I meant simon of course. :)

[Eric Seidel (no email)]

Wow, this disaster never ends. :(

So I have a working patch which disables HW Compositing and makes the media tests reliably pass.

However, now I just saw this test fail:
http/tests/messaging/cross-domain-message-event-dispatch.html -> crashed

With a crash under:
Thread 8 Crashed:
0   ...uickTimeStreaming.component	0x1d4a939e QTSInetCachedDownload_SetProperty + 315
1   ...uickTimeStreaming.component	0x1d4af2ca QTSInetDownload_TransactionNotification + 324
2   ...uickTimeStreaming.component	0x1d4b28ca QTSInetTransaction_ProcessResponseHeader + 1659
3   ...uickTimeStreaming.component	0x1d4b39ea QTSInetTransaction_ReadStreamCallback + 699
4   com.apple.CoreFoundation      	0x955cab1b _signalEventSync + 91
5   com.apple.CoreFoundation      	0x955cabbe _cfstream_solo_signalEventSync + 126
6   com.apple.CoreFoundation      	0x955cd057 CFReadStreamSignalEvent + 39
7   com.apple.CFNetwork           	0x91d00e00 HTTPReadStream::streamEvent(unsigned long) + 68

I'll attach the crash log to the bug.  It may be completely unrelated.

[Eric Seidel (no email)]

Created attachment 40129 [details]
Possibly unrelated crashlog seen with HW compositing disabled (during http/tests/messaging/cross-domain-message-event-dispatch.html)

[Eric Seidel (no email)]

My "disaster" comment was intentional hyperbole, but I realize now it might not read that way. ;)  All and all this mess has been annoying, but certainly not a "disaster".

[Eric Seidel (no email)]

Also, these two tests fail with HW compositing disabled:
fast/media/mq-transform-02.html
fast/media/mq-transform-03.html

I guess I could add special "passing" test results for Leopard since I assume these failures just need to be tracked as new bugs.

[Eric Seidel (no email)]

Created attachment 40140 [details]
Proposed solution for disabling hardware compositing on Leopard

[Eric Seidel (no email)]

Simon: I'm very interested in your comments as to this approach.

1.  If you think this is a good way to go about stopping the crashes.
2.  If I did my OS version check correctly.
3.  If I did my QT version check correctly (and if 7.6.5 is a reasonable revision to assume this might get fixed in).

Thanks!

[Eric Seidel (no email)]

Furthermore, I only saw the crash seen in https://bugs.webkit.org/attachment.cgi?id=40129 once.  I've run the layout tests 3 times with this patch.  I expect we might be trading one bout of flakiness for another by disabling this if the crash from https://bugs.webkit.org/attachment.cgi?id=40129 reproduces again.  But hopefully we're moving one step further towards stability.

[Eric Seidel (no email)]

4.  If you would rather I skip fast/media/mq-transform-02.html and fast/media/mq-transform-03.html on mac-leopard instead of checking in new results, let me know.

[Eric Seidel (no email)]

I will need to slightly amend my change or fix bug 29751 or others may be bitten by bug 29751.

[Eric Seidel (no email)]

Created attachment 40151 [details]
Set setAcceleratedCompositingEnabled:YES to work around bug 29751

[Eric Seidel (no email)]

Comment on attachment 40151 [details]
Set setAcceleratedCompositingEnabled:YES to work around bug 29751

Thanks for the review.  I'll close out the rest of the bugs after we verify these crashes are gone for a couple days.

[WebKit Commit Bot]

Comment on attachment 40151 [details]
Set setAcceleratedCompositingEnabled:YES to work around bug 29751

Rejecting patch 40151 from commit-queue.

Patch https://bugs.webkit.org/attachment.cgi?id=40151 from bug 28624 failed to download and apply.

[WebKit Commit Bot]

Committed r48833: <http://trac.webkit.org/changeset/48833>

[Eric Seidel (no email)]

*** Bug 29624 has been marked as a duplicate of this bug. ***