Bug 149816 - http/tests/media/media-source/SourceBuffer-abort-updating.html is flaky
Summary: http/tests/media/media-source/SourceBuffer-abort-updating.html is flaky
Status: REOPENED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Jer Noble
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-10-05 13:19 PDT by Ryan Haddad
Modified: 2015-10-13 09:40 PDT (History)
6 users (show)

See Also:


Attachments
Patch (1.25 KB, patch)
2015-10-05 15:57 PDT, Ryan Haddad
no flags Details | Formatted Diff | Diff
Patch (1.35 KB, patch)
2015-10-08 14:37 PDT, Ryan Haddad
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ryan Haddad 2015-10-05 13:19:56 PDT
http/tests/media/media-source/SourceBuffer-abort-updating.html is flaky, frequently crashing or timing out during test runs

http://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2Fmedia%2Fmedia-source%2FSourceBuffer-abort-updating.html
Comment 1 Ryan Haddad 2015-10-05 15:54:25 PDT
Correction: the crashes are only on GTK. Mac builds get timeouts and failures.
Comment 2 Ryan Haddad 2015-10-05 15:57:21 PDT
Created attachment 262473 [details]
Patch
Comment 3 Ryan Haddad 2015-10-05 16:37:23 PDT
http/tests/media/video-load-suspend.html timed out as well on the following build:
https://build.webkit.org/builders/Apple%20El%20Capitan%20Debug%20WK1%20(Tests)/builds/113

http/tests/media/media-source/mediasource-remove.html crashed on the following build:
https://build.webkit.org/builders/Apple%20Yosemite%20Release%20WK1%20(Tests)/builds/8830

Jer, is there anything bad hiding here?
Comment 4 Jer Noble 2015-10-06 09:32:50 PDT
(In reply to comment #3)
> http/tests/media/video-load-suspend.html timed out as well on the following
> build:
> https://build.webkit.org/builders/
> Apple%20El%20Capitan%20Debug%20WK1%20(Tests)/builds/113
> 
> http/tests/media/media-source/mediasource-remove.html crashed on the
> following build:
> https://build.webkit.org/builders/Apple%20Yosemite%20Release%20WK1%20(Tests)/
> builds/8830

This is a crash in an underlying framework which was fixed in El Capitan.

> Jer, is there anything bad hiding here?

Can't say for sure. We'd need to get a spin log of the failing test to know for sure.  It could be that the test is crashing due to (one or more of) the above bug(s), but showing up as a timeout.
Comment 5 Jer Noble 2015-10-06 09:37:40 PDT
(In reply to comment #4)
> (In reply to comment #3)
> > http/tests/media/video-load-suspend.html timed out as well on the following
> > build:
> > https://build.webkit.org/builders/
> > Apple%20El%20Capitan%20Debug%20WK1%20(Tests)/builds/113
> > 
> > http/tests/media/media-source/mediasource-remove.html crashed on the
> > following build:
> > https://build.webkit.org/builders/Apple%20Yosemite%20Release%20WK1%20(Tests)/
> > builds/8830
> 
> This is a crash in an underlying framework which was fixed in El Capitan.
> 
> > Jer, is there anything bad hiding here?
> 
> Can't say for sure. We'd need to get a spin log of the failing test to know
> for sure.  It could be that the test is crashing due to (one or more of) the
> above bug(s), but showing up as a timeout.

Oh I see, it's timing out on El Capitan as well as Yosemite.
Comment 6 WebKit Commit Bot 2015-10-07 09:32:24 PDT
Comment on attachment 262473 [details]
Patch

Rejecting attachment 262473 [details] from commit-queue.

Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-01', 'apply-attachment', '--no-update', '--non-interactive', 262473, '--port=mac']" exit_code: 2 cwd: /Volumes/Data/EWS/WebKit

Last 500 characters of output:
, u'Darin Adler']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

Parsed 2 diffs from patch file(s).
patching file LayoutTests/ChangeLog
Hunk #1 succeeded at 1 with fuzz 3.
patching file LayoutTests/platform/mac/TestExpectations
Hunk #1 FAILED at 1331.
1 out of 1 hunk FAILED -- saving rejects to file LayoutTests/platform/mac/TestExpectations.rej

Failed to run "[u'/Volumes/Data/EWS/WebKit/Tools/Scripts/svn-apply', '--force', '--reviewer', u'Darin Adler']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

Full output: http://webkit-queues.webkit.org/results/254497
Comment 7 Ryan Haddad 2015-10-08 14:37:03 PDT
Created attachment 262711 [details]
Patch
Comment 8 WebKit Commit Bot 2015-10-08 15:53:48 PDT
Comment on attachment 262711 [details]
Patch

Clearing flags on attachment: 262711

Committed r190753: <http://trac.webkit.org/changeset/190753>
Comment 9 WebKit Commit Bot 2015-10-08 15:53:53 PDT
All reviewed patches have been landed.  Closing bug.
Comment 10 Ryan Haddad 2015-10-08 15:54:36 PDT
Leaving open for investigation
Comment 11 Myles C. Maxfield 2015-10-08 17:54:37 PDT
Did this work? After this patch, http/tests/media/media-source/SourceBuffer-abort-updating.html seems to always fail on Mavericks.

https://build.webkit.org/builders/Apple%20Mavericks%20Release%20WK2%20(Tests)?numbuilds=50
Comment 12 Myles C. Maxfield 2015-10-08 17:57:11 PDT
Seems that your new line is being used instead of the existing

[ Mavericks ] http/tests/media/media-source/ [ WontFix ]
Comment 13 BJ Burg 2015-10-08 22:19:41 PDT
Comment on attachment 262711 [details]
Patch

This expectation accidentally re-enables the test for Mavericks, where we have set !ENABLE(MEDIA_SESSION), so the test always fails.
Comment 14 BJ Burg 2015-10-08 22:26:49 PDT
Committed r190777: <http://trac.webkit.org/changeset/190777>