Bug 161723 – [Media Source] Import updated w3c/web-platform-tests and rebaseline

• Status: RESOLVED
• Resolution: FIXED
• Component: Media

[Jer Noble]

[Media Source] Import updated w3c/web-platform-tests and rebaseline

[Jer Noble]

Created attachment 288338 [details]
Patch

[Build Bot]

Comment on attachment 288338 [details]
Patch

Attachment 288338 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/2037124

Number of test failures exceeded the failure limit.

[Build Bot]

Created attachment 288347 [details]
Archive of layout-test-results from ews126 for ios-simulator-elcapitan-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews126  Port: ios-simulator-elcapitan-wk2  Platform: Mac OS X 10.11.5

[Build Bot]

Comment on attachment 288338 [details]
Patch

Attachment 288338 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/2037151

New failing tests:
imported/w3c/web-platform-tests/media-source/mediasource-addsourcebuffer.html
imported/w3c/web-platform-tests/media-source/mediasource-remove.html
imported/w3c/web-platform-tests/media-source/mediasource-sourcebuffer-mode.html
imported/w3c/web-platform-tests/media-source/mediasource-is-type-supported.html
http/tests/media/media-source/mediasource-append-stream.html
imported/w3c/web-platform-tests/media-source/mediasource-append-buffer.html
http/tests/media/media-source/mediasource-play-then-seek-back-with-remote-control.html
imported/w3c/web-platform-tests/media-source/mediasource-config-change-mp4-v-bitrate.html
imported/w3c/web-platform-tests/media-source/mediasource-play.html

[Build Bot]

Created attachment 288350 [details]
Archive of layout-test-results from ews107 for mac-yosemite-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews107  Port: mac-yosemite-wk2  Platform: Mac OS X 10.10.5

[Build Bot]

Comment on attachment 288338 [details]
Patch

Attachment 288338 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/2037160

New failing tests:
imported/w3c/web-platform-tests/media-source/mediasource-addsourcebuffer.html
imported/w3c/web-platform-tests/media-source/mediasource-remove.html
imported/w3c/web-platform-tests/media-source/mediasource-sourcebuffer-mode.html
imported/w3c/web-platform-tests/media-source/mediasource-is-type-supported.html
http/tests/media/media-source/mediasource-append-stream.html
imported/w3c/web-platform-tests/media-source/mediasource-append-buffer.html
http/tests/media/media-source/mediasource-play-then-seek-back-with-remote-control.html
imported/w3c/web-platform-tests/media-source/mediasource-config-change-mp4-v-bitrate.html
imported/w3c/web-platform-tests/media-source/mediasource-play.html

[Build Bot]

Created attachment 288353 [details]
Archive of layout-test-results from ews113 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews113  Port: mac-yosemite  Platform: Mac OS X 10.10.5

[Build Bot]

Comment on attachment 288338 [details]
Patch

Attachment 288338 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/2037210

New failing tests:
imported/w3c/web-platform-tests/media-source/mediasource-addsourcebuffer.html
imported/w3c/web-platform-tests/media-source/mediasource-remove.html
imported/w3c/web-platform-tests/media-source/mediasource-sourcebuffer-mode.html
imported/w3c/web-platform-tests/media-source/mediasource-is-type-supported.html
http/tests/media/media-source/mediasource-append-stream.html
imported/w3c/web-platform-tests/media-source/mediasource-append-buffer.html
http/tests/media/media-source/mediasource-play-then-seek-back-with-remote-control.html
imported/w3c/web-platform-tests/media-source/mediasource-config-change-mp4-v-bitrate.html
imported/w3c/web-platform-tests/media-source/mediasource-play.html

[Build Bot]

Created attachment 288356 [details]
Archive of layout-test-results from ews101 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews101  Port: mac-yosemite  Platform: Mac OS X 10.10.5

[Jer Noble]

Created attachment 288412 [details]
Patch

[Build Bot]

Comment on attachment 288412 [details]
Patch

Attachment 288412 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/2042503

New failing tests:
imported/w3c/web-platform-tests/media-source/mediasource-addsourcebuffer.html
imported/w3c/web-platform-tests/media-source/mediasource-remove.html
imported/w3c/web-platform-tests/media-source/mediasource-sourcebuffer-mode.html
imported/w3c/web-platform-tests/media-source/mediasource-is-type-supported.html
imported/w3c/web-platform-tests/media-source/mediasource-append-buffer.html
http/tests/media/media-source/mediasource-play-then-seek-back-with-remote-control.html
imported/w3c/web-platform-tests/media-source/mediasource-config-change-mp4-v-bitrate.html
imported/w3c/web-platform-tests/media-source/mediasource-play.html

[Build Bot]

Created attachment 288418 [details]
Archive of layout-test-results from ews102 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews102  Port: mac-yosemite  Platform: Mac OS X 10.10.5

[Build Bot]

Comment on attachment 288412 [details]
Patch

Attachment 288412 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/2042532

Number of test failures exceeded the failure limit.

[Build Bot]

Created attachment 288420 [details]
Archive of layout-test-results from ews125 for ios-simulator-elcapitan-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews125  Port: ios-simulator-elcapitan-wk2  Platform: Mac OS X 10.11.5

[Build Bot]

Comment on attachment 288412 [details]
Patch

Attachment 288412 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/2042554

New failing tests:
imported/w3c/web-platform-tests/media-source/mediasource-addsourcebuffer.html
imported/w3c/web-platform-tests/media-source/mediasource-remove.html
imported/w3c/web-platform-tests/media-source/mediasource-sourcebuffer-mode.html
imported/w3c/web-platform-tests/media-source/mediasource-is-type-supported.html
imported/w3c/web-platform-tests/media-source/mediasource-append-buffer.html
http/tests/media/media-source/mediasource-play-then-seek-back-with-remote-control.html
imported/w3c/web-platform-tests/media-source/mediasource-config-change-mp4-v-bitrate.html
imported/w3c/web-platform-tests/media-source/mediasource-play.html

[Build Bot]

Created attachment 288421 [details]
Archive of layout-test-results from ews107 for mac-yosemite-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews107  Port: mac-yosemite-wk2  Platform: Mac OS X 10.10.5

[Build Bot]

Comment on attachment 288412 [details]
Patch

Attachment 288412 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/2042556

New failing tests:
imported/w3c/web-platform-tests/media-source/mediasource-addsourcebuffer.html
imported/w3c/web-platform-tests/media-source/mediasource-remove.html
imported/w3c/web-platform-tests/media-source/mediasource-sourcebuffer-mode.html
imported/w3c/web-platform-tests/media-source/mediasource-is-type-supported.html
imported/w3c/web-platform-tests/media-source/mediasource-append-buffer.html
http/tests/media/media-source/mediasource-play-then-seek-back-with-remote-control.html
imported/w3c/web-platform-tests/media-source/mediasource-config-change-mp4-v-bitrate.html
imported/w3c/web-platform-tests/media-source/mediasource-play.html

[Build Bot]

Created attachment 288422 [details]
Archive of layout-test-results from ews116 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews116  Port: mac-yosemite  Platform: Mac OS X 10.10.5

[youenn fablet]

As I understand it, you moved the files from http/tests/media to imported/w3c/web-platform-tests. That seems good to me.

It would be great if a follow-up patch could do the following:
- Delete LayoutTests/imported/w3c/web-platform-tests/media-source
- Update LayoutTests/imported/w3c/resources/ImportExpectations to comment/remove the web-platform-tests/media-source line
- Run 'Tools/Scripts/import-w3c-tests -t web-platform-tests/media-source' to reimport up-to-date media source tests

The first step allows removing potentially obsolete tests.
If some of these tests are still useful, they should be moved back to http/tests, as in a later refresh, I may remove them without knowing that they are actually useful.

[Jer Noble]

(In reply to comment #19)
> As I understand it, you moved the files from http/tests/media to
> imported/w3c/web-platform-tests. That seems good to me.
> 
> It would be great if a follow-up patch could do the following:
> - Delete LayoutTests/imported/w3c/web-platform-tests/media-source

Did you mean, delete http/tests/media/media-source ?  I can't really do that; there's some non-w3c media-source tests in there.

> - Update LayoutTests/imported/w3c/resources/ImportExpectations to
> comment/remove the web-platform-tests/media-source line

That's in this patch.

> - Run 'Tools/Scripts/import-w3c-tests -t web-platform-tests/media-source' to
> reimport up-to-date media source tests

That was done to generate this patch.

> The first step allows removing potentially obsolete tests.
> If some of these tests are still useful, they should be moved back to
> http/tests, as in a later refresh, I may remove them without knowing that
> they are actually useful.

[Jer Noble]

Created attachment 288427 [details]
Patch

[youenn fablet]

Ah, you did it all in one step, that is just great!
I am surprised that there is no w3c-import.log in the patch though.
Or maybe I missed it.
Anyway sounds good to me if the bots are happy.

[youenn fablet]

Comment on attachment 288427 [details]
Patch

r = me as long as the bots are happy

[Build Bot]

Comment on attachment 288427 [details]
Patch

Attachment 288427 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/2042939

Number of test failures exceeded the failure limit.

[Build Bot]

Created attachment 288431 [details]
Archive of layout-test-results from ews123 for ios-simulator-elcapitan-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews123  Port: ios-simulator-elcapitan-wk2  Platform: Mac OS X 10.11.5

[Build Bot]

Comment on attachment 288427 [details]
Patch

Attachment 288427 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/2043011

New failing tests:
http/tests/media/media-source/mediasource-play-then-seek-back-with-remote-control.html

[Build Bot]

Created attachment 288433 [details]
Archive of layout-test-results from ews103 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103  Port: mac-yosemite  Platform: Mac OS X 10.10.5

[Jer Noble]

Looks like I need to add a rule to the iOS TestExpectations file to skip these (moved) media-source tests.

[Build Bot]

Comment on attachment 288427 [details]
Patch

Attachment 288427 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/2043016

New failing tests:
http/tests/media/media-source/mediasource-play-then-seek-back-with-remote-control.html

[Build Bot]

Created attachment 288434 [details]
Archive of layout-test-results from ews105 for mac-yosemite-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews105  Port: mac-yosemite-wk2  Platform: Mac OS X 10.10.5

[Jer Noble]

And it looks like this most recent test failure is because I moved the webm/ and mp4/ directories out of http/tests/media/media-source.

[Build Bot]

Comment on attachment 288427 [details]
Patch

Attachment 288427 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/2043018

New failing tests:
http/tests/media/media-source/mediasource-play-then-seek-back-with-remote-control.html

[Build Bot]

Created attachment 288435 [details]
Archive of layout-test-results from ews115 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews115  Port: mac-yosemite  Platform: Mac OS X 10.10.5

[Jer Noble]

Created attachment 288436 [details]
Patch for landing

[WebKit Commit Bot]

Comment on attachment 288436 [details]
Patch for landing

Clearing flags on attachment: 288436

Committed r205777: <http://trac.webkit.org/changeset/205777>

[WebKit Commit Bot]

Comment on attachment 288436 [details]
Patch for landing

Rejecting attachment 288436 [details] from commit-queue.

Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-02', 'apply-attachment', '--no-update', '--non-interactive', 288436, '--port=mac']" exit_code: 2 cwd: /Volumes/Data/EWS/WebKit

Last 500 characters of output:
json	(revision 0)
|+++ LayoutTests/http/tests/media/media-source/mp4/test-a-128k-44100Hz-1ch-manifest.json	(working copy)
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored
Original content of LayoutTests/http/tests/media/media-source/mp4/test-a-128k-44100Hz-1ch.mp4 mismatches at /Volumes/Data/EWS/WebKit/Tools/Scripts/svn-apply line 271.

Failed to run "[u'/Volumes/Data/EWS/WebKit/Tools/Scripts/svn-apply', '--force']" exit_code: 2 cwd: /Volumes/Data/EWS/WebKit

Full output: http://webkit-queues.webkit.org/results/2059906