Bug 161723

Summary: [Media Source] Import updated w3c/web-platform-tests and rebaseline
Product: WebKit Reporter: Jer Noble <jer.noble>
Component: MediaAssignee: Jer Noble <jer.noble>
Status: RESOLVED FIXED    
Severity: Normal CC: buildbot, commit-queue, rniwa, youennf
Priority: P2    
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Bug Depends on:    
Bug Blocks: 161725    
Attachments:
Description Flags
Patch
none
Archive of layout-test-results from ews126 for ios-simulator-elcapitan-wk2
none
Archive of layout-test-results from ews107 for mac-yosemite-wk2
none
Archive of layout-test-results from ews113 for mac-yosemite
none
Archive of layout-test-results from ews101 for mac-yosemite
none
Patch
none
Archive of layout-test-results from ews102 for mac-yosemite
none
Archive of layout-test-results from ews125 for ios-simulator-elcapitan-wk2
none
Archive of layout-test-results from ews107 for mac-yosemite-wk2
none
Archive of layout-test-results from ews116 for mac-yosemite
none
Patch
youennf: review+, buildbot: commit-queue-
Archive of layout-test-results from ews123 for ios-simulator-elcapitan-wk2
none
Archive of layout-test-results from ews103 for mac-yosemite
none
Archive of layout-test-results from ews105 for mac-yosemite-wk2
none
Archive of layout-test-results from ews115 for mac-yosemite
none
Patch for landing none

Jer Noble
Reported 2016-09-07 17:45:56 PDT
[Media Source] Import updated w3c/web-platform-tests and rebaseline
Attachments
Patch (4.28 MB, patch)
2016-09-08 15:26 PDT, Jer Noble
no flags
Archive of layout-test-results from ews126 for ios-simulator-elcapitan-wk2 (deleted)
2016-09-08 16:25 PDT, Build Bot
no flags
Archive of layout-test-results from ews107 for mac-yosemite-wk2 (1.33 MB, application/zip)
2016-09-08 16:29 PDT, Build Bot
no flags
Archive of layout-test-results from ews113 for mac-yosemite (1.46 MB, application/zip)
2016-09-08 16:35 PDT, Build Bot
no flags
Archive of layout-test-results from ews101 for mac-yosemite (1.27 MB, application/zip)
2016-09-08 16:42 PDT, Build Bot
no flags
Patch (4.28 MB, patch)
2016-09-09 10:36 PDT, Jer Noble
no flags
Archive of layout-test-results from ews102 for mac-yosemite (1.11 MB, application/zip)
2016-09-09 11:28 PDT, Build Bot
no flags
Archive of layout-test-results from ews125 for ios-simulator-elcapitan-wk2 (804.19 KB, application/zip)
2016-09-09 11:40 PDT, Build Bot
no flags
Archive of layout-test-results from ews107 for mac-yosemite-wk2 (1.09 MB, application/zip)
2016-09-09 11:45 PDT, Build Bot
no flags
Archive of layout-test-results from ews116 for mac-yosemite (1.45 MB, application/zip)
2016-09-09 11:46 PDT, Build Bot
no flags
Patch (4.28 MB, patch)
2016-09-09 12:15 PDT, Jer Noble
youennf: review+
buildbot: commit-queue-
Archive of layout-test-results from ews123 for ios-simulator-elcapitan-wk2 (8.88 MB, application/zip)
2016-09-09 13:09 PDT, Build Bot
no flags
Archive of layout-test-results from ews103 for mac-yosemite (1.13 MB, application/zip)
2016-09-09 13:16 PDT, Build Bot
no flags
Archive of layout-test-results from ews105 for mac-yosemite-wk2 (1.02 MB, application/zip)
2016-09-09 13:20 PDT, Build Bot
no flags
Archive of layout-test-results from ews115 for mac-yosemite (1.48 MB, application/zip)
2016-09-09 13:25 PDT, Build Bot
no flags
Patch for landing (3.86 MB, patch)
2016-09-09 13:34 PDT, Jer Noble
no flags
Jer Noble
Comment 1 2016-09-08 15:26:16 PDT
Build Bot
Comment 2 2016-09-08 16:25:39 PDT
Comment on attachment 288338 [details] Patch Attachment 288338 [details] did not pass ios-sim-ews (ios-simulator-wk2): Output: http://webkit-queues.webkit.org/results/2037124 Number of test failures exceeded the failure limit.
Build Bot
Comment 3 2016-09-08 16:25:41 PDT
Created attachment 288347 [details] Archive of layout-test-results from ews126 for ios-simulator-elcapitan-wk2 The attached test failures were seen while running run-webkit-tests on the ios-sim-ews. Bot: ews126 Port: ios-simulator-elcapitan-wk2 Platform: Mac OS X 10.11.5
Build Bot
Comment 4 2016-09-08 16:29:14 PDT
Comment on attachment 288338 [details] Patch Attachment 288338 [details] did not pass mac-wk2-ews (mac-wk2): Output: http://webkit-queues.webkit.org/results/2037151 New failing tests: imported/w3c/web-platform-tests/media-source/mediasource-addsourcebuffer.html imported/w3c/web-platform-tests/media-source/mediasource-remove.html imported/w3c/web-platform-tests/media-source/mediasource-sourcebuffer-mode.html imported/w3c/web-platform-tests/media-source/mediasource-is-type-supported.html http/tests/media/media-source/mediasource-append-stream.html imported/w3c/web-platform-tests/media-source/mediasource-append-buffer.html http/tests/media/media-source/mediasource-play-then-seek-back-with-remote-control.html imported/w3c/web-platform-tests/media-source/mediasource-config-change-mp4-v-bitrate.html imported/w3c/web-platform-tests/media-source/mediasource-play.html
Build Bot
Comment 5 2016-09-08 16:29:16 PDT
Created attachment 288350 [details] Archive of layout-test-results from ews107 for mac-yosemite-wk2 The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews. Bot: ews107 Port: mac-yosemite-wk2 Platform: Mac OS X 10.10.5
Build Bot
Comment 6 2016-09-08 16:35:25 PDT
Comment on attachment 288338 [details] Patch Attachment 288338 [details] did not pass mac-debug-ews (mac): Output: http://webkit-queues.webkit.org/results/2037160 New failing tests: imported/w3c/web-platform-tests/media-source/mediasource-addsourcebuffer.html imported/w3c/web-platform-tests/media-source/mediasource-remove.html imported/w3c/web-platform-tests/media-source/mediasource-sourcebuffer-mode.html imported/w3c/web-platform-tests/media-source/mediasource-is-type-supported.html http/tests/media/media-source/mediasource-append-stream.html imported/w3c/web-platform-tests/media-source/mediasource-append-buffer.html http/tests/media/media-source/mediasource-play-then-seek-back-with-remote-control.html imported/w3c/web-platform-tests/media-source/mediasource-config-change-mp4-v-bitrate.html imported/w3c/web-platform-tests/media-source/mediasource-play.html
Build Bot
Comment 7 2016-09-08 16:35:27 PDT
Created attachment 288353 [details] Archive of layout-test-results from ews113 for mac-yosemite The attached test failures were seen while running run-webkit-tests on the mac-debug-ews. Bot: ews113 Port: mac-yosemite Platform: Mac OS X 10.10.5
Build Bot
Comment 8 2016-09-08 16:42:35 PDT
Comment on attachment 288338 [details] Patch Attachment 288338 [details] did not pass mac-ews (mac): Output: http://webkit-queues.webkit.org/results/2037210 New failing tests: imported/w3c/web-platform-tests/media-source/mediasource-addsourcebuffer.html imported/w3c/web-platform-tests/media-source/mediasource-remove.html imported/w3c/web-platform-tests/media-source/mediasource-sourcebuffer-mode.html imported/w3c/web-platform-tests/media-source/mediasource-is-type-supported.html http/tests/media/media-source/mediasource-append-stream.html imported/w3c/web-platform-tests/media-source/mediasource-append-buffer.html http/tests/media/media-source/mediasource-play-then-seek-back-with-remote-control.html imported/w3c/web-platform-tests/media-source/mediasource-config-change-mp4-v-bitrate.html imported/w3c/web-platform-tests/media-source/mediasource-play.html
Build Bot
Comment 9 2016-09-08 16:42:37 PDT
Created attachment 288356 [details] Archive of layout-test-results from ews101 for mac-yosemite The attached test failures were seen while running run-webkit-tests on the mac-ews. Bot: ews101 Port: mac-yosemite Platform: Mac OS X 10.10.5
Jer Noble
Comment 10 2016-09-09 10:36:48 PDT
Build Bot
Comment 11 2016-09-09 11:27:58 PDT
Comment on attachment 288412 [details] Patch Attachment 288412 [details] did not pass mac-ews (mac): Output: http://webkit-queues.webkit.org/results/2042503 New failing tests: imported/w3c/web-platform-tests/media-source/mediasource-addsourcebuffer.html imported/w3c/web-platform-tests/media-source/mediasource-remove.html imported/w3c/web-platform-tests/media-source/mediasource-sourcebuffer-mode.html imported/w3c/web-platform-tests/media-source/mediasource-is-type-supported.html imported/w3c/web-platform-tests/media-source/mediasource-append-buffer.html http/tests/media/media-source/mediasource-play-then-seek-back-with-remote-control.html imported/w3c/web-platform-tests/media-source/mediasource-config-change-mp4-v-bitrate.html imported/w3c/web-platform-tests/media-source/mediasource-play.html
Build Bot
Comment 12 2016-09-09 11:28:02 PDT
Created attachment 288418 [details] Archive of layout-test-results from ews102 for mac-yosemite The attached test failures were seen while running run-webkit-tests on the mac-ews. Bot: ews102 Port: mac-yosemite Platform: Mac OS X 10.10.5
Build Bot
Comment 13 2016-09-09 11:40:16 PDT
Comment on attachment 288412 [details] Patch Attachment 288412 [details] did not pass ios-sim-ews (ios-simulator-wk2): Output: http://webkit-queues.webkit.org/results/2042532 Number of test failures exceeded the failure limit.
Build Bot
Comment 14 2016-09-09 11:40:19 PDT
Created attachment 288420 [details] Archive of layout-test-results from ews125 for ios-simulator-elcapitan-wk2 The attached test failures were seen while running run-webkit-tests on the ios-sim-ews. Bot: ews125 Port: ios-simulator-elcapitan-wk2 Platform: Mac OS X 10.11.5
Build Bot
Comment 15 2016-09-09 11:45:13 PDT
Comment on attachment 288412 [details] Patch Attachment 288412 [details] did not pass mac-wk2-ews (mac-wk2): Output: http://webkit-queues.webkit.org/results/2042554 New failing tests: imported/w3c/web-platform-tests/media-source/mediasource-addsourcebuffer.html imported/w3c/web-platform-tests/media-source/mediasource-remove.html imported/w3c/web-platform-tests/media-source/mediasource-sourcebuffer-mode.html imported/w3c/web-platform-tests/media-source/mediasource-is-type-supported.html imported/w3c/web-platform-tests/media-source/mediasource-append-buffer.html http/tests/media/media-source/mediasource-play-then-seek-back-with-remote-control.html imported/w3c/web-platform-tests/media-source/mediasource-config-change-mp4-v-bitrate.html imported/w3c/web-platform-tests/media-source/mediasource-play.html
Build Bot
Comment 16 2016-09-09 11:45:15 PDT
Created attachment 288421 [details] Archive of layout-test-results from ews107 for mac-yosemite-wk2 The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews. Bot: ews107 Port: mac-yosemite-wk2 Platform: Mac OS X 10.10.5
Build Bot
Comment 17 2016-09-09 11:46:41 PDT
Comment on attachment 288412 [details] Patch Attachment 288412 [details] did not pass mac-debug-ews (mac): Output: http://webkit-queues.webkit.org/results/2042556 New failing tests: imported/w3c/web-platform-tests/media-source/mediasource-addsourcebuffer.html imported/w3c/web-platform-tests/media-source/mediasource-remove.html imported/w3c/web-platform-tests/media-source/mediasource-sourcebuffer-mode.html imported/w3c/web-platform-tests/media-source/mediasource-is-type-supported.html imported/w3c/web-platform-tests/media-source/mediasource-append-buffer.html http/tests/media/media-source/mediasource-play-then-seek-back-with-remote-control.html imported/w3c/web-platform-tests/media-source/mediasource-config-change-mp4-v-bitrate.html imported/w3c/web-platform-tests/media-source/mediasource-play.html
Build Bot
Comment 18 2016-09-09 11:46:44 PDT
Created attachment 288422 [details] Archive of layout-test-results from ews116 for mac-yosemite The attached test failures were seen while running run-webkit-tests on the mac-debug-ews. Bot: ews116 Port: mac-yosemite Platform: Mac OS X 10.10.5
youenn fablet
Comment 19 2016-09-09 12:10:29 PDT
As I understand it, you moved the files from http/tests/media to imported/w3c/web-platform-tests. That seems good to me. It would be great if a follow-up patch could do the following: - Delete LayoutTests/imported/w3c/web-platform-tests/media-source - Update LayoutTests/imported/w3c/resources/ImportExpectations to comment/remove the web-platform-tests/media-source line - Run 'Tools/Scripts/import-w3c-tests -t web-platform-tests/media-source' to reimport up-to-date media source tests The first step allows removing potentially obsolete tests. If some of these tests are still useful, they should be moved back to http/tests, as in a later refresh, I may remove them without knowing that they are actually useful.
Jer Noble
Comment 20 2016-09-09 12:12:20 PDT
(In reply to comment #19) > As I understand it, you moved the files from http/tests/media to > imported/w3c/web-platform-tests. That seems good to me. > > It would be great if a follow-up patch could do the following: > - Delete LayoutTests/imported/w3c/web-platform-tests/media-source Did you mean, delete http/tests/media/media-source ? I can't really do that; there's some non-w3c media-source tests in there. > - Update LayoutTests/imported/w3c/resources/ImportExpectations to > comment/remove the web-platform-tests/media-source line That's in this patch. > - Run 'Tools/Scripts/import-w3c-tests -t web-platform-tests/media-source' to > reimport up-to-date media source tests That was done to generate this patch. > The first step allows removing potentially obsolete tests. > If some of these tests are still useful, they should be moved back to > http/tests, as in a later refresh, I may remove them without knowing that > they are actually useful.
Jer Noble
Comment 21 2016-09-09 12:15:02 PDT
youenn fablet
Comment 22 2016-09-09 12:20:33 PDT
Ah, you did it all in one step, that is just great! I am surprised that there is no w3c-import.log in the patch though. Or maybe I missed it. Anyway sounds good to me if the bots are happy.
youenn fablet
Comment 23 2016-09-09 12:21:51 PDT
Comment on attachment 288427 [details] Patch r = me as long as the bots are happy
Build Bot
Comment 24 2016-09-09 13:09:33 PDT
Comment on attachment 288427 [details] Patch Attachment 288427 [details] did not pass ios-sim-ews (ios-simulator-wk2): Output: http://webkit-queues.webkit.org/results/2042939 Number of test failures exceeded the failure limit.
Build Bot
Comment 25 2016-09-09 13:09:36 PDT
Created attachment 288431 [details] Archive of layout-test-results from ews123 for ios-simulator-elcapitan-wk2 The attached test failures were seen while running run-webkit-tests on the ios-sim-ews. Bot: ews123 Port: ios-simulator-elcapitan-wk2 Platform: Mac OS X 10.11.5
Build Bot
Comment 26 2016-09-09 13:16:47 PDT
Comment on attachment 288427 [details] Patch Attachment 288427 [details] did not pass mac-ews (mac): Output: http://webkit-queues.webkit.org/results/2043011 New failing tests: http/tests/media/media-source/mediasource-play-then-seek-back-with-remote-control.html
Build Bot
Comment 27 2016-09-09 13:16:50 PDT
Created attachment 288433 [details] Archive of layout-test-results from ews103 for mac-yosemite The attached test failures were seen while running run-webkit-tests on the mac-ews. Bot: ews103 Port: mac-yosemite Platform: Mac OS X 10.10.5
Jer Noble
Comment 28 2016-09-09 13:18:15 PDT
Looks like I need to add a rule to the iOS TestExpectations file to skip these (moved) media-source tests.
Build Bot
Comment 29 2016-09-09 13:20:55 PDT
Comment on attachment 288427 [details] Patch Attachment 288427 [details] did not pass mac-wk2-ews (mac-wk2): Output: http://webkit-queues.webkit.org/results/2043016 New failing tests: http/tests/media/media-source/mediasource-play-then-seek-back-with-remote-control.html
Build Bot
Comment 30 2016-09-09 13:20:58 PDT
Created attachment 288434 [details] Archive of layout-test-results from ews105 for mac-yosemite-wk2 The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews. Bot: ews105 Port: mac-yosemite-wk2 Platform: Mac OS X 10.10.5
Jer Noble
Comment 31 2016-09-09 13:24:46 PDT
And it looks like this most recent test failure is because I moved the webm/ and mp4/ directories out of http/tests/media/media-source.
Build Bot
Comment 32 2016-09-09 13:25:56 PDT
Comment on attachment 288427 [details] Patch Attachment 288427 [details] did not pass mac-debug-ews (mac): Output: http://webkit-queues.webkit.org/results/2043018 New failing tests: http/tests/media/media-source/mediasource-play-then-seek-back-with-remote-control.html
Build Bot
Comment 33 2016-09-09 13:25:59 PDT
Created attachment 288435 [details] Archive of layout-test-results from ews115 for mac-yosemite The attached test failures were seen while running run-webkit-tests on the mac-debug-ews. Bot: ews115 Port: mac-yosemite Platform: Mac OS X 10.10.5
Jer Noble
Comment 34 2016-09-09 13:34:47 PDT
Created attachment 288436 [details] Patch for landing
WebKit Commit Bot
Comment 35 2016-09-09 17:51:32 PDT
Comment on attachment 288436 [details] Patch for landing Clearing flags on attachment: 288436 Committed r205777: <http://trac.webkit.org/changeset/205777>
WebKit Commit Bot
Comment 36 2016-09-12 09:33:45 PDT
Comment on attachment 288436 [details] Patch for landing Rejecting attachment 288436 [details] from commit-queue. Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-02', 'apply-attachment', '--no-update', '--non-interactive', 288436, '--port=mac']" exit_code: 2 cwd: /Volumes/Data/EWS/WebKit Last 500 characters of output: json (revision 0) |+++ LayoutTests/http/tests/media/media-source/mp4/test-a-128k-44100Hz-1ch-manifest.json (working copy) -------------------------- No file to patch. Skipping patch. 1 out of 1 hunk ignored Original content of LayoutTests/http/tests/media/media-source/mp4/test-a-128k-44100Hz-1ch.mp4 mismatches at /Volumes/Data/EWS/WebKit/Tools/Scripts/svn-apply line 271. Failed to run "[u'/Volumes/Data/EWS/WebKit/Tools/Scripts/svn-apply', '--force']" exit_code: 2 cwd: /Volumes/Data/EWS/WebKit Full output: http://webkit-queues.webkit.org/results/2059906
Note You need to log in before you can comment on or make changes to this bug.