Bug 178129 – Rebaseline tests at media/modern-media-controls/media-controller

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Antoine Quint]

Rebaseline tests at media/modern-media-controls/media-controller

[Antoine Quint]

Created attachment 323300 [details]
Patch

[Build Bot]

Comment on attachment 323300 [details]
Patch

Attachment 323300 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/4811988

New failing tests:
media/modern-media-controls/media-controller/media-controller-inline-to-fullscreen-to-pip-to-inline.html

[Build Bot]

Created attachment 323304 [details]
Archive of layout-test-results from ews100 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews100  Port: mac-elcapitan  Platform: Mac OS X 10.11.6

[Build Bot]

Comment on attachment 323300 [details]
Patch

Attachment 323300 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/4812041

New failing tests:
media/modern-media-controls/media-controller/media-controller-inline-to-fullscreen-to-pip-to-inline.html

[Build Bot]

Created attachment 323305 [details]
Archive of layout-test-results from ews104 for mac-elcapitan-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104  Port: mac-elcapitan-wk2  Platform: Mac OS X 10.11.6

[Build Bot]

Comment on attachment 323300 [details]
Patch

Attachment 323300 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/4812052

New failing tests:
media/modern-media-controls/media-controller/media-controller-auto-hide-mouse-leave-after-play.html
media/modern-media-controls/media-controller/media-controller-ios-only-enable-tap-gesture-recognizer-with-fades-when-idle.html

[Build Bot]

Created attachment 323306 [details]
Archive of layout-test-results from ews126 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews126  Port: ios-simulator-wk2  Platform: Mac OS X 10.12.6

[Build Bot]

Comment on attachment 323300 [details]
Patch

Attachment 323300 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/4812053

New failing tests:
media/modern-media-controls/media-controller/media-controller-inline-to-fullscreen-to-pip-to-inline.html

[Build Bot]

Created attachment 323307 [details]
Archive of layout-test-results from ews115 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews115  Port: mac-elcapitan  Platform: Mac OS X 10.11.6

[Dean Jackson]

Comment on attachment 323300 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=323300&action=review

r+ assuming you fix all the EWS failures

> LayoutTests/platform/mac/TestExpectations:1551
> +media/modern-media-controls/media-controller/media-controller-ios-only-enable-tap-gesture-recognizer-with-fades-when-idle.html [ Skip ]
> +media/modern-media-controls/media-controller/media-controller-scale-factor-audio.html [ Skip ]
> +media/modern-media-controls/media-controller/media-controller-scale-factor.html [ Skip ]

Maybe put iOS tests in a sub directory?

> LayoutTests/platform/mac/TestExpectations:1562
> +# These tests are timing-related and can fail due to performance issues on bots
> +media/modern-media-controls/media-controller/media-controller-auto-hide-mouse-enter-and-mouse-leave.html [ Pass Failure ]
> +media/modern-media-controls/media-controller/media-controller-auto-hide-mouse-leave-after-play.html [ Pass Failure ]
> +media/modern-media-controls/media-controller/media-controller-auto-hide-pause.html [ Pass Failure ]
> +media/modern-media-controls/media-controller/media-controller-auto-hide-rewind-with-mouse-enter.html [ Pass Failure ]
> +media/modern-media-controls/media-controller/media-controller-auto-hide.html [ Pass Failure ]

You should add a comment to the bug to fix these.

[Antoine Quint]

Created attachment 323411 [details]
Patch

[Antoine Quint]

(In reply to Dean Jackson from comment #10)
> Comment on attachment 323300 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=323300&action=review
> 
> r+ assuming you fix all the EWS failures
> 
> > LayoutTests/platform/mac/TestExpectations:1551
> > +media/modern-media-controls/media-controller/media-controller-ios-only-enable-tap-gesture-recognizer-with-fades-when-idle.html [ Skip ]
> > +media/modern-media-controls/media-controller/media-controller-scale-factor-audio.html [ Skip ]
> > +media/modern-media-controls/media-controller/media-controller-scale-factor.html [ Skip ]
> 
> Maybe put iOS tests in a sub directory?

Done in the latest patch.

> > LayoutTests/platform/mac/TestExpectations:1562
> > +# These tests are timing-related and can fail due to performance issues on bots
> > +media/modern-media-controls/media-controller/media-controller-auto-hide-mouse-enter-and-mouse-leave.html [ Pass Failure ]
> > +media/modern-media-controls/media-controller/media-controller-auto-hide-mouse-leave-after-play.html [ Pass Failure ]
> > +media/modern-media-controls/media-controller/media-controller-auto-hide-pause.html [ Pass Failure ]
> > +media/modern-media-controls/media-controller/media-controller-auto-hide-rewind-with-mouse-enter.html [ Pass Failure ]
> > +media/modern-media-controls/media-controller/media-controller-auto-hide.html [ Pass Failure ]
> 
> You should add a comment to the bug to fix these.

These cannot be fixed. By nature they will be flaky due to bots performance and accurate timing requirements. They are likely to pass the vast majority of the time, but fail sometimes. I'd rather keep them as [ Pass Failure ] than not have them at all, if they ever turn red consistently, I'd eventually catch it routinely looking at the flakiness dashboard for those tests.

[Sam Weinig]

Comment on attachment 323411 [details]
Patch

rs=me when things pass.

[Sam Weinig]

Oh, Dean already did that. Sorry.

[Antoine Quint]

(In reply to Sam Weinig from comment #14)
> Oh, Dean already did that. Sorry.

Your rs carries much more weight than dino's r.

[WebKit Commit Bot]

Comment on attachment 323411 [details]
Patch

Clearing flags on attachment: 323411

Committed r223176: <http://trac.webkit.org/changeset/223176>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/34932465>