Bug 215459 – Migrate some Apple internal Mac expectations here into WebKit sources

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Darin Adler]

Migrate some Apple internal Mac expectations here into WebKit sources

[Darin Adler]

Created attachment 406531 [details]
Patch

[Darin Adler]

<rdar://problem/66161781>

[Darin Adler]

Created attachment 406532 [details]
Patch

[Darin Adler]

Comment on attachment 406532 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=406532&action=review

> LayoutTests/platform/mac-wk2/TestExpectations:1211
> +[ arm64 ] http/tests/security/storage-blocking-loosened-plugin.html [ Skip ]

As suggested by Alexey, can and will break this into two lines to get rid of the run-webkit-tests lint warning:

[ arm64 Release ] http/tests/security/storage-blocking-loosened-plugin.html [ Skip ]
[ arm64 Debug ] http/tests/security/storage-blocking-loosened-plugin.html [ Skip ]

[Darin Adler]

This is associated with the wrong bug in Radar. How do I fix that?

[Ryan Haddad]

(In reply to Darin Adler from comment #5)
> This is associated with the wrong bug in Radar. How do I fix that?
Paste the rdar URL in a new comment and I believe the importer should take care of it.

[Darin Adler]

rdar://67026638

[Darin Adler]

Looking for review on this; it will be easier to merge if it’s done before people make a lot of other changes to test expectations.

[Jonathan Bedard]

Comment on attachment 406532 [details]
Patch

Looks like our flakey test is not one we had an expectation change for, so let's land!

[Darin Adler]

Committed r265639: <https://trac.webkit.org/changeset/265639>

[Alexey Proskuryakov]

Comment on attachment 406532 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=406532&action=review

This is a very large patch that mixes different kinds of changes. I think that some of these are desirable, but not all.

Only did a spot check, as this is so large, and already landed.

> LayoutTests/platform/mac-wk1/TestExpectations:1087
> +# <rdar://problem/61461935> [ macOS wk2 ] fast/scrolling/rtl-point-in-iframe.html is flaky failing
> +[ Mojave ] fast/scrolling/rtl-point-in-iframe.html [ Pass Failure ]

This isn't failing anywhere, open source or internal.

> LayoutTests/platform/mac/TestExpectations:1944
> +# <rdar://problem/31017631> Stress GC bot failing LayoutTest step with timeout while running WebCryptoAPI/derive_bits_keys tests
> +[ Debug ] imported/w3c/web-platform-tests/WebCryptoAPI/derive_bits_keys [ Skip ]

How is this a reason to skip the test on all Mac bots?

> LayoutTests/platform/mac/TestExpectations:1948
> +# The navigator.language tests rely on functionality only available in recent releases of macOS Catalina and onward.
> +# <rdar://problem/59461792> REGRESSION: [ Mac iOS ] ( r254389 - r254411 ) fast/text/international/system-language/declarative-language.html is failing internally
> +fast/text/international/system-language/declarative-language.html [ Pass Failure ]

The expectation doesn't match the comment, and neither matches the actual behavior.

> LayoutTests/platform/mac/TestExpectations:1955
> +# <rdar://problem/59636115> REGRESSION: [ iOS & macOS ] two imported/w3c/web-platform-tests/WebCryptoAPI are failing
> +imported/w3c/web-platform-tests/WebCryptoAPI/import_export/rsa_importKey.https.worker.html [ Pass Failure ]
> +imported/w3c/web-platform-tests/WebCryptoAPI/import_export/test_rsa_importKey.https.html [ Pass Failure ]

This is an OS specific regression, so marking the tests flaky unconditionally regresses test coverage.

> LayoutTests/platform/mac/TestExpectations:2041
> +# <rdar://problem/61114827> [ Stress GC ] http/tests/navigation/page-cache-fontfaceset.html is flaky crashing.
> +[ Mojave ] http/tests/navigation/page-cache-fontfaceset.html [ Pass Crash ]

All of these are a very internal hack, enabled by running Stress GC on Mojave only. This is seems confusing and misplaced in open source expectations.

> LayoutTests/platform/mac/TestExpectations:2203
> +# rdar://66802945 (REGRESSION (20A2323e-20A234b): [ Layout Test] 12 canvas layout tests are a constant failure)
> +[ arm64 ] canvas/philip/tests/2d.line.miter.obtuse.html [ Failure ]
> +[ arm64 ] canvas/philip/tests/2d.path.arc.selfintersect.1.html [ Failure ]
> +[ arm64 ] canvas/philip/tests/2d.path.arc.shape.3.html [ Failure ]
> +[ arm64 ] fast/canvas/canvas-ellipse-zero-lineto.html [ Failure ]
> +[ arm64 ] imported/w3c/canvas/2d.line.miter.obtuse.html [ Failure ]
> +[ arm64 ] imported/w3c/canvas/2d.path.arc.selfintersect.1.html [ Failure ]
> +[ arm64 ] imported/w3c/canvas/2d.path.arc.shape.3.html [ Failure ]
> +[ arm64 ] imported/w3c/web-platform-tests/html/canvas/element/line-styles/2d.line.miter.obtuse.html [ Failure ]
> +[ arm64 ] imported/w3c/web-platform-tests/html/canvas/element/path-objects/2d.path.arc.selfintersect.1.html [ Failure ]
> +[ arm64 ] imported/w3c/web-platform-tests/html/canvas/element/path-objects/2d.path.arc.shape.3.html [ Failure ]

As these are due to an issue below WebKit, moving the expectations to open source makes them harder to keep track of, I think.

[Darin Adler]

I can see that you and I have very different ideas about open source vs. internal. As far as I am concerned, only secret changes should be internal.

[Alexey Proskuryakov]

I agree with this as a basic philosophy! What I don't believe in is that mechanically moving everything into open source is the right first step. This is an essentially non-reversible move that loses information.

Tracking issues that are below WebKit and are only affecting future OSes, possibly for a short period of time that isn't aligned with public beta schedule, is somewhat of a gray area for me.

[Alexey Proskuryakov]

Comment on attachment 406532 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=406532&action=review

> LayoutTests/ChangeLog:23
> +        * platform/mac-bigsur-wk1/editing/pasteboard/drop-text-without-selection-expected.txt: Added.

We all missed the big issue. Expectations for latest macOS live in platform/mac (because that's the present and also the likely future), and only older OSes get named directories.

Will get these updated.