Bug 280378 – [ Sequoia arm64 ] 14x http/tests/media/fairplay/* are constant failures

• Status: NEW
• Component: Media

[Anfernee Viduya]

http/tests/media/fairplay/fps-hls-key-rotation.html
http/tests/media/fairplay/fps-hls-update-reject.html
http/tests/media/fairplay/fps-hls.html
http/tests/media/fairplay/fps-init-data-cenc.html
http/tests/media/fairplay/fps-init-data-sinf.html
http/tests/media/fairplay/fps-init-data-skd.html
http/tests/media/fairplay/fps-mse-multi-key-renewal.html
http/tests/media/fairplay/fps-mse-play-while-not-in-dom.html
http/tests/media/fairplay/fps-mse-unmuxed-audio-only.html
http/tests/media/fairplay/fps-mse-unmuxed-key-renewal.html
http/tests/media/fairplay/fps-mse-unmuxed-key-rotation.html
http/tests/media/fairplay/fps-mse-unmuxed-multiple-keys.html
http/tests/media/fairplay/fps-mse-unmuxed-same-key.html
http/tests/media/fairplay/legacy-fairplay-hls.html
are constant text failure. 

HISTORY: 
https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=http%2Ftests%2Fmedia%2Ffairplay%2Ffps-hls-key-rotation.html&test=http%2Ftests%2Fmedia%2Ffairplay%2Ffps-hls-update-reject.html&test=http%2Ftests%2Fmedia%2Ffairplay%2Ffps-hls.html&test=http%2Ftests%2Fmedia%2Ffairplay%2Ffps-init-data-cenc.html&test=http%2Ftests%2Fmedia%2Ffairplay%2Ffps-init-data-sinf.html&test=http%2Ftests%2Fmedia%2Ffairplay%2Ffps-init-data-skd.html&test=http%2Ftests%2Fmedia%2Ffairplay%2Ffps-mse-multi-key-renewal.html&test=http%2Ftests%2Fmedia%2Ffairplay%2Ffps-mse-play-while-not-in-dom.html&test=http%2Ftests%2Fmedia%2Ffairplay%2Ffps-mse-unmuxed-audio-only.html&test=http%2Ftests%2Fmedia%2Ffairplay%2Ffps-mse-unmuxed-key-renewal.html&test=http%2Ftests%2Fmedia%2Ffairplay%2Ffps-mse-unmuxed-key-rotation.html&test=http%2Ftests%2Fmedia%2Ffairplay%2Ffps-mse-unmuxed-multiple-keys.html&test=http%2Ftests%2Fmedia%2Ffairplay%2Ffps-mse-unmuxed-same-key.html&test=http%2Ftests%2Fmedia%2Ffairplay%2Flegacy-fairplay-hls.html

DIFF
All tests are producing this diff.

+Server returned malformed response: SyntaxError: The string did not match the expected pattern. FAIL

DIFF LINK:
https://build.webkit.org/results/Apple-Sequoia-Release-AppleSilicon-WK2-Tests/284242%40main%20(65)/results.html#


REPRODUCIBILITY: 
I was not able to reproduce locally. 

DESCRIPTION: 
This has been failing since Sequoia bring up.

[Radar WebKit Bug Importer]

<rdar://problem/136721224>

[Anfernee Viduya]

Pull request: https://github.com/WebKit/WebKit/pull/34275

[EWS]

Test gardening commit 284257@main (4e564de91bd1): <https://commits.webkit.org/284257@main>

Reviewed commits have been landed. Closing PR #34275 and removing active labels.

[Anfernee Viduya]

Pull request: https://github.com/WebKit/WebKit/pull/34304

[EWS]

Test gardening commit 284288@main (e3ce412f95c0): <https://commits.webkit.org/284288@main>

Reviewed commits have been landed. Closing PR #34304 and removing active labels.

[Anfernee Viduya]

Pull request: https://github.com/WebKit/WebKit/pull/34864

[EWS]

Test gardening commit 284843@main (7bceb4d559e8): <https://commits.webkit.org/284843@main>

Reviewed commits have been landed. Closing PR #34864 and removing active labels.

[Anfernee Viduya]

(In reply to Anfernee Viduya from comment #6)
> Pull request: https://github.com/WebKit/WebKit/pull/34864

New Debug Sequoia builders were brought up and these tests were failing. Adjusted proper expectations cover the debug builders.

[Jer Noble]

It looks like these tests are failing due to a failure to initialize the correct prerequisites for the FairPlay Keyserver:

https://build.webkit.org/results/Apple-Sequoia-Release-AppleSilicon-WK2-Tests/285903@main%20(534)/error_log.txt

[Wed Oct 30 09:41:01.217948 2024] [cgi:error] [pid 71118] [client 127.0.0.1:62642] AH01215: subprocess.CalledProcessError: Command '['/Applications/Xcode.app/Contents/Developer/usr/bin/python3', '/tmp/Crypto-71781/pycryptodome-3.10.1/setup.py', 'install', '--home=/Volumes/Data/worker/Apple-Sequoia-Release-AppleSilicon-WK2-Tests/build/Tools/Scripts/libraries/autoinstalled/python-3-arm64', '--root=/', '--prefix=', '--install-lib=/Volumes/Data/worker/Apple-Sequoia-Release-AppleSilicon-WK2-Tests/build/Tools/Scripts/libraries/autoinstalled/python-3-arm64', '--install-scripts=/Volumes/Data/worker/Apple-Sequoia-Release-AppleSilicon-WK2-Tests/build/Tools/Scripts/libraries/autoinstalled/python-3-arm64/bin', '--install-data=/Volumes/Data/worker/Apple-Sequoia-Release-AppleSilicon-WK2-Tests/build/Tools/Scripts/libraries/autoinstalled/python-3-arm64/data', '--install-headers=/Volumes/Data/worker/Apple-Sequoia-Release-AppleSilicon-WK2-Tests/build/Tools/Scripts/libraries/autoinstalled/python-3-arm64/headers']' returned non-zero exit status 1.: /Volumes/Data/worker/Apple-Sequoia-Release-AppleSilicon-WK2-Tests/build/LayoutTests/http/tests/media/fairplay/resources/index.py, referer: http://127.0.0.1:8000/media/fairplay/fps-hls-key-rotation.html

[Sam Sneddon [:gsnedders]]

Traceback (most recent call last):
   File "/tmp/Crypto-71781/pycryptodome-3.10.1/setup.py", line 478, in <module>
     setup(
   File "/Volumes/Data/worker/Apple-Sequoia-Release-AppleSilicon-WK2-Tests/build/Tools/Scripts/libraries/autoinstalled/python-3-arm64/setuptools/__init__.py", line 153, in setup
     return distutils.core.setup(**attrs)
   File "/Volumes/Data/worker/Apple-Sequoia-Release-AppleSilicon-WK2-Tests/build/Tools/Scripts/libraries/autoinstalled/python-3-arm64/setuptools/_distutils/core.py", line 109, in setup
     _setup_distribution = dist = klass(attrs)
   File "/Volumes/Data/worker/Apple-Sequoia-Release-AppleSilicon-WK2-Tests/build/Tools/Scripts/libraries/autoinstalled/python-3-arm64/setuptools/dist.py", line 459, in __init__
     _Distribution.__init__(
   File "/Volumes/Data/worker/Apple-Sequoia-Release-AppleSilicon-WK2-Tests/build/Tools/Scripts/libraries/autoinstalled/python-3-arm64/setuptools/_distutils/dist.py", line 293, in __init__
     self.finalize_options()
   File "/Volumes/Data/worker/Apple-Sequoia-Release-AppleSilicon-WK2-Tests/build/Tools/Scripts/libraries/autoinstalled/python-3-arm64/setuptools/dist.py", line 836, in finalize_options
     for ep in sorted(loaded, key=by_order):
   File "/Volumes/Data/worker/Apple-Sequoia-Release-AppleSilicon-WK2-Tests/build/Tools/Scripts/libraries/autoinstalled/python-3-arm64/setuptools/dist.py", line 835, in <lambda>
     loaded = map(lambda e: e.load(), filtered)
   File "/Volumes/Data/worker/Apple-Sequoia-Release-AppleSilicon-WK2-Tests/build/Tools/Scripts/libraries/autoinstalled/python-3-arm64/pkg_resources/__init__.py", line 2464, in load
     self.require(*args, **kwargs)
   File "/Volumes/Data/worker/Apple-Sequoia-Release-AppleSilicon-WK2-Tests/build/Tools/Scripts/libraries/autoinstalled/python-3-arm64/pkg_resources/__init__.py", line 2487, in require
     items = working_set.resolve(reqs, env, installer, extras=self.extras)
   File "/Volumes/Data/worker/Apple-Sequoia-Release-AppleSilicon-WK2-Tests/build/Tools/Scripts/libraries/autoinstalled/python-3-arm64/pkg_resources/__init__.py", line 782, in resolve
     raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (setuptools 59.8.0 (/Volumes/Data/worker/Apple-Sequoia-Release-AppleSilicon-WK2-Tests/build/Tools/Scripts/libraries/autoinstalled/python-3-arm64), Requirement.parse('setuptools>=61.0'))

And that requirement seems to be coming from:

/Library/Python/3.9/site-packages/incremental-24.7.2.dist-info/METADATA:Requires-Dist: setuptools >=61.0

[Anfernee Viduya]

Additional info, these two tests recently regressed to fail on all queues now instead of a select few with a different diff. Regression seems to point to 285731@main
http/tests/media/fairplay/fps-mse-unmuxed-key-rotation.html
http/tests/media/fairplay/legacy-fairplay-hls.html

DIFF:
 EVENT(webkitneedkey)
 FETCH: resources/cert.der OK
-EVENT(webkitkeymessage)
-PROMISE: licenseResponse resolved
-EVENT(canplaythrough)
+EVENT(webkitkeyerror)
+FAIL: update() failed with code: undefined, systemCode: undefined

[EWS]

Test gardening commit 298106@main (3fd59228c4d8): <https://commits.webkit.org/298106@main>

Reviewed commits have been landed. Closing PR #48800 and removing active labels.

[Bryan Medina]

Updated test expectation for:

http/tests/media/fairplay/fps-mse-play-while-not-in-dom.html 

As it is now flakey failing on both architectures of macOS Sequoia wk2.

[Bryan Medina]

*** Bug 297803 has been marked as a duplicate of this bug. ***

[Bryan Medina]

Updated test expectation for:

http/tests/media/fairplay/fps-mse-play-while-not-in-dom.html 

As it is now impacting all of macOS

[EWS]

Test gardening commit 299071@main (58b98437ef18): <https://commits.webkit.org/299071@main>

Reviewed commits have been landed. Closing PR #49789 and removing active labels.