263527 2023-10-23 09:41:11 -0700 REGRESSION(267278@main): Multiple imported audio encoder/decoder layout tests are constantly failing. 2023-10-23 10:09:55 -0700 1 1 1 Unclassified WebKit Media WebKit Nightly Build Unspecified Unspecified NEW https://bugs.webkit.org/show_bug.cgi?id=261777 https://bugs.webkit.org/show_bug.cgi?id=261107 https://bugs.webkit.org/show_bug.cgi?id=259874 InRadar P2 Normal --- 1 ben_schwartz webkit-unassigned mattwoodrow webkit-bot-watchers-bugzilla webkit-bug-importer oldest_to_newest 1986831 0 ben_schwartz 2023-10-23 09:41:11 -0700 imported/w3c/web-platform-tests/webcodecs/audio-encoder.https.any.html This test, previously constantly crashing on wk2 debug and constantly failing on wk2 release, is now constantly failing on all Apple configurations. HISTORY: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fwebcodecs%2Faudio-encoder.https.any.html TEXT DIFF: +CONSOLE MESSAGE: Error: assert_unreached: error: NotSupportedError: Not supported Reached unreachable code +CONSOLE MESSAGE: Error: assert_unreached: error: NotSupportedError: Not supported Reached unreachable code +CONSOLE MESSAGE: NotSupportedError: AudioData creation failed -PASS Simple audio encoding -PASS Test reset during flush -PASS Encode audio with negative timestamp -FAIL Channel number variation: 1 assert_unreached: Should have rejected: undefined Reached unreachable code -FAIL Channel number variation: 2 assert_unreached: Should have rejected: undefined Reached unreachable code -FAIL Sample rate variation: 3000 assert_unreached: Should have rejected: undefined Reached unreachable code -FAIL Sample rate variation: 13000 assert_unreached: Should have rejected: undefined Reached unreachable code -FAIL Sample rate variation: 23000 assert_unreached: Should have rejected: undefined Reached unreachable code -FAIL Sample rate variation: 33000 assert_unreached: Should have rejected: undefined Reached unreachable code -FAIL Sample rate variation: 43000 assert_unreached: Should have rejected: undefined Reached unreachable code -FAIL Sample rate variation: 53000 assert_unreached: Should have rejected: undefined Reached unreachable code -FAIL Sample rate variation: 63000 assert_unreached: Should have rejected: undefined Reached unreachable code -FAIL Sample rate variation: 73000 assert_unreached: Should have rejected: undefined Reached unreachable code -FAIL Sample rate variation: 83000 assert_unreached: Should have rejected: undefined Reached unreachable code -FAIL Sample rate variation: 93000 assert_unreached: Should have rejected: undefined Reached unreachable code -FAIL Encoding and decoding assert_true: expected true got false -PASS Emit decoder config and extra data. -PASS encodeQueueSize test -PASS Test encoding Opus with additional parameters: Empty Opus config -PASS Test encoding Opus with additional parameters: Opus with frameDuration -PASS Test encoding Opus with additional parameters: Opus with complexity -PASS Test encoding Opus with additional parameters: Opus with useinbandfec -PASS Test encoding Opus with additional parameters: Opus with usedtx -PASS Test encoding Opus with additional parameters: Opus mixed parameters +Harness Error (FAIL), message = NotSupportedError: AudioData creation failed +FAIL Simple audio encoding promise_test: Unhandled rejection with value: object "NotSupportedError: AudioData creation failed" +FAIL Test reset during flush assert_unreached: unexpected error Reached unreachable code +FAIL Encode audio with negative timestamp promise_test: Unhandled rejection with value: object "NotSupportedError: AudioData creation failed" + DIFF URL: https://build.webkit.org/results/Apple-Ventura-Release-AppleSilicon-WK2-Tests/269648@main%20(5586)/imported/w3c/web-platform-tests/webcodecs/audio-encoder.https.any-pretty-diff.html REPRODUCIBILITY: I was able to reproduce this issue on macOS Ventura (13.6, 22G120/arm64) ToT using the following command: run-webkit-tests --debug --clobber-old-results --verbose imported/w3c/web-platform-tests/webcodecs/audio-encoder.https.any.html REGRESSION: I was able to bisect a regression point at 267963@main (issue did not reproduce at 267962@main). Changes at that point are directly related to this issue, and are likely what caused it. 1986832 1 webkit-bug-importer 2023-10-23 09:41:42 -0700 <rdar://problem/117357248> 1986844 2 ben_schwartz 2023-10-23 10:07:47 -0700 It appears this issue already has test expectations for multiple failing layout tests with no related bug. I'm going to file a gardening commit that has a link back to this bug for reference. The following tests are constantly failing (with extremely flaky crashes) on all Apple platforms (regression at 267278@main): imported/w3c/web-platform-tests/webcodecs/audio-data-serialization.any.html imported/w3c/web-platform-tests/webcodecs/audio-data.any.html imported/w3c/web-platform-tests/webcodecs/audio-data.any.worker.html imported/w3c/web-platform-tests/webcodecs/audio-decoder.crossOriginIsolated.https.any.html imported/w3c/web-platform-tests/webcodecs/audio-decoder.crossOriginIsolated.https.any.worker.html imported/w3c/web-platform-tests/webcodecs/audio-encoder-codec-specific.https.any.html imported/w3c/web-platform-tests/webcodecs/audio-encoder.https.any.html imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.html?adts_aac imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.html?mp3 imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.html?mp4_aac imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.html?opus imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.html?pcm_alaw imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.html?pcm_mulaw imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.worker.html?adts_aac imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.worker.html?mp3 imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.worker.html?mp4_aac imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.worker.html?opus imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.worker.html?pcm_alaw imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.worker.html?pcm_mulaw The following tests, previously constantly failing on all Apple platforms, regressed at 267403@main and are constantly passing after 267743@main: imported/w3c/web-platform-tests/webcodecs/audio-decoder.https.any.html imported/w3c/web-platform-tests/webcodecs/audio-decoder.https.any.worker.html The following tests, previously constantly failing on all Apple platforms, regressed at 267963@main and are constantly passing after 268781@main: imported/w3c/web-platform-tests/webcodecs/audio-encoder-config.https.any.html imported/w3c/web-platform-tests/webcodecs/audio-encoder-config.https.any.worker.html The following tests, marked as constant failures, appear never to have been failing: imported/w3c/web-platform-tests/webcodecs/encoded-audio-chunk.any.html imported/w3c/web-platform-tests/webcodecs/encoded-audio-chunk.any.worker.html imported/w3c/web-platform-tests/webcodecs/encoded-audio-chunk.crossOriginIsolated.https.any.html imported/w3c/web-platform-tests/webcodecs/encoded-audio-chunk.crossOriginIsolated.https.any.worker.html