imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode-interface/no-dezippering.html https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fwebaudio%2Fthe-audio-api%2Fthe-biquadfilternode-interface%2Fno-dezippering.html This always fails on iOS simulator arm64 bot. It's not clear whether this is a regression from GPUP work because we have no other arm64 bots. It appears skipped on Mac arm64 bots.
--- /Volumes/Data/worker/ios-simulator-15-release-gpuprocess-arm64-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode-interface/no-dezippering-expected.txt +++ /Volumes/Data/worker/ios-simulator-15-release-gpuprocess-arm64-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode-interface/no-dezippering-actual.txt @@ -36,9 +36,20 @@ PASS Output from gain setter matches setValueAtTime output is true. PASS < [Test 3] All assertions passed. (total 5 assertions) PASS > [Test 4] No dezippering of frequency vs JS filter -PASS Output from lowpass filter equals [expected array] with an element-wise tolerance of {"absoluteThreshold":5.9607e-7,"relativeThreshold":0}. -PASS Output matches JS filter results is true. -PASS < [Test 4] All assertions passed. (total 2 assertions) +FAIL X Output from lowpass filter does not equal [expected array] with an element-wise tolerance of {"absoluteThreshold":5.9607e-7,"relativeThreshold":0}. + Index Actual Expected AbsError RelError Test threshold + [82] -5.640129e-1 -5.640122e-1 6.556511e-7 1.162477e-6 5.960700e-7 + [83] -4.615946e-1 -4.615939e-1 6.854534e-7 1.484971e-6 5.960700e-7 + [84] -3.461582e-1 -3.461576e-1 6.854534e-7 1.980178e-6 5.960700e-7 + [85] -2.209939e-1 -2.209932e-1 6.705523e-7 3.034267e-6 5.960700e-7 + [86] -8.966466e-2 -8.966400e-2 6.556511e-7 7.312311e-6 5.960700e-7 + ...and 1 more errors. + Max AbsError of 6.854534e-7 at index of 83. + Max RelError of 1.419336e-5 at index of 87. + [87] 4.409384e-2 4.409447e-2 6.258488e-7 1.419336e-5 5.960700e-7 + assert_true: expected true got false +FAIL X Output matches JS filter results is not true. Got false. assert_true: expected true got false +FAIL < [Test 4] 2 out of 2 assertions were failed. assert_true: expected true got false PASS > [Test 5] Test with modulation PASS Modulation: At time 0, frequency is equal to 350. PASS Modulation: At time 0.015625, frequency is equal to 10. @@ -46,5 +57,5 @@ PASS Modulation: Output from frequency setter equals [expected array] with an element-wise tolerance of {"absoluteThreshold":0,"relativeThreshold":0}. PASS Modulation: Output from frequency setter matches setValueAtTime output is true. PASS < [Test 5] All assertions passed. (total 5 assertions) -PASS # AUDIT TASK RUNNER FINISHED: 6 tasks ran successfully. +FAIL # AUDIT TASK RUNNER FINISHED: 1 out of 6 tasks were failed. assert_true: expected true got false
(In reply to Alexey Proskuryakov from comment #1) > --- > /Volumes/Data/worker/ios-simulator-15-release-gpuprocess-arm64-tests-wk2/ > build/layout-test-results/imported/w3c/web-platform-tests/webaudio/the-audio- > api/the-biquadfilternode-interface/no-dezippering-expected.txt > +++ > /Volumes/Data/worker/ios-simulator-15-release-gpuprocess-arm64-tests-wk2/ > build/layout-test-results/imported/w3c/web-platform-tests/webaudio/the-audio- > api/the-biquadfilternode-interface/no-dezippering-actual.txt > @@ -36,9 +36,20 @@ > PASS Output from gain setter matches setValueAtTime output is true. > PASS < [Test 3] All assertions passed. (total 5 assertions) > PASS > [Test 4] No dezippering of frequency vs JS filter > -PASS Output from lowpass filter equals [expected array] with an > element-wise tolerance of > {"absoluteThreshold":5.9607e-7,"relativeThreshold":0}. > -PASS Output matches JS filter results is true. > -PASS < [Test 4] All assertions passed. (total 2 assertions) > +FAIL X Output from lowpass filter does not equal [expected array] with an > element-wise tolerance of > {"absoluteThreshold":5.9607e-7,"relativeThreshold":0}. > + Index Actual Expected AbsError RelError Test threshold > + [82] -5.640129e-1 -5.640122e-1 6.556511e-7 1.162477e-6 5.960700e-7 > + [83] -4.615946e-1 -4.615939e-1 6.854534e-7 1.484971e-6 5.960700e-7 > + [84] -3.461582e-1 -3.461576e-1 6.854534e-7 1.980178e-6 5.960700e-7 > + [85] -2.209939e-1 -2.209932e-1 6.705523e-7 3.034267e-6 5.960700e-7 > + [86] -8.966466e-2 -8.966400e-2 6.556511e-7 7.312311e-6 5.960700e-7 > + ...and 1 more errors. > + Max AbsError of 6.854534e-7 at index of 83. > + Max RelError of 1.419336e-5 at index of 87. > + [87] 4.409384e-2 4.409447e-2 6.258488e-7 1.419336e-5 5.960700e-7 > + assert_true: expected true got false > +FAIL X Output matches JS filter results is not true. Got false. > assert_true: expected true got false > +FAIL < [Test 4] 2 out of 2 assertions were failed. assert_true: expected > true got false > PASS > [Test 5] Test with modulation > PASS Modulation: At time 0, frequency is equal to 350. > PASS Modulation: At time 0.015625, frequency is equal to 10. > @@ -46,5 +57,5 @@ > PASS Modulation: Output from frequency setter equals [expected array] > with an element-wise tolerance of > {"absoluteThreshold":0,"relativeThreshold":0}. > PASS Modulation: Output from frequency setter matches setValueAtTime > output is true. > PASS < [Test 5] All assertions passed. (total 5 assertions) > -PASS # AUDIT TASK RUNNER FINISHED: 6 tasks ran successfully. > +FAIL # AUDIT TASK RUNNER FINISHED: 1 out of 6 tasks were failed. > assert_true: expected true got false The actual and expected values are very close. This just looks like a precision issue because we're dealing with floating point values and vectorization (and different CPU architectures). I believe we'll just need to update the test to use a slightly error tolerance. In the mean time, a platform-specific baseline is the right thing to do IMO.
<rdar://problem/91607355>
Doesn't reproduce on my machine with either: Tools/Scripts/run-webkit-tests --release --no-build --ios-simulator --force imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode-interface/no-dezippering.html or Tools/Scripts/run-webkit-tests --release --no-build --ios-simulator --force imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode-interface/no-dezippering.html --use-gpu-process
(In reply to Chris Dumez from comment #4) > Doesn't reproduce on my machine with either: > Tools/Scripts/run-webkit-tests --release --no-build --ios-simulator --force > imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode- > interface/no-dezippering.html > or > Tools/Scripts/run-webkit-tests --release --no-build --ios-simulator --force > imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode- > interface/no-dezippering.html --use-gpu-process I guess because my host is Intel. This probably need arm64 to repro.
Created attachment 457555 [details] Patch
This patch modifies the imported WPT tests. Please ensure that any changes on the tests (not coming from a WPT import) are exported to WPT. Please see https://trac.webkit.org/wiki/WPTExportProcess
Committed r292832 (249605@main): <https://commits.webkit.org/249605@main> All reviewed patches have been landed. Closing bug and clearing flags on attachment 457555 [details].
The test has recovered on the bot and I have submitted the PR upstream: https://github.com/web-platform-tests/wpt/pull/33642.