Bug 238790 – imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode-interface/no-dezippering.html

• Status: RESOLVED
• Resolution: FIXED
• Component: Web Audio

[Jon Lee]

imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode-interface/no-dezippering.html

https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fwebaudio%2Fthe-audio-api%2Fthe-biquadfilternode-interface%2Fno-dezippering.html

This always fails on iOS simulator arm64 bot. It's not clear whether this is a regression from GPUP work because we have no other arm64 bots. It appears skipped on Mac arm64 bots.

[Alexey Proskuryakov]

--- /Volumes/Data/worker/ios-simulator-15-release-gpuprocess-arm64-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode-interface/no-dezippering-expected.txt
+++ /Volumes/Data/worker/ios-simulator-15-release-gpuprocess-arm64-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode-interface/no-dezippering-actual.txt
@@ -36,9 +36,20 @@
 PASS   Output from gain setter matches setValueAtTime output is true.
 PASS < [Test 3] All assertions passed. (total 5 assertions)
 PASS > [Test 4] No dezippering of frequency vs JS filter
-PASS   Output from lowpass filter equals [expected array] with an element-wise tolerance of {"absoluteThreshold":5.9607e-7,"relativeThreshold":0}.
-PASS   Output matches JS filter results is true.
-PASS < [Test 4] All assertions passed. (total 2 assertions)
+FAIL X Output from lowpass filter does not equal [expected array] with an element-wise tolerance of {"absoluteThreshold":5.9607e-7,"relativeThreshold":0}.
+	Index	Actual			Expected		AbsError		RelError		Test threshold
+	[82]	-5.640129e-1	-5.640122e-1	6.556511e-7	1.162477e-6	5.960700e-7
+	[83]	-4.615946e-1	-4.615939e-1	6.854534e-7	1.484971e-6	5.960700e-7
+	[84]	-3.461582e-1	-3.461576e-1	6.854534e-7	1.980178e-6	5.960700e-7
+	[85]	-2.209939e-1	-2.209932e-1	6.705523e-7	3.034267e-6	5.960700e-7
+	[86]	-8.966466e-2	-8.966400e-2	6.556511e-7	7.312311e-6	5.960700e-7
+	...and 1 more errors.
+	Max AbsError of 6.854534e-7 at index of 83.
+	Max RelError of 1.419336e-5 at index of 87.
+	[87]	4.409384e-2	4.409447e-2	6.258488e-7	1.419336e-5	5.960700e-7
+ assert_true: expected true got false
+FAIL X Output matches JS filter results is not true. Got false. assert_true: expected true got false
+FAIL < [Test 4] 2 out of 2 assertions were failed. assert_true: expected true got false
 PASS > [Test 5] Test with modulation
 PASS   Modulation: At time 0, frequency is equal to 350.
 PASS   Modulation: At time 0.015625, frequency is equal to 10.
@@ -46,5 +57,5 @@
 PASS   Modulation: Output from frequency setter equals [expected array] with an element-wise tolerance of {"absoluteThreshold":0,"relativeThreshold":0}.
 PASS   Modulation: Output from frequency setter matches setValueAtTime output is true.
 PASS < [Test 5] All assertions passed. (total 5 assertions)
-PASS # AUDIT TASK RUNNER FINISHED: 6 tasks ran successfully.
+FAIL # AUDIT TASK RUNNER FINISHED: 1 out of 6 tasks were failed. assert_true: expected true got false

[Chris Dumez]

(In reply to Alexey Proskuryakov from comment #1)
> ---
> /Volumes/Data/worker/ios-simulator-15-release-gpuprocess-arm64-tests-wk2/
> build/layout-test-results/imported/w3c/web-platform-tests/webaudio/the-audio-
> api/the-biquadfilternode-interface/no-dezippering-expected.txt
> +++
> /Volumes/Data/worker/ios-simulator-15-release-gpuprocess-arm64-tests-wk2/
> build/layout-test-results/imported/w3c/web-platform-tests/webaudio/the-audio-
> api/the-biquadfilternode-interface/no-dezippering-actual.txt
> @@ -36,9 +36,20 @@
>  PASS   Output from gain setter matches setValueAtTime output is true.
>  PASS < [Test 3] All assertions passed. (total 5 assertions)
>  PASS > [Test 4] No dezippering of frequency vs JS filter
> -PASS   Output from lowpass filter equals [expected array] with an
> element-wise tolerance of
> {"absoluteThreshold":5.9607e-7,"relativeThreshold":0}.
> -PASS   Output matches JS filter results is true.
> -PASS < [Test 4] All assertions passed. (total 2 assertions)
> +FAIL X Output from lowpass filter does not equal [expected array] with an
> element-wise tolerance of
> {"absoluteThreshold":5.9607e-7,"relativeThreshold":0}.
> +	Index	Actual			Expected		AbsError		RelError		Test threshold
> +	[82]	-5.640129e-1	-5.640122e-1	6.556511e-7	1.162477e-6	5.960700e-7
> +	[83]	-4.615946e-1	-4.615939e-1	6.854534e-7	1.484971e-6	5.960700e-7
> +	[84]	-3.461582e-1	-3.461576e-1	6.854534e-7	1.980178e-6	5.960700e-7
> +	[85]	-2.209939e-1	-2.209932e-1	6.705523e-7	3.034267e-6	5.960700e-7
> +	[86]	-8.966466e-2	-8.966400e-2	6.556511e-7	7.312311e-6	5.960700e-7
> +	...and 1 more errors.
> +	Max AbsError of 6.854534e-7 at index of 83.
> +	Max RelError of 1.419336e-5 at index of 87.
> +	[87]	4.409384e-2	4.409447e-2	6.258488e-7	1.419336e-5	5.960700e-7
> + assert_true: expected true got false
> +FAIL X Output matches JS filter results is not true. Got false.
> assert_true: expected true got false
> +FAIL < [Test 4] 2 out of 2 assertions were failed. assert_true: expected
> true got false
>  PASS > [Test 5] Test with modulation
>  PASS   Modulation: At time 0, frequency is equal to 350.
>  PASS   Modulation: At time 0.015625, frequency is equal to 10.
> @@ -46,5 +57,5 @@
>  PASS   Modulation: Output from frequency setter equals [expected array]
> with an element-wise tolerance of
> {"absoluteThreshold":0,"relativeThreshold":0}.
>  PASS   Modulation: Output from frequency setter matches setValueAtTime
> output is true.
>  PASS < [Test 5] All assertions passed. (total 5 assertions)
> -PASS # AUDIT TASK RUNNER FINISHED: 6 tasks ran successfully.
> +FAIL # AUDIT TASK RUNNER FINISHED: 1 out of 6 tasks were failed.
> assert_true: expected true got false

The actual and expected values are very close. This just looks like a precision issue because we're dealing with floating point values and vectorization (and different CPU architectures). I believe we'll just need to update the test to use a slightly error tolerance. In the mean time, a platform-specific baseline is the right thing to do IMO.

[Radar WebKit Bug Importer]

<rdar://problem/91607355>

[Chris Dumez]

Doesn't reproduce on my machine with either:
Tools/Scripts/run-webkit-tests --release --no-build --ios-simulator --force imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode-interface/no-dezippering.html
or
Tools/Scripts/run-webkit-tests --release --no-build --ios-simulator --force imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode-interface/no-dezippering.html --use-gpu-process

[Chris Dumez]

(In reply to Chris Dumez from comment #4)
> Doesn't reproduce on my machine with either:
> Tools/Scripts/run-webkit-tests --release --no-build --ios-simulator --force
> imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode-
> interface/no-dezippering.html
> or
> Tools/Scripts/run-webkit-tests --release --no-build --ios-simulator --force
> imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode-
> interface/no-dezippering.html --use-gpu-process

I guess because my host is Intel. This probably need arm64 to repro.

[Chris Dumez]

Created attachment 457555 [details]
Patch

[EWS Watchlist]

This patch modifies the imported WPT tests. Please ensure that any changes on the tests (not coming from a WPT import) are exported to WPT. Please see https://trac.webkit.org/wiki/WPTExportProcess

[EWS]

Committed r292832 (249605@main): <https://commits.webkit.org/249605@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 457555 [details].

[Chris Dumez]

The test has recovered on the bot and I have submitted the PR upstream: https://github.com/web-platform-tests/wpt/pull/33642.