Bug 216813

Summary:

REGRESSION(r267383): fast/mediastream/getUserMedia-webaudio.html is failing

Product:

WebKit

Reporter:

Lauro Moura <lmoura>

Component:

Media

Assignee:

Philippe Normand <pnormand>

Status:

RESOLVED FIXED

Severity:

Normal

CC:

aperez, bugs-noreply, calvaris, cdumez, dpino, eric.carlson, ews-watchlist, glenn, jer.noble, philipj, pnormand, sergio, vjaquez, webkit-bug-importer

Priority:

Keywords:

InRadar

Version:

WebKit Nightly Build

Hardware:

Unspecified

OS:

Unspecified

Attachments:

Description	Flags
Patch	aperez: review+

Description Lauro Moura 2020-09-21 20:44:52 PDT

fast/mediastream/getUserMedia-webaudio.html

--- /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/fast/mediastream/getUserMedia-webaudio-expected.txt
+++ /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/fast/mediastream/getUserMedia-webaudio-actual.txt
@@ -2,5 +2,5 @@
 PASS Plugging in getUserMedia audio stream into Web Audio 
 PASS Web Audio should work even if number of channels of a track increases from 1 to 2 
 PASS Web Audio should work with mock audio destination 
-PASS Check echo cancellation can be disabled in getUserMedia call 
+FAIL Check echo cancellation can be disabled in getUserMedia call assert_true: heard noise expected true got false

Comment 1 Diego Pino 2020-09-22 10:44:37 PDT

The following test also started failing in the same revision:

  imported/w3c/web-platform-tests/webaudio/the-audio-api/the-analysernode-interface/realtimeanalyser-fft-scaling.html [ Failure ]
  webaudio/Analyser/realtimeanalyser-freq-data-smoothing.html [ Failure ]
  webaudio/Analyser/realtimeanalyser-freq-data.html [ Failure ]
  webrtc/audio-peer-connection-webaudio.html [ Failure ]
  webrtc/peer-connection-audio-unmute.html [ Failure ]


Diff: https://build.webkit.org/results/GTK%20Linux%2064-bit%20Release%20(Tests)/r267411%20(15970)/imported/w3c/web-platform-tests/webaudio/the-audio-api/the-analysernode-interface/realtimeanalyser-fft-scaling-diff.txt

--- /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/imported/w3c/web-platform-tests/webaudio/the-audio-api/the-analysernode-interface/realtimeanalyser-fft-scaling-expected.txt
+++ /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/imported/w3c/web-platform-tests/webaudio/the-audio-api/the-analysernode-interface/realtimeanalyser-fft-scaling-actual.txt
@@ -3,28 +3,28 @@
 PASS Executing "FFT scaling tests" 
 PASS Audit report 
 PASS > [FFT scaling tests] Test Scaling of FFT in AnalyserNode 
-PASS   32-point FFT peak position is equal to 1. 
-PASS   32-point FFT peak value in dBFS is greater than or equal to -14.43. 
-PASS   64-point FFT peak position is equal to 2. 
-PASS   64-point FFT peak value in dBFS is greater than or equal to -13.56. 
-PASS   128-point FFT peak position is equal to 4. 
-PASS   128-point FFT peak value in dBFS is greater than or equal to -13.56. 
-PASS   256-point FFT peak position is equal to 8. 
-PASS   256-point FFT peak value in dBFS is greater than or equal to -13.56. 
-PASS   512-point FFT peak position is equal to 16. 
-PASS   512-point FFT peak value in dBFS is greater than or equal to -13.56. 
-PASS   1024-point FFT peak position is equal to 32. 
-PASS   1024-point FFT peak value in dBFS is greater than or equal to -13.56. 
-PASS   2048-point FFT peak position is equal to 64. 
-PASS   2048-point FFT peak value in dBFS is greater than or equal to -13.56. 
-PASS   4096-point FFT peak position is equal to 128. 
-PASS   4096-point FFT peak value in dBFS is greater than or equal to -13.56. 
-PASS   8192-point FFT peak position is equal to 256. 
-PASS   8192-point FFT peak value in dBFS is greater than or equal to -13.56. 
-PASS   16384-point FFT peak position is equal to 512. 
-PASS   16384-point FFT peak value in dBFS is greater than or equal to -13.56. 
-PASS   32768-point FFT peak position is equal to 1024. 
-PASS   32768-point FFT peak value in dBFS is greater than or equal to -13.56. 
-PASS < [FFT scaling tests] All assertions passed. (total 22 assertions) 
-PASS # AUDIT TASK RUNNER FINISHED: 1 tasks ran successfully. 
+FAIL X 32-point FFT peak position is not equal to 1. Got 0. assert_true: expected true got false
+FAIL X 32-point FFT peak value in dBFS is not greater than or equal to -14.43. Got -1000. assert_true: expected true got false
+FAIL X 64-point FFT peak position is not equal to 2. Got 0. assert_true: expected true got false
+FAIL X 64-point FFT peak value in dBFS is not greater than or equal to -13.56. Got -1000. assert_true: expected true got false
+FAIL X 128-point FFT peak position is not equal to 4. Got 0. assert_true: expected true got false
+FAIL X 128-point FFT peak value in dBFS is not greater than or equal to -13.56. Got -1000. assert_true: expected true got false
+FAIL X 256-point FFT peak position is not equal to 8. Got 0. assert_true: expected true got false
+FAIL X 256-point FFT peak value in dBFS is not greater than or equal to -13.56. Got -1000. assert_true: expected true got false
+FAIL X 512-point FFT peak position is not equal to 16. Got 0. assert_true: expected true got false
+FAIL X 512-point FFT peak value in dBFS is not greater than or equal to -13.56. Got -1000. assert_true: expected true got false
+FAIL X 1024-point FFT peak position is not equal to 32. Got 0. assert_true: expected true got false
+FAIL X 1024-point FFT peak value in dBFS is not greater than or equal to -13.56. Got -1000. assert_true: expected true got false
+FAIL X 2048-point FFT peak position is not equal to 64. Got 0. assert_true: expected true got false
+FAIL X 2048-point FFT peak value in dBFS is not greater than or equal to -13.56. Got -1000. assert_true: expected true got false
+FAIL X 4096-point FFT peak position is not equal to 128. Got 0. assert_true: expected true got false
+FAIL X 4096-point FFT peak value in dBFS is not greater than or equal to -13.56. Got -1000. assert_true: expected true got false
+FAIL X 8192-point FFT peak position is not equal to 256. Got 0. assert_true: expected true got false
+FAIL X 8192-point FFT peak value in dBFS is not greater than or equal to -13.56. Got -1000. assert_true: expected true got false
+FAIL X 16384-point FFT peak position is not equal to 512. Got 0. assert_true: expected true got false
+FAIL X 16384-point FFT peak value in dBFS is not greater than or equal to -13.56. Got -1000. assert_true: expected true got false
+FAIL X 32768-point FFT peak position is not equal to 1024. Got 0. assert_true: expected true got false
+FAIL X 32768-point FFT peak value in dBFS is not greater than or equal to -13.56. Got -1000. assert_true: expected true got false
+FAIL < [FFT scaling tests] 22 out of 22 assertions were failed. assert_true: expected true got false
+FAIL # AUDIT TASK RUNNER FINISHED: 1 out of 1 tasks were failed. assert_true: expected true got false

Diff: https://build.webkit.org/results/GTK%20Linux%2064-bit%20Release%20(Tests)/r267411%20(15970)/webaudio/Analyser/realtimeanalyser-freq-data-smoothing-diff.txt

--- /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/webaudio/Analyser/realtimeanalyser-freq-data-smoothing-expected.txt
+++ /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/webaudio/Analyser/realtimeanalyser-freq-data-smoothing-actual.txt
@@ -3,10 +3,55 @@
 PASS Executing "smoothing test" 
 PASS Audit report 
 PASS > [smoothing test]  
-PASS   First 512-point FFT at frame 512 equals [expected array] with an element-wise tolerance of {"absoluteThreshold":0,"relativeThreshold":0.0000059207}. 
-PASS   512-point byte FFT equals [0,0,14,28,43,209,255,255,252,186,37,53,197,242,250,225...] with an element-wise tolerance of {"absoluteThreshold":0,"relativeThreshold":0}. 
-PASS   Smoothed 512-point FFT at frame 1536 equals [expected array] with an element-wise tolerance of {"absoluteThreshold":0,"relativeThreshold":0.000025332}. 
-PASS   512-point byte FFT equals [19,25,36,46,52,222,255,255,255,199,45,77,209,255,255,238...] with an element-wise tolerance of {"absoluteThreshold":0,"relativeThreshold":0}. 
-PASS < [smoothing test] All assertions passed. (total 4 assertions) 
-PASS # AUDIT TASK RUNNER FINISHED: 1 tasks ran successfully. 
+FAIL X First 512-point FFT at frame 512 does not equal [-100,-100,-96.06226348876953,-92.18651580810547,-88.14221954345703,-42.45729446411133,-28.346561431884766,-24.99445915222168,-30.678197860717773,-48.66815185546875,-89.71967315673828,-85.35321807861328,-45.8773307800293,-33.445945739746094,-31.21860694885254,-38.1253547668457...] with an element-wise tolerance of {"absoluteThreshold":0,"relativeThreshold":0.0000059207}.
+	Index	Actual			Expected		AbsError		RelError		Test threshold
+	[2]	-1.0000000000000000e+2	-9.6062263488769531e+1	3.9377365112304688e+0	4.0991502471632087e-2	5.6875584343795771e-4
+	[3]	-1.0000000000000000e+2	-9.2186515808105469e+1	7.8134841918945313e+0	8.4757343559431209e-2	5.4580870414505005e-4
+	[4]	-1.0000000000000000e+2	-8.8142219543457031e+1	1.1857780456542969e+1	1.3453008692045348e-1	5.2186363925094603e-4
+	[5]	-1.0000000000000000e+2	-4.2457294464111328e+1	5.7542705535888672e+1	1.3553078749407566e+0	2.5137690333366393e-4
+	[6]	-1.0000000000000000e+2	-2.8346561431884766e+1	7.1653438568115234e+1	2.5277647428346652e+0	1.6783148626976012e-4
+	...and 177 more errors.
+	Max AbsError of 7.5005540847778320e+1 at index of 7.
+	[7]	-1.0000000000000000e+2	-2.4994459152221680e+1	7.5005540847778320e+1	3.0008867321744512e+0	1.4798469430255890e-4
+	Max RelError of 3.0008867321744512e+0 at index of 7.
+	[7]	-1.0000000000000000e+2	-2.4994459152221680e+1	7.5005540847778320e+1	3.0008867321744512e+0	1.4798469430255890e-4
+ assert_true: expected true got false
+FAIL X 512-point byte FFT does not equal [0,0,14,28,43,209,255,255,252,186,37,53,197,242,250,225...] with an element-wise tolerance of {"absoluteThreshold":0,"relativeThreshold":0}.
+	Index	Actual			Expected		AbsError		RelError		Test threshold
+	[2]	0.0000000000000000e+0	1.4000000000000000e+1	1.4000000000000000e+1	1.0000000000000000e+0	0.0000000000000000e+0
+	[3]	0.0000000000000000e+0	2.8000000000000000e+1	2.8000000000000000e+1	1.0000000000000000e+0	0.0000000000000000e+0
+	[4]	0.0000000000000000e+0	4.3000000000000000e+1	4.3000000000000000e+1	1.0000000000000000e+0	0.0000000000000000e+0
+	[5]	0.0000000000000000e+0	2.0900000000000000e+2	2.0900000000000000e+2	1.0000000000000000e+0	0.0000000000000000e+0
+	[6]	0.0000000000000000e+0	2.5500000000000000e+2	2.5500000000000000e+2	1.0000000000000000e+0	0.0000000000000000e+0
+	...and 176 more errors.
+	Max AbsError of 2.5500000000000000e+2 at index of 6.
+	Max RelError of 1.0000000000000000e+0 at index of 2.
+ assert_true: expected true got false
+FAIL X Smoothed 512-point FFT at frame 1536 does not equal [-94.55308532714844,-93.13590240478516,-90.07640838623047,-87.21326446533203,-85.55164337158203,-38.93035125732422,-24.825944900512695,-21.471540451049805,-27.1594295501709,-45.108985900878906,-87.53975677490234,-78.74081420898438,-42.37689208984375,-29.92169761657715,-27.697038650512695,-34.60657501220703...] with an element-wise tolerance of {"absoluteThreshold":0,"relativeThreshold":0.000025332}.
+	Index	Actual			Expected		AbsError		RelError		Test threshold
+	[0]	-1.0000000000000000e+2	-9.4553085327148438e+1	5.4469146728515625e+0	5.7606948033536290e-2	2.3952187575073244e-3
+	[1]	-1.0000000000000000e+2	-9.3135902404785156e+1	6.8640975952148438e+0	7.3699802310201040e-2	2.3593186797180176e-3
+	[2]	-1.0000000000000000e+2	-9.0076408386230469e+1	9.9235916137695313e+0	1.1016859787769359e-1	2.2818155772399902e-3
+	[3]	-1.0000000000000000e+2	-8.7213264465332031e+1	1.2786735534667969e+1	1.4661457305901901e-1	2.2092864154357912e-3
+	[4]	-1.0000000000000000e+2	-8.5551643371582031e+1	1.4448356628417969e+1	1.6888461821432790e-1	2.1671942298889160e-3
+	...and 187 more errors.
+	Max AbsError of 7.8528459548950195e+1 at index of 7.
+	[7]	-1.0000000000000000e+2	-2.1471540451049805e+1	7.8528459548950195e+1	3.6573276951403231e+0	5.4391706270599371e-4
+	Max RelError of 3.6573276951403231e+0 at index of 7.
+	[7]	-1.0000000000000000e+2	-2.1471540451049805e+1	7.8528459548950195e+1	3.6573276951403231e+0	5.4391706270599371e-4
+ assert_true: expected true got false
+FAIL X 512-point byte FFT does not equal [19,25,36,46,52,222,255,255,255,199,45,77,209,255,255,238...] with an element-wise tolerance of {"absoluteThreshold":0,"relativeThreshold":0}.
+	Index	Actual			Expected		AbsError		RelError		Test threshold
+	[0]	0.0000000000000000e+0	1.9000000000000000e+1	1.9000000000000000e+1	1.0000000000000000e+0	0.0000000000000000e+0
+	[1]	0.0000000000000000e+0	2.5000000000000000e+1	2.5000000000000000e+1	1.0000000000000000e+0	0.0000000000000000e+0
+	[2]	0.0000000000000000e+0	3.6000000000000000e+1	3.6000000000000000e+1	1.0000000000000000e+0	0.0000000000000000e+0
+	[3]	0.0000000000000000e+0	4.6000000000000000e+1	4.6000000000000000e+1	1.0000000000000000e+0	0.0000000000000000e+0
+	[4]	0.0000000000000000e+0	5.2000000000000000e+1	5.2000000000000000e+1	1.0000000000000000e+0	0.0000000000000000e+0
+	...and 187 more errors.
+	Max AbsError of 2.5500000000000000e+2 at index of 6.
+	[6]	0.0000000000000000e+0	2.5500000000000000e+2	2.5500000000000000e+2	1.0000000000000000e+0	0.0000000000000000e+0
+	Max RelError of 1.0000000000000000e+0 at index of 0.
+ assert_true: expected true got false
+FAIL < [smoothing test] 4 out of 4 assertions were failed. assert_true: expected true got false
+FAIL # AUDIT TASK RUNNER FINISHED: 1 out of 1 tasks were failed. assert_true: expected true got false
 
Diff: https://build.webkit.org/results/GTK%20Linux%2064-bit%20Release%20(Tests)/r267411%20(15970)/webrtc/audio-peer-connection-webaudio-diff.txt

--- /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/webrtc/audio-peer-connection-webaudio-expected.txt
+++ /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/webrtc/audio-peer-connection-webaudio-actual.txt
@@ -1,3 +1,3 @@
 
-PASS Basic audio playback through a peer connection 
+FAIL Basic audio playback through a peer connection assert_true: heard hum bip bop expected true got false
 
Diff: https://build.webkit.org/results/GTK%20Linux%2064-bit%20Release%20(Tests)/r267411%20(15970)/webrtc/peer-connection-audio-unmute-diff.txt

--- /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/webrtc/peer-connection-audio-unmute-expected.txt
+++ /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/webrtc/peer-connection-audio-unmute-actual.txt
@@ -1,3 +1,3 @@
 
-PASS Muting a local audio track before adding it should be correctly handled 
+FAIL Muting a local audio track before adding it should be correctly handled assert_true: Should hear hum expected true got false

Comment 2 Chris Dumez 2020-09-22 11:22:22 PDT

Something is wrong with the FFTFrameGsteamer implementation it seems.

Comment 3 Chris Dumez 2020-09-22 12:05:21 PDT

I re-introduced the GTK-specific implementation of FFTFrame::multiply() in <https://trac.webkit.org/changeset/267428> to try and be more conservative.

I could use help from a GTK person to figure out these issues if they persist since I am unable to reproduce the issue.

Comment 4 Chris Dumez 2020-09-22 12:11:18 PDT

(In reply to Chris Dumez from comment #3)
> I re-introduced the GTK-specific implementation of FFTFrame::multiply() in
> <https://trac.webkit.org/changeset/267428> to try and be more conservative.
> 
> I could use help from a GTK person to figure out these issues if they
> persist since I am unable to reproduce the issue.

One difference between the GTK and the Mac implementation is that FFTFrame::multiply() does this on Mac but not on GTK:
    // Multiply the packed DC/nyquist component
    realP1[0] = real0 * realP2[0];
    imagP1[0] = imag0 * imagP2[0];

Comment 5 Chris Dumez 2020-09-22 12:14:28 PDT

Would anyone be able to try to following change to see if it helps?

diff --git a/Source/WebCore/platform/audio/gstreamer/FFTFrameGStreamer.cpp b/Source/WebCore/platform/audio/gstreamer/FFTFrameGStreamer.cpp
index 213dd93d8540..21438f80aa1a 100644
--- a/Source/WebCore/platform/audio/gstreamer/FFTFrameGStreamer.cpp
+++ b/Source/WebCore/platform/audio/gstreamer/FFTFrameGStreamer.cpp
@@ -113,8 +113,15 @@ void FFTFrame::multiply(const FFTFrame& frame)
     const float* realP2 = frame2.realData();
     const float* imagP2 = frame2.imagData();
 
+    float real0 = realP1[0];
+    float imag0 = imagP1[0];
+
     size_t size = unpackedFFTDataSize(m_FFTSize);
     VectorMath::zvmul(realP1, imagP1, realP2, imagP2, realP1, imagP1, size);
+
+    // Multiply the packed DC/nyquist component
+    realP1[0] = real0 * realP2[0];
+    imagP1[0] = imag0 * imagP2[0];
 }
 
 void FFTFrame::doFFT(const float* data)

Comment 6 Víctor M. Jáquez L. 2020-09-23 04:09:02 PDT

(In reply to Chris Dumez from comment #5)
> Would anyone be able to try to following change to see if it helps?
> 
> diff --git a/Source/WebCore/platform/audio/gstreamer/FFTFrameGStreamer.cpp
> b/Source/WebCore/platform/audio/gstreamer/FFTFrameGStreamer.cpp
> index 213dd93d8540..21438f80aa1a 100644
> --- a/Source/WebCore/platform/audio/gstreamer/FFTFrameGStreamer.cpp
> +++ b/Source/WebCore/platform/audio/gstreamer/FFTFrameGStreamer.cpp
> @@ -113,8 +113,15 @@ void FFTFrame::multiply(const FFTFrame& frame)
>      const float* realP2 = frame2.realData();
>      const float* imagP2 = frame2.imagData();
>  
> +    float real0 = realP1[0];
> +    float imag0 = imagP1[0];
> +
>      size_t size = unpackedFFTDataSize(m_FFTSize);
>      VectorMath::zvmul(realP1, imagP1, realP2, imagP2, realP1, imagP1, size);
> +
> +    // Multiply the packed DC/nyquist component
> +    realP1[0] = real0 * realP2[0];
> +    imagP1[0] = imag0 * imagP2[0];
>  }
>  
>  void FFTFrame::doFFT(const float* data)

This patch, doing a preliminary test, seems to fix 

imported/w3c/web-platform-tests/webaudio/the-audio-api/the-analysernode-interface/realtimeanalyser-fft-scaling.html
webaudio/Analyser/realtimeanalyser-freq-data-smoothing.html
webaudio/Analyser/realtimeanalyser-freq-data.html
webrtc/audio-peer-connection-webaudio.html
webrtc/peer-connection-audio-unmute.html

But not fast/mediastream/getUserMedia-webaudio.html

Comment 7 Víctor M. Jáquez L. 2020-09-23 04:19:32 PDT

(In reply to Víctor M. Jáquez L. from comment #6)
> (In reply to Chris Dumez from comment #5)
> > Would anyone be able to try to following change to see if it helps?
> > 
> > diff --git a/Source/WebCore/platform/audio/gstreamer/FFTFrameGStreamer.cpp
> > b/Source/WebCore/platform/audio/gstreamer/FFTFrameGStreamer.cpp
> > index 213dd93d8540..21438f80aa1a 100644
> > --- a/Source/WebCore/platform/audio/gstreamer/FFTFrameGStreamer.cpp
> > +++ b/Source/WebCore/platform/audio/gstreamer/FFTFrameGStreamer.cpp
> > @@ -113,8 +113,15 @@ void FFTFrame::multiply(const FFTFrame& frame)
> >      const float* realP2 = frame2.realData();
> >      const float* imagP2 = frame2.imagData();
> >  
> > +    float real0 = realP1[0];
> > +    float imag0 = imagP1[0];
> > +
> >      size_t size = unpackedFFTDataSize(m_FFTSize);
> >      VectorMath::zvmul(realP1, imagP1, realP2, imagP2, realP1, imagP1, size);
> > +
> > +    // Multiply the packed DC/nyquist component
> > +    realP1[0] = real0 * realP2[0];
> > +    imagP1[0] = imag0 * imagP2[0];
> >  }
> >  
> >  void FFTFrame::doFFT(const float* data)
> 
> This patch, doing a preliminary test, seems to fix 
> 
> imported/w3c/web-platform-tests/webaudio/the-audio-api/the-analysernode-
> interface/realtimeanalyser-fft-scaling.html
> webaudio/Analyser/realtimeanalyser-freq-data-smoothing.html
> webaudio/Analyser/realtimeanalyser-freq-data.html
> webrtc/audio-peer-connection-webaudio.html
> webrtc/peer-connection-audio-unmute.html
> 
> But not fast/mediastream/getUserMedia-webaudio.html

False alarm. Sorry, I didn't unflagged the tests. :(

Comment 8 Philippe Normand 2020-09-23 05:01:11 PDT

Created attachment 409462 [details]
Patch

Comment 9 Philippe Normand 2020-09-23 06:36:48 PDT

Committed r267471: <https://trac.webkit.org/changeset/267471>

Comment 10 Radar WebKit Bug Importer 2020-09-23 06:37:18 PDT

<rdar://problem/69432656>

Comment 11 Chris Dumez 2020-09-23 08:18:22 PDT

Nice, Thanks for fixing Philippe!