Bug 236081 – [WPE][GStreamer] Increased CPU usage on new window.AudioContext()

• Status: RESOLVED
• Resolution: WORKSFORME
• Component: WPE WebKit

[Bastian Krause]

Hi,

I'm observing high CPU loads after "new window.AudioContext()" calls (factor 50 on an i.MX6 Quad). The reason for this seems to be the "audiointerleave" GStreamer element. I did not see this on 2.30.3, but sound did not work back then in the affected application. The high load seems to affect all stables releases (at least) since wpewebkit 2.32.3 (I did not test 2.32.{0,1,2}).

[Philippe Normand]

Do you see this after creating an AudioContext and do nothing with it? I can't reproduce this here. Please share a complete use-case.

[Bastian Krause]

Created attachment 451083 [details]
AudioContext reproducer

[Bastian Krause]

I think my comparison with 2.30.x is misleading. It seems that the GStreamer pipeline might have been constructed at a different point in time then (not on window.AudioContext()).

On wpewebkit 2.34.4 with gstreamer 1.20, I see less CPU load compared to wpewebkit 2.32.3 with gstreamer 1.18.3. It is still half of the overall CPU load from the WPEWebProcess, which is at ~50% (htop) on this SoC.

For testing purposes, I've reverted the "audiointerleave" element to "interleave" (https://bugs.webkit.org/show_bug.cgi?id=211124).
I've also added a GStreamer "queue" element between the "webaudioSrc" elements and "interleave". With this, the main load is on the 2 "queue" elements. This probably means that my issue happens somewhere downstream (audioconvert/audioresample/alsasink), right? So I guess WPE has nothing to do with it.

[Philippe Normand]

I'd need to check, but IIRC the AudioDestination is not explicitly created when the AudioContext is? And even so, (IIRC again :)) I think you need to explicitly start rendering to have the webkitaudiosrc element playing. So I'm not sure why you're seeing this issue...

[Bastian Krause]

(In reply to Philippe Normand from comment #4)
> I'd need to check, but IIRC the AudioDestination is not explicitly created
> when the AudioContext is?
It is created. Setting..

sound_context = new window.AudioContext();
sound_context.destination.channelCount = 1;

..decreases the CPU load, because only 1 webaudioSrc feeds into the audiointerleave.

> And even so, (IIRC again :)) I think you need to
> explicitly start rendering to have the webkitaudiosrc element playing. So
> I'm not sure why you're seeing this issue...
No. With GST_DEBUG_DUMP_DOT_DIR=/some/path , I see a playing pipeline with nothing else in the DOM/javascript as in the reproducer.

The CPU load stems from the pipeline itself. This is the pipeline I came up with to reproduce the load:

gst-launch-1.0 -v audiointerleave name=i ! audioconvert ! audioresample ! alsasink audiotestsrc samplesperbuffer=128 ! queue ! "audio/x-raw,format=F32LE,layout=interleaved,rate=44100,channels=1,channel-mask=(bitmask)0x1" ! i. audiotestsrc samplesperbuffer=128 ! queue ! "audio/x-raw,format=F32LE,layout=interleaved,rate=44100,channels=1,channel-mask=(bitmask)0x2" ! i.

With GStreamer 1.20.0 (compared to 1.18.4) the CPU load is way lower. I guess this is okay, setting the channel number to "1" decreases the load (-2..5%).

So, I think this can be closed, do you agree?

[Bastian Krause]

Argh, it is in fact not an i.MX6 Quad, but a DualLite. Sorry, just too many different prototypes on my desk.