Bug 180696 – createMediaElementSource() not working with Hls stream

• Component: Web Audio
• Status: NEW

[Francesco Cretti]

Created attachment 329112 [details]
Source code of demo page

Hi everyone,
I'm developing an interactive web application based on WebAudio Api and Hls streams.

I know that a previous issue has been fixed (https://bugs.webkit.org/show_bug.cgi?id=143332), that prevented to import audio content from an HTML5 <audio> tag inside the WebAudio routing graph.

I found out that if the content source of the HTML5 <audio> tag it's an HLS stream (or other stream protocols, such as icecast) the bud is still present.

I prepared an online demo here: http://anaconda.polito.it:9080/~cretti/Hls_createMediaElementsource/

In this page an HLS stream is loaded with a an HTML5 <audio> tag and then is wrapped inside the WebAudio API Context with the createMediaElementSource() method. 
Later, the song is filtered with a LPF at 500 Hz using a BiquadFilter Node. Furthermore, the song volume is modulated by the mouse vertical position, such as in this example. 

The script works with Chrome, Firefox or Opera but it does not in Safari, where the stream plays, but the WebAudioAPI cannot apply any effect.

Thanks in any case,
Francesco

[Jaewon Choi]

Yeah, I also visited here for the first time to report the same problem.

Hope it gets fixed soon.

[Radar WebKit Bug Importer]

<rdar://problem/36477841>

[Francesco Cretti]

I would also add that on iOS the createMeadiaElementSource() does not work with an entire MP3 file either, while it works on Safari desktop (https://bugs.webkit.org/show_bug.cgi?id=143332).

[Tom]

We have a similar problem on desktop. It doesn't seem to work with MSE. It's as if the element is not routed into the web audio graph.

The sound is still heard if nothing is connected to the audio context destination, and any gain nodes have no effect.

Would be great to have this fixed as it means we currently cannot enable a feature on Safari.

[Tom]

This might also be related: https://openradar.appspot.com/37084774

[Guy]

This is still not working also for Safari 15 version. 

createMediaElementSource will work ok with Audio mp3 or Video mp4 as the source for the HTMLMediaElement, but for m3u8/hls stream, as the source the HTMLMediaElement's audio is not correctly routed to the AudioContext and instead the audio is not passed.

[Ollie]

Is this ever going to be fixed or even looked at? 3 years later and we still can't do audio visualisations from live HLS/MP3 streams in Safari, this is really holding our product back currently as we have to provide a CSS fallback.

Related 
https://bugs.webkit.org/show_bug.cgi?id=195043
https://bugs.webkit.org/show_bug.cgi?id=231656

Multiple reports on various JS libs as well.

https://github.com/foobar404/Wave.js/issues/17
https://audiomotion.dev/#/?id=visualization-of-live-streams-wont-work-on-safari
https://discourse.threejs.org/t/audioanalyser-not-working-with-audio-source-as-stream-type-in-safari/18950

[Sam Sneddon [:gsnedders]]

<rdar://84328473>

Our product is also severely impacted by this bug. 

While other major browsers like Chrome and Firefox have no issue with using HLS-backed media elements within their Web Audio contexts, Safari still seems to not support this. 

We have confirmed that this bug persists on Safari 15.5 and Safari Tech Preview 16.0 even in the case where all HLS assets are hosted from the same origin, so it seems that Safari is not adhering to the W3C spec (based on my understanding).

We would greatly appreciate if this bug could receive higher priority and be resolved soon. Thanks!

[Benny Lichtner]

Having the same issue here. We are unable to serve HLS media to our ios users until this bug is fixed :(

Seems to be related to or a duplicate of: https://bugs.webkit.org/show_bug.cgi?id=231656

[Sam Sneddon [:gsnedders]]

*** Bug 231656 has been marked as a duplicate of this bug. ***

[Sam Chang]

I have the same issue on latest iOS and macOS Safari.
Leave a comment to raise attention, hopefully.

[brett]

running into the same issue here. has anyone figured out any solutions yet? this seems to be quite old. 

tested on Safari 16.6 (macOS 13.5.2) + Safari Preview Release 178 + iOS 16.6.1.

[Erik Miller-Galow]

I'm running into the same thing, trying to render visuals to react to my icecast stream. It works in all browsers I've tested besides Safari. Audio does play in Safari but I've yet to find a way to retrieve the frequency data from my analyser (it's all zeroes in Safari).

[Robert]

Hello, I had the displeasure of encountering this longstanding bug today when I decided to use Safari to view the website I had been developing
Without getting into details, this is quite a kafkaesque situation considering it all works perfectly on other browsers and there are no easy solutions to this bug.
This is a StackOverflow question from 2016 about this bug: https://stackoverflow.com/questions/38936642
you can see that in October 2024, the original poster came back and said: "This still haunts me 8 years later"

I'd really appreciate if this bug could be fixed to save future devs from PTSD

[Jonas Lidén]

We're also having this issue in our products in the M&E space. We are using the Media Source Extensions API together with Web Audio API and that works great with Chrome and Firefox but not in Safari. We use the Web Audio API to extract waveform data and it's reporting all zeroes. We also use it for other purposes, like audio routing and solo/mute functionality that does not work, it's like the source nodes are not connected to the audio graph.

[dj8000]

Dear Sirs:

I would like to the add to the chorus as regards this bug. The people want stream visualizers! Let them visualize the stream! WebKit's total disregard for createMediaElementSource spec is a major blow to those who would like to explore the richness of the Web Audio API.

[Adesh Pandey]

Hey! I totally agree with @dj800, we have been using visualisers widely and this has been a pain to manage. 
What I have done rn and might be a intermediate fix for people is, play the audio using audio element and then also play it in another audio context (make sure to implement logics that ensure they are always at the same state). 
Now, play the context along with audio element but do not add the gain to the source just set the gain node as 1. 
This works like a charm but very hack.