Bug 180696 - createMediaElementSource() not working with Hls stream
Summary: createMediaElementSource() not working with Hls stream
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Web Audio (show other bugs)
Version: Safari 10
Hardware: Mac All
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2017-12-12 07:54 PST by Francesco Cretti
Modified: 2022-08-25 15:13 PDT (History)
10 users (show)

See Also:


Attachments
Source code of demo page (5.77 KB, text/html)
2017-12-12 07:54 PST, Francesco Cretti
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Francesco Cretti 2017-12-12 07:54:31 PST
Created attachment 329112 [details]
Source code of demo page

Hi everyone,
I'm developing an interactive web application based on WebAudio Api and Hls streams.

I know that a previous issue has been fixed (https://bugs.webkit.org/show_bug.cgi?id=143332), that prevented to import audio content from an HTML5 <audio> tag inside the WebAudio routing graph.

I found out that if the content source of the HTML5 <audio> tag it's an HLS stream (or other stream protocols, such as icecast) the bud is still present.

I prepared an online demo here: http://anaconda.polito.it:9080/~cretti/Hls_createMediaElementsource/

In this page an HLS stream is loaded with a an HTML5 <audio> tag and then is wrapped inside the WebAudio API Context with the createMediaElementSource() method. 
Later, the song is filtered with a LPF at 500 Hz using a BiquadFilter Node. Furthermore, the song volume is modulated by the mouse vertical position, such as in this example. 

The script works with Chrome, Firefox or Opera but it does not in Safari, where the stream plays, but the WebAudioAPI cannot apply any effect.

Thanks in any case,
Francesco
Comment 1 Jaewon Choi 2017-12-26 15:04:56 PST
Yeah, I also visited here for the first time to report the same problem.

Hope it gets fixed soon.
Comment 2 Radar WebKit Bug Importer 2018-01-12 11:45:56 PST
<rdar://problem/36477841>
Comment 3 Francesco Cretti 2018-01-17 01:01:27 PST
I would also add that on iOS the createMeadiaElementSource() does not work with an entire MP3 file either, while it works on Safari desktop (https://bugs.webkit.org/show_bug.cgi?id=143332).
Comment 4 Tom 2018-02-01 08:35:58 PST
We have a similar problem on desktop. It doesn't seem to work with MSE. It's as if the element is not routed into the web audio graph.

The sound is still heard if nothing is connected to the audio context destination, and any gain nodes have no effect.

Would be great to have this fixed as it means we currently cannot enable a feature on Safari.
Comment 5 Tom 2018-02-01 08:36:49 PST
This might also be related: https://openradar.appspot.com/37084774
Comment 6 Guy 2021-10-12 18:55:31 PDT
This is still not working also for Safari 15 version. 

createMediaElementSource will work ok with Audio mp3 or Video mp4 as the source for the HTMLMediaElement, but for m3u8/hls stream, as the source the HTMLMediaElement's audio is not correctly routed to the AudioContext and instead the audio is not passed.
Comment 7 Ollie 2022-01-18 04:25:51 PST
Is this ever going to be fixed or even looked at? 3 years later and we still can't do audio visualisations from live HLS/MP3 streams in Safari, this is really holding our product back currently as we have to provide a CSS fallback.

Related 
https://bugs.webkit.org/show_bug.cgi?id=195043
https://bugs.webkit.org/show_bug.cgi?id=231656

Multiple reports on various JS libs as well.

https://github.com/foobar404/Wave.js/issues/17
https://audiomotion.dev/#/?id=visualization-of-live-streams-wont-work-on-safari
https://discourse.threejs.org/t/audioanalyser-not-working-with-audio-source-as-stream-type-in-safari/18950
Comment 8 Sam Sneddon [:gsnedders] 2022-02-14 19:03:19 PST
<rdar://84328473>
Comment 9 justin 2022-08-25 15:13:26 PDT
Our product is also severely impacted by this bug. 

While other major browsers like Chrome and Firefox have no issue with using HLS-backed media elements within their Web Audio contexts, Safari still seems to not support this. 

We have confirmed that this bug persists on Safari 15.5 and Safari Tech Preview 16.0 even in the case where all HLS assets are hosted from the same origin, so it seems that Safari is not adhering to the W3C spec (based on my understanding).

We would greatly appreciate if this bug could receive higher priority and be resolved soon. Thanks!