Bug 146351 - [GStreamer] Audio/video sources do not load via custom URI scheme
Summary: [GStreamer] Audio/video sources do not load via custom URI scheme
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Media (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-06-26 10:50 PDT by Philip Chimento
Modified: 2023-10-21 12:56 PDT (History)
8 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Philip Chimento 2015-06-26 10:50:29 PDT
When trying to play a video loaded via a custom URI scheme, the request for the video file somehow never makes it to the custom URI handler. I tested this on 2.6.2 and 2.8.3.

This is perhaps best illustrated by the following GJS script: https://gist.github.com/ptomato/e180132d2313d07102d2 (You have to specify the path in the custom:// URI to where you put the test.html file, and you have to supply your own video.ogv and subtitles.vtt files in the same directory)

You will notice that the custom URI handler finds test.html correctly (shown by the message on stderr), but does not load video.ogv. In the inspector, you can see that a request for video.ogv was fired, but it never reached the custom URI handler because there was no message for it on stderr. What makes me think this bug is only about videos, is that if you select "Spanish" from the captions menu, then subtitles.vtt is correctly loaded via the custom URI handler and prints out a corresponding message on stderr.
Comment 1 Adrian Perez 2019-10-22 13:47:31 PDT
Hi Philip! Sorry for taking so long in commenting on this bug.
It looks like it has been fixed since: the provided test script
is working for me now with WebKitGTK 2.26.1; I tried using
a WebM video instead, but I expect OGV (as seen originally in
the script) would work as well.

Do you want to double-check with your original video to confirm
it works for you as well before we close the bug?
Comment 2 Philip Chimento 2019-10-22 15:25:10 PDT
Hi,

Thanks for checking, did you have to modify the test script at all to get it to work? I ask because I could not load either a OGV or WebM video, not with 2.24 or 2.26. In the inspector it tells me "The resource was requested insecurely." (The script still works if I comment out the <source> element and uncomment the onload code.)
Comment 3 Adrian Perez 2019-10-23 08:59:39 PDT
(In reply to Philip Chimento from comment #2)
> Hi,
> 
> Thanks for checking, did you have to modify the test script at all to get it
> to work? I ask because I could not load either a OGV or WebM video, not with
> 2.24 or 2.26. In the inspector it tells me "The resource was requested
> insecurely." (The script still works if I comment out the <source> element
> and uncomment the onload code.)

Yes, I had to fix line 15 to avoid a warning about the Gio.content_guess_type()
needing to have two arguments instead of three. As you point out I commented
the <source> element and uncommented the unload code to test, but I never
got to try with the <source> element in there -- I'll try later today when
I have a computer around again.

Just to be 100% clear: I understand that loading the video should work
*both* ways, isn't it?
Comment 4 Philip Chimento 2019-10-23 14:13:02 PDT
(In reply to Adrian Perez from comment #3)
> (In reply to Philip Chimento from comment #2)
> > Hi,
> > 
> > Thanks for checking, did you have to modify the test script at all to get it
> > to work? I ask because I could not load either a OGV or WebM video, not with
> > 2.24 or 2.26. In the inspector it tells me "The resource was requested
> > insecurely." (The script still works if I comment out the <source> element
> > and uncomment the onload code.)
> 
> Yes, I had to fix line 15 to avoid a warning about the
> Gio.content_guess_type()
> needing to have two arguments instead of three. As you point out I commented
> the <source> element and uncommented the unload code to test, but I never
> got to try with the <source> element in there -- I'll try later today when
> I have a computer around again.

True, I also modified line 15, I've updated the gist now to fix that. 

> Just to be 100% clear: I understand that loading the video should work
> *both* ways, isn't it?

That was also my understanding when I filed the bug :-)
Comment 5 Yu-Wei Wu 2022-12-13 03:08:08 PST
I saw the release log in 2.36.5 and thought it's fixed but looks like the issue still remains.
I have tried different video formats but none is working. However, using file URL can still load the video.
Here's my repo to reproduce it: https://github.com/wusyong/gtkbrowser/tree/video
Comment 6 Adrian Perez 2023-02-16 07:21:02 PST
(In reply to Yu-Wei Wu from comment #5)
> I saw the release log in 2.36.5 and thought it's fixed but looks like the
> issue still remains.
> I have tried different video formats but none is working. However, using
> file URL can still load the video.
> Here's my repo to reproduce it:
> https://github.com/wusyong/gtkbrowser/tree/video

Probably the JS code the YouTube website tries to do something with
video elements, and there is some exception being thrown (because of
the media support being disabled in the build) and then other JS code
needed to load the comments (and other content) does never get a chance
to run.
Comment 7 Yu-Wei Wu 2023-02-20 04:18:00 PST
(In reply to Adrian Perez from comment #6)
> Probably the JS code the YouTube website tries to do something with
> video elements, and there is some exception being thrown (because of
> the media support being disabled in the build) and then other JS code
> needed to load the comments (and other content) does never get a chance
> to run.

My reproduced example doesn't include any JS code. Not sure where could I start to look from, but I would like to find the cause too. How do I backtrace in this case? Should I build a webkitgtk from source instead?
Comment 8 Vladimir Porotnikov 2023-08-13 17:38:34 PDT
Hi everyone. Would like to mention that this bug affects not only video but also audio.

I can reproduce it using new Audio("custom:///sound.mp3"), then I get error message in devtools console:
Unhandled Promise Rejection: NotSupportedError: The operation is not supported

I guess bug can also be reproduced same way using <audio> element. Can we update the bug title according to this?

For your information, testing on version built from source using Gentoo net-libs/webkit-gtk-2.40.5-r410 package. I stumbled upon this 8 year old bug while building offline music player using Tauri.
Comment 9 Vladimir Porotnikov 2023-08-13 19:58:42 PDT
Sorry for second email. Tried exploring webkit source code and found some little extra info which may help:


const audio = new Audio("custom://a.mp3");
console.log(audio.error) // prints { code: 4, message: "" }
console.log(audio.currentSrc) // same as audio.src
audio.play() // NotSupportedError

Code 4 is MEDIA_ERR_SRC_NOT_SUPPORTED


Somewhere in HTMLMediaElement::loadResource some conditions do not pass and
mediaLoadingFailed(MediaPlayer::NetworkState::FormatError) is being called
which executes noneSupported which leads to error log I mentioned in previous comment.
I could not make it any further :) Hope someone will investigate this
Comment 10 Alexander Yanovskyy 2023-10-05 14:31:37 PDT
After some investigation, the error is linked with GStream. The error is an MediaPlayer::NetworkState::FormatError and is set because GST_RESOURCE_ERROR_NOT_FOUND is match when handling message in MediaPlayerPrivateGStreamer::handleMessage(GstMessage* message). 

I will try to further investigate this and maybe found a fix for this issue.
Comment 11 Alexander Yanovskyy 2023-10-05 14:37:11 PDT
(In reply to Philip Chimento from comment #0)
> When trying to play a video loaded via a custom URI scheme, the request for
> the video file somehow never makes it to the custom URI handler. I tested
> this on 2.6.2 and 2.8.3.
> 
> This is perhaps best illustrated by the following GJS script:
> https://gist.github.com/ptomato/e180132d2313d07102d2 (You have to specify
> the path in the custom:// URI to where you put the test.html file, and you
> have to supply your own video.ogv and subtitles.vtt files in the same
> directory)
> 
> You will notice that the custom URI handler finds test.html correctly (shown
> by the message on stderr), but does not load video.ogv. In the inspector,
> you can see that a request for video.ogv was fired, but it never reached the
> custom URI handler because there was no message for it on stderr. What makes
> me think this bug is only about videos, is that if you select "Spanish" from
> the captions menu, then subtitles.vtt is correctly loaded via the custom URI
> handler and prints out a corresponding message on stderr.

To whoever who wants to reproduce the bug you could just build webkit and open a video/audio by doing /path_to_webkit_build/Tools/Scripts/run-minibrowser --gtk /home/you/a_video.mp4.
Comment 12 Michael Catanzaro 2023-10-05 15:34:01 PDT
(In reply to Alexander Yanovskyy from comment #11)
> To whoever who wants to reproduce the bug you could just build webkit and
> open a video/audio by doing
> /path_to_webkit_build/Tools/Scripts/run-minibrowser --gtk
> /home/you/a_video.mp4.

Um, are you sure this is the same bug? Philip and Yu-Wei and Vladimir are all complaining that custom URI handlers are broken. But MiniBrowser does not use custom URI handlers.
Comment 13 Alexander Yanovskyy 2023-10-06 02:30:52 PDT
(In reply to Michael Catanzaro from comment #12)
> (In reply to Alexander Yanovskyy from comment #11)
> > To whoever who wants to reproduce the bug you could just build webkit and
> > open a video/audio by doing
> > /path_to_webkit_build/Tools/Scripts/run-minibrowser --gtk
> > /home/you/a_video.mp4.
> 
> Um, are you sure this is the same bug? Philip and Yu-Wei and Vladimir are
> all complaining that custom URI handlers are broken. But MiniBrowser does
> not use custom URI handlers.

Yes you are right ! I was doing it wrong but it seems that I've discovered another bug not related to this issue. Anyway, I picked https://github.com/wusyong/gtkbrowser/tree/video and tested with what I've compiled and the error is different now, it seems that now the error is about GST_CORE_ERROR_MISSING_PLUGIN and message given is apprently about no url handler for custom protocol, in the c example it use gtk custom protocol. It is weird because this error is not appearing when loading an img. It seems it is somehow closely related to gstreamer ?
Comment 14 Alexander Yanovskyy 2023-10-07 00:22:12 PDT
Filled an issue to gstreamer repo with a little bit more explanation about gstreamer issue: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3021
Comment 15 Philippe Normand 2023-10-08 03:12:46 PDT
(In reply to Yu-Wei Wu from comment #5)
> I saw the release log in 2.36.5 and thought it's fixed but looks like the
> issue still remains.
> I have tried different video formats but none is working. However, using
> file URL can still load the video.
> Here's my repo to reproduce it:
> https://github.com/wusyong/gtkbrowser/tree/video

But that's a totally different use-case than the one initially shared by Philip, isn't it?

In Philip's case the custom URI handler was responding with an HTML document, while in your case, the handler responds with a media (webm) payload.

There is not support for this in the current media player. We'd need a custom GStreamer source element that would be hooked to the LegacySchemeRegistry and would somehow get the response data and push it on a source pad.

Currently the DOMImplementation creates a MediaDocument for this case and then in the <video> element created by that document, tries to load the URI with custom scheme.

I couldn't find any evidence in the AVF (used by Apple ports) player for this use-case either. Why would you go such convoluted scenario anyway?

If Philip's use-case now works, I would close this bug.

If you want support for directly loading media content with custom URI schemes you should open a new bug report.
Comment 16 Philip Chimento 2023-10-21 12:56:19 PDT
(In reply to Philippe Normand from comment #15)
> If Philip's use-case now works, I would close this bug.

Sadly, it does not. I tested on 2.42.1.

I updated https://gist.github.com/ptomato/e180132d2313d07102d2 to adapt the reproducer case to the webkitgtk6.0 API.