Bug 147822 - [GTK][GStreamer] Install missing media plugins API is totally broken, test /webkit2/WebKitWebView/install-missing-plugins-permission-request times out in the bots
Summary: [GTK][GStreamer] Install missing media plugins API is totally broken, test /w...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: Gtk
Depends on:
Blocks:
 
Reported: 2015-08-09 23:38 PDT by Carlos Garcia Campos
Modified: 2018-10-13 08:25 PDT (History)
7 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Carlos Garcia Campos 2015-08-09 23:38:21 PDT
My guess is that there's no installer installed in the bots and gst_install_plugins_supported() returns false, so we don't even get a permission request because no installer will be launched.
Comment 1 Carlos Garcia Campos 2015-08-09 23:41:50 PDT
Skipped for now in r188205.
Comment 2 Michael Catanzaro 2015-08-11 01:58:16 PDT
Carlos Lopez, I think we need gstreamer1.0-packagekit on the bots?
Comment 3 Michael Catanzaro 2016-11-06 20:24:39 PST
So I tried hooking up missing media plugin permission requests in Epiphany the other day. I tested various websites but never saw a request. I also the exact same page that is used by this test:

<html><body><video src=\"test.mp4\" autoplay></video></body></html>

with some test.mp4 file in the same directory. But WebKit never creates a permission request. It doesn't work in MiniBrowser either. I think it's broken, and we never noticed because this test has always been skipped. Missing codec notifications are working just fine for me in e.g. Totem.
Comment 4 Michael Catanzaro 2016-11-06 20:26:09 PST
(In reply to comment #3)
> the other day.

Er, a strange expression to use for "just now"
Comment 5 Philippe Normand 2016-11-06 22:54:41 PST
Because we now white-list supported codecs based on what's installed, because that's what you wanted :)
Comment 6 Philippe Normand 2016-11-06 23:30:41 PST
(In reply to comment #5)
> Because we now white-list supported codecs based on what's installed,
> because that's what you wanted :)

See bug 135972
Comment 7 Michael Catanzaro 2016-11-07 05:55:21 PST
So how is this supposed to work?
Comment 8 Philippe Normand 2016-11-07 06:03:09 PST
I'm afraid you won't trick the gst codec installer unless our player has video/mp4 in the mime-type cache it builds.
Comment 9 Michael Catanzaro 2016-11-07 06:32:58 PST
(In reply to comment #8)
> I'm afraid you won't trick the gst codec installer unless our player has
> video/mp4 in the mime-type cache it builds.

So it seems like our options are:

 * Deprecate this API, because it can never work
 * Decide that YouTube is broken (is it?) and convince YouTube engineers to fix it so that we can bring back the codec installer

Is this right? Is there another option?

I don't want to break YouTube under any circumstances.
Comment 10 Philippe Normand 2016-11-07 06:46:30 PST
I'm not sure what to do...

On the end-user POV I think I'd like this to work:

1. user opens youtube video in Epiphany
2. a prompt appears asking permission to install H264 codec
3.a user accepts, codec is installed, playback starts
3.b user refuses, the decision about refusing H264 is stored somewhere on-disk, the webm version of the video starts to play
Comment 11 Michael Catanzaro 2016-11-07 08:36:48 PST
(In reply to comment #10)
> I'm not sure what to do...
> 
> On the end-user POV I think I'd like this to work:

I agree on that desired behavior, but it seems like we don't know how to make it possible.
Comment 12 Philippe Normand 2016-11-07 09:30:33 PST
Can you check that adding video/mp4 in mime-type set at least triggers the codec installer?
Comment 13 Michael Catanzaro 2016-11-07 12:57:59 PST
(In reply to comment #12)
> Can you check that adding video/mp4 in mime-type set at least triggers the
> codec installer?

Unfortunately that does not trigger the codec installer (tested Vimeo, the test page in comment #3, Epiphany and MiniBrowser).html><body><video src=\"test.mp4\" autoplay></video></body></html>
Comment 14 Philippe Normand 2016-11-08 01:16:30 PST
Seems like an issue with our jhbuild (as usual).

Try this:

export GST_INSTALL_PLUGINS_HELPER=$PWD/gst-codec-install-helper.sh

and in gst-codec-install-helper.sh:

#!/bin/sh

echo "Installation of gst plugin requested..."
echo $@

exit 0


(make it executable of course :))

Then try to load a mp4 video after making sure video/mp4 is in the mime-type set and you'll see MiniBrowser showing the permission request.
Comment 15 Philippe Normand 2016-11-21 00:21:08 PST
I don't think the API is broken.
We should (partly at least) revert the changes made to the player. If video/mp4 isn't in the supported mime-types list there is no chance the codec installer fires up when a h264 video is being loaded.

Then some investigation is needed to know what to do when the user refuses to install a codec. In that case the player should likely fire a load error so that the next <source> element is probed.
Comment 16 Michael Catanzaro 2016-11-21 07:02:35 PST
(In reply to comment #15)
> We should (partly at least) revert the changes made to the player. If
> video/mp4 isn't in the supported mime-types list there is no chance the
> codec installer fires up when a h264 video is being loaded.

Do you know how to do it without breaking YouTube? YouTube is more important than this API.

Ideally the behavior would be something along the lines of:

 * Try H.264, see it's not supported by the client
 * Fallback to WebM if supported by the server
 * Run codec installer only server does not have WebM available
Comment 17 Philippe Normand 2016-11-21 08:31:07 PST
(In reply to comment #16)
> (In reply to comment #15)
> > We should (partly at least) revert the changes made to the player. If
> > video/mp4 isn't in the supported mime-types list there is no chance the
> > codec installer fires up when a h264 video is being loaded.
> 
> Do you know how to do it without breaking YouTube? YouTube is more important
> than this API.
> 
> Ideally the behavior would be something along the lines of:
> 
>  * Try H.264, see it's not supported by the client
>  * Fallback to WebM if supported by the server
>  * Run codec installer only server does not have WebM available

I don't see how that would work. What's wrong with the proposal in comment 10 ?
Comment 18 Michael Catanzaro 2016-11-21 10:02:38 PST
(In reply to comment #17)
> I don't see how that would work. What's wrong with the proposal in comment
> 10 ?

Hm, I didn't read it properly the first time, sorry. The problem is we should only show "additional media codecs are required" if they are really actually required. If a WebM version of the video is available, we need to try playing it *before* giving up and saying it cannot be played with currently-available codecs, right? I think getting that right so we have a seamless user experience on YouTube is more important than having support for installing missing codecs, though of course I'd like to be able to do both.
Comment 19 Michael Catanzaro 2017-08-27 10:55:03 PDT
Hey Charlie, do you think our install missing media codecs API is fixable? If not, we will have to deprecate it. That would be a shame as this is an important feature that is not currently functional, but we really can't have missing codec prompts appearing for regular YouTube videos.
Comment 20 Ms2ger (he/him; ⌚ UTC+1/+2) 2017-09-04 00:33:45 PDT
I started looking at this, but it seems like a default build of webkit has a gstreamer with mp4 support. Is there a way to turn that off or another format that would allow me to reproduce this bug?
Comment 21 Philippe Normand 2017-09-04 03:04:56 PDT
(In reply to Ms2ger from comment #20)
> I started looking at this, but it seems like a default build of webkit has a
> gstreamer with mp4 support. Is there a way to turn that off or another
> format that would allow me to reproduce this bug?

You can temporarily move the plugin file away for instance.
Comment 22 Michael Catanzaro 2018-02-09 07:02:02 PST
So, probably now would be a good time to decide how to handle this:

(In reply to Michael Catanzaro from comment #19)
> do you think our install missing media codecs API is fixable?
> If not, we will have to deprecate it. That would be a shame as this is an
> important feature that is not currently functional, but we really can't have
> missing codec prompts appearing for regular YouTube videos.

(In reply to Michael Catanzaro from comment #18)
> The problem is we
> should only show "additional media codecs are required" if they are really
> actually required. If a WebM version of the video is available, we need to
> try playing it *before* giving up and saying it cannot be played with
> currently-available codecs, right? I think getting that right so we have a
> seamless user experience on YouTube is more important than having support
> for installing missing codecs, though of course I'd like to be able to do
> both.
Comment 23 Philippe Normand 2018-02-09 07:13:15 PST
And what if the user had no VPX codec installed? 

You can't work around the codec installer. I already suggested a solution in comment 10. Please read it again.
Comment 24 Michael Catanzaro 2018-02-09 08:13:35 PST
(In reply to Philippe Normand from comment #23)
> And what if the user had no VPX codec installed? 

The desired behavior would be to run the missing codec installer if and only if no suitable codecs to play the video are installed.

> You can't work around the codec installer. I already suggested a solution in
> comment 10. Please read it again.

If that's the best we can do, then OK. It'll be fine so long as the decision about refusing H264 is stored somewhere on disk, so the prompt only occurs once per missing codec, and not again and again. Where would you store it, and will that be the responsibility of WebKit or Epiphany?
Comment 25 Philippe Normand 2018-02-09 09:39:20 PST
(In reply to Michael Catanzaro from comment #24)
> (In reply to Philippe Normand from comment #23)
> > And what if the user had no VPX codec installed? 
> 
> The desired behavior would be to run the missing codec installer if and only
> if no suitable codecs to play the video are installed.
> 
> > You can't work around the codec installer. I already suggested a solution in
> > comment 10. Please read it again.
> 
> If that's the best we can do, then OK. It'll be fine so long as the decision
> about refusing H264 is stored somewhere on disk, so the prompt only occurs
> once per missing codec, and not again and again. Where would you store it,
> and will that be the responsibility of WebKit or Epiphany?

WebKit could notify the app with a signal that the required codec installation was denied and Epiphany could store the result in a cache file perhaps.
Comment 26 Michael Catanzaro 2018-02-09 10:00:34 PST
(In reply to Philippe Normand from comment #25)
> WebKit could notify the app with a signal that the required codec
> installation was denied and Epiphany could store the result in a cache file
> perhaps.

What would the WebKit API look like?

Epiphany already has a permissions request cache (EphyPermissionsManager), but it's keyed on the website's security origin (protocol, host, and port). This storage would need to be keyed on the codec instead, so we would not be able to use EphyPermissionsManager for this.
Comment 27 Michael Catanzaro 2018-02-09 10:01:14 PST
Note I have WIP code for this at https://git.gnome.org/browse/epiphany/log/?h=wip/missing-codecs