Bug 207547 - [GStreamer][MSE] AV1 support
Summary: [GStreamer][MSE] AV1 support
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Platform (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Philippe Normand
URL:
Keywords: InRadar
Depends on: 212964
Blocks:
  Show dependency treegraph
 
Reported: 2020-02-11 06:48 PST by Philippe Normand
Modified: 2020-07-09 02:26 PDT (History)
11 users (show)

See Also:


Attachments
Patch (6.13 KB, patch)
2020-02-11 06:54 PST, Philippe Normand
calvaris: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Philippe Normand 2020-02-11 06:48:07 PST
Vimeo now uses AV1. Youtube as well, if you enable a specific setting in your account.
Comment 1 Philippe Normand 2020-02-11 06:54:30 PST
Created attachment 390366 [details]
Patch
Comment 2 Philippe Normand 2020-02-11 07:19:42 PST
This most likely will break layout tests because the only AV1 decoder currently in jhbuild is av1dec.

When the flatpak SDK lands I will try to add the gst-plugins-rs inside. For the time being I think this patch shouldn't land, I just had it from some months ago in a local branch and thought it would be good to upload here.
Comment 3 Xabier Rodríguez Calvar 2020-02-11 08:05:46 PST
Comment on attachment 390366 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=390366&action=review

You can have your patch sitting here as long as you want :) And it's r+ as well.

> Source/WebCore/platform/graphics/gstreamer/GStreamerRegistryScanner.cpp:87
> +            GST_DEBUG("All elements for caps %" GST_PTR_FORMAT " are blacklisted", caps.get());

Maybe warning?

> Source/WebCore/platform/graphics/gstreamer/GStreamerRegistryScanner.cpp:211
> +    av1DecodersBlacklist.append(String("av1dec"));

I think you want to append unchecked here.
Comment 4 Charlie Turner 2020-02-11 16:55:48 PST
Comment on attachment 390366 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=390366&action=review

>> Source/WebCore/platform/graphics/gstreamer/GStreamerRegistryScanner.cpp:211
>> +    av1DecodersBlacklist.append(String("av1dec"));
> 
> I think you want to append unchecked here.

Or even Vector<String> av1DecodersBlacklist { "av1dec"_s }; This will do the reservation & *unchecked* append for you.
Comment 5 Philippe Normand 2020-02-12 06:34:03 PST
👍
Comment 6 Philippe Normand 2020-03-25 07:46:02 PDT
media/media-can-play-av1.html fails indeed. I can't add gst-plugins-rs to the SDK until its next stable release which will ship the dav1d-based decoder.
Comment 7 Michael Catanzaro 2020-06-11 08:11:41 PDT
So this has become more important because YouTube is now sending AV1 videos for a small minority (maybe 5%?) of videos. Such videos simply do not play. We don't know why it's trying to use AV1 when not supported. The vast majority of videos use VP8. Not all videos support VP8, but these videos previously fell back to H.264 rather than AV1. (Fallback to H.264 used to fail because it was broken in Tech Preview for a long time, but it should be working now.)

Example video: https://www.youtube.com/watch?v=N_RWY04aT1k
Comment 8 Michael Catanzaro 2020-06-11 08:29:00 PDT
(In reply to Michael Catanzaro from comment #7)
> So this has become more important because YouTube is now sending AV1 videos
> for a small minority (maybe 5%?) of videos. Such videos simply do not play.
> We don't know why it's trying to use AV1 when not supported. The vast
> majority of videos use VP8. Not all videos support VP8, but these videos
> previously fell back to H.264 rather than AV1. (Fallback to H.264 used to
> fail because it was broken in Tech Preview for a long time, but it should be
> working now.)
> 
> Example video: https://www.youtube.com/watch?v=N_RWY04aT1k

Apologies for the noise. Phil says this is AVC1, and that is different from AV1. Ignore.
Comment 9 Philippe Normand 2020-07-09 01:56:36 PDT
(In reply to Philippe Normand from comment #6)
> media/media-can-play-av1.html fails indeed. I can't add gst-plugins-rs to
> the SDK until its next stable release which will ship the dav1d-based
> decoder.

I will land this patch and mark this test as failing for now. Would be sad to miss 2.30 because of gst 1.18 delays.
Comment 10 Philippe Normand 2020-07-09 02:25:06 PDT
Committed r264162: <https://trac.webkit.org/changeset/264162>
Comment 11 Radar WebKit Bug Importer 2020-07-09 02:26:22 PDT
<rdar://problem/65263697>