[GStreamer][MSE] AV1 support

• Component: Platform
• Status: RESOLVED FIXED

[Philippe Normand]

Vimeo now uses AV1. Youtube as well, if you enable a specific setting in your account.

[Philippe Normand]

Created attachment 390366 [details]
Patch

[Philippe Normand]

This most likely will break layout tests because the only AV1 decoder currently in jhbuild is av1dec.

When the flatpak SDK lands I will try to add the gst-plugins-rs inside. For the time being I think this patch shouldn't land, I just had it from some months ago in a local branch and thought it would be good to upload here.

[Xabier Rodríguez Calvar]

Comment on attachment 390366 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=390366&action=review

You can have your patch sitting here as long as you want :) And it's r+ as well.

> Source/WebCore/platform/graphics/gstreamer/GStreamerRegistryScanner.cpp:87
> +            GST_DEBUG("All elements for caps %" GST_PTR_FORMAT " are blacklisted", caps.get());

Maybe warning?

> Source/WebCore/platform/graphics/gstreamer/GStreamerRegistryScanner.cpp:211
> +    av1DecodersBlacklist.append(String("av1dec"));

I think you want to append unchecked here.

[Charlie Turner]

Comment on attachment 390366 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=390366&action=review

>> Source/WebCore/platform/graphics/gstreamer/GStreamerRegistryScanner.cpp:211
>> +    av1DecodersBlacklist.append(String("av1dec"));
> 
> I think you want to append unchecked here.

Or even Vector<String> av1DecodersBlacklist { "av1dec"_s }; This will do the reservation & *unchecked* append for you.

[Philippe Normand]

👍

[Philippe Normand]

media/media-can-play-av1.html fails indeed. I can't add gst-plugins-rs to the SDK until its next stable release which will ship the dav1d-based decoder.

[Michael Catanzaro]

So this has become more important because YouTube is now sending AV1 videos for a small minority (maybe 5%?) of videos. Such videos simply do not play. We don't know why it's trying to use AV1 when not supported. The vast majority of videos use VP8. Not all videos support VP8, but these videos previously fell back to H.264 rather than AV1. (Fallback to H.264 used to fail because it was broken in Tech Preview for a long time, but it should be working now.)

Example video: https://www.youtube.com/watch?v=N_RWY04aT1k

[Michael Catanzaro]

(In reply to Michael Catanzaro from comment #7)
> So this has become more important because YouTube is now sending AV1 videos
> for a small minority (maybe 5%?) of videos. Such videos simply do not play.
> We don't know why it's trying to use AV1 when not supported. The vast
> majority of videos use VP8. Not all videos support VP8, but these videos
> previously fell back to H.264 rather than AV1. (Fallback to H.264 used to
> fail because it was broken in Tech Preview for a long time, but it should be
> working now.)
> 
> Example video: https://www.youtube.com/watch?v=N_RWY04aT1k

Apologies for the noise. Phil says this is AVC1, and that is different from AV1. Ignore.

[Philippe Normand]

(In reply to Philippe Normand from comment #6)
> media/media-can-play-av1.html fails indeed. I can't add gst-plugins-rs to
> the SDK until its next stable release which will ship the dav1d-based
> decoder.

I will land this patch and mark this test as failing for now. Would be sad to miss 2.30 because of gst 1.18 delays.

[Philippe Normand]

Committed r264162: <https://trac.webkit.org/changeset/264162>

[Radar WebKit Bug Importer]

<rdar://problem/65263697>