iPadOS 15 / iOS 15 unable to decode VP9 stream

• Component: WebRTC
• Status: RESOLVED FIXED

[Dan Jenkins]

Using apprtc we're able to reproduce this issue

Both Safari Tech Preview and iOS 15/iPadOS 15 should be able to send and receive VP9 now.

I used https://appr.tc/?hd=true&vsc=vp9&vrc=VP9 to reproduce

I was able to reproduce the issue using Chrome Canary and Safari Tech Preview.

Expected result - you send video and receive video
Current result - on Chrome/STP you can see both sending and receiving streams, on iPadOS 15 and iOS 15 Safari you only see the sending stream.

Logs can be found over on a github gist - https://gist.github.com/danjenkins/ead6e9be8984c286e1727399a00b4766

You can see the RTCRtpReceiver says it can receive VP9 in the output from the console.

[Radar WebKit Bug Importer]

<rdar://problem/83391595>

[Alexey Proskuryakov]

Could you please confirm if this is new to iOS 15? Did this work in prior versions of iOS?

[[e-mail]]

yes, it is happening on latest iOS 15 version, can't tell about earlier versions, but vp9 support was meant to be a new feature on iOS 15 (AFAIK)

[youenn fablet]

Created attachment 439144 [details]
Patch

[Dan Jenkins]

Youenn can you confirm if its just ipads that use hardware decode and not iphones?

This is the line from the release notes for 15 that caused me to think that...

Added hardware accelerated VP9 and WebM in MSE on all iPads that support iPadOS 15.

So turns out some ipads running ipadOs 15 don't support hardware accelerated decoding of vp9? But they can encode it fine? If its just iPads that support hardware acceleration does that mean iphones are doing software based encode/decode and if thats the case would we expect older iPads to use software decode?

Just looking for some clarification seeing as I don't fully understand the patch you've proposed :) thanks!

[youenn fablet]

Created attachment 439148 [details]
Patch

[Eric Carlson]

Comment on attachment 439148 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=439148&action=review

r=me once the bots are happy

> Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp:249
> +    static bool hasGPUProcessHardwareVP9;
> +    static std::once_flag onceFlag;
> +    std::call_once(onceFlag, [] {
> +        WebProcess::singleton().ensureGPUProcessConnection().connection().sendSync(Messages::LibWebRTCCodecsProxy::HasVP9Decoder { }, Messages::LibWebRTCCodecsProxy::HasVP9Decoder::Reply(hasGPUProcessHardwareVP9), 0);
> +    });
> +    WebProcess::singleton().libWebRTCCodecs().setVP9VTBSupport(hasGPUProcessHardwareVP9);

Instead of doing this as a synchronous message, could we push the state back from the GPUConnectionToWebProcess when it opens?

[youenn fablet]

Created attachment 439149 [details]
Patch

[youenn fablet]

Created attachment 439349 [details]
Patch

[youenn fablet]

(In reply to Eric Carlson from comment #7)
> Comment on attachment 439148 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=439148&action=review
> 
> r=me once the bots are happy
> 
> > Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp:249
> > +    static bool hasGPUProcessHardwareVP9;
> > +    static std::once_flag onceFlag;
> > +    std::call_once(onceFlag, [] {
> > +        WebProcess::singleton().ensureGPUProcessConnection().connection().sendSync(Messages::LibWebRTCCodecsProxy::HasVP9Decoder { }, Messages::LibWebRTCCodecsProxy::HasVP9Decoder::Reply(hasGPUProcessHardwareVP9), 0);
> > +    });
> > +    WebProcess::singleton().libWebRTCCodecs().setVP9VTBSupport(hasGPUProcessHardwareVP9);
> 
> Instead of doing this as a synchronous message, could we push the state back
> from the GPUConnectionToWebProcess when it opens?

Good idea, updated the patch accordingly

[youenn fablet]

> So turns out some ipads running ipadOs 15 don't support hardware accelerated
> decoding of vp9? But they can encode it fine? If its just iPads that support
> hardware acceleration does that mean iphones are doing software based
> encode/decode and if thats the case would we expect older iPads to use
> software decode?

All iOS devices support VP9 encoder in WebRTC using libvpx.
Most iOS devices support VP9 decoder via VTB, but not all, in which case we rely on libvpx.

[Dan Jenkins]

makes sense, thanks for the clarification Youenn!

[EWS]

Committed r283116 (242174@main): <https://commits.webkit.org/242174@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 439349 [details].

[youenn fablet]

*** Bug 231074 has been marked as a duplicate of this bug. ***