Problem: In a WebRTC call, the local SDP from Safari claims supporting video codec H264 with profile-level-id 42e01f which is the Base Profile. The remote SDP also supports video codec H264 with profile-level-id 42c01f which is also the Base Profile. In this case, both sides should agree on H264 Base Profile. However, we observed the Safari browser is sending High Profile stream instead. Expected: H264 Base Profile stream is important for many video systems. Please fix this issue by sending Base Profile stream according to the negotiated profile-level-id. Or please let us know how to configure Safari browser to send Base Profile stream only.
<rdar://problem/48453085>
Hello, Do you need any further information? Did you successfully reproduce this issue? When should I anticipate a fix on this issue? Please let me know if there is any progress. Thank you. Thanks, Long Cheng
Hello, Do you need any further information? Thanks, Long Cheng
Hi Long Chen, Thanks for the report and sorry for the delay. I think we have sufficient information now. Please feel free to share additional information here or privately like repro cases or services/devices that are affected.
Thanks for your reply. May I ask for the status of this bug? Is it in progress of a fix? Thanks, Long Cheng
This is under investigation. Any information that may help us prioritizing this would be useful.
Hello Youenn, Thanks for your reply. Please investigate the issue and keep us updated. Appreciate your help. We are providing video conference services here in Compunetix. One major job we do is to mix all media streams from a variety of devices (including legacy video room terminals). We are excited to see a great progress of WebRTC implementation in WebKit in the recent two years. There are many customers prefer to use Safari on iOS and Mac. However, due to this issue, these customers have to use Chrome instead. Here is a test conference you may use to reproduce the issue as well as verify the fix. https://evergreen.compunetix.com/cx-full/conference/join?autoJoinPasscode=35678554 Thanks, Long Cheng
Safari now supports VP8, H264 base profile and H264 high profile. The default codec is H264 high profile, which might be your issue here. Selection of the base profile can be forced by munging SDP to remove VP8 and H264 high profile. Or it should be selected if the other party only supports H264 base profile. I put up https://youennf.github.io/webrtc-tests/src/content/peerconnection/pc-h264-baseline/ which shows that baseline can be forced by munging SDP. Can you verify on your side whether munging SDP would fix your issue? I am marking this issue as Invalid for now. Please reopen the bug if needed.
Created attachment 367321 [details] SDP Offer from Safari 12.1
Created attachment 367322 [details] SDP Answer from our product
Comment from our team member: Hello Youenn, We are currently already doing what you suggest. If you examine the offer and answer images we have attached you will see that each side supports H.264 base profile and neither supports H.264 high profile. What we have observed is that even with this SDP exchange we are receiving an H.264 RTP stream that contains a profile-level-id set to high profile in the H.264 header and features only available in high profile are present in the video stream. This should not be allowed based on the SDP exchange. Can you please examine this issue from this perspective and let us know if you need anything from us to expedite this? This was tested on a brand new and fully updated Mac Book Pro running Mojave and the latest version of Safari.
Hello Youenn, Have you successfully verified the H.264 stream headers and features? We believe with the SDP forcing base profile, the H.264 RTP stream still contains a profile-level-id set to high profile in the H.264 header and features only available in high profile are present in the video stream. Thanks, Long Cheng
When using VTB, it seems to work fine with the hardware H264 encoder on my MacBookPro. This does not work when not using the hardware H264 encoder. When using VCP, this does not work properly, no matter whether hardware or not. kVTCompressionPropertyKey_ProfileLevel does not seem to be respected.
Created attachment 371293 [details] Patch
Comment on attachment 371293 [details] Patch Attachment 371293 [details] did not pass win-ews (win): Output: https://webkit-queues.webkit.org/results/12374570 New failing tests: fast/block/float/float-with-anonymous-previous-sibling.html
Created attachment 371302 [details] Archive of layout-test-results from ews215 for win-future The attached test failures were seen while running run-webkit-tests on the win-ews. Bot: ews215 Port: win-future Platform: CYGWIN_NT-10.0-17763-3.0.5-338.x86_64-x86_64-64bit
Created attachment 371305 [details] Patch
Created attachment 371306 [details] Patch
Created attachment 371307 [details] Patch
Created attachment 371345 [details] Patch
Comment on attachment 371345 [details] Patch Attachment 371345 [details] did not pass mac-wk2-ews (mac-wk2): Output: https://webkit-queues.webkit.org/results/12378704 Number of test failures exceeded the failure limit.
Created attachment 371358 [details] Archive of layout-test-results from ews107 for mac-highsierra-wk2 The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews. Bot: ews107 Port: mac-highsierra-wk2 Platform: Mac OS X 10.13.6
Created attachment 371365 [details] Patch
Created attachment 371408 [details] Patch
Comment on attachment 371408 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=371408&action=review > Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/VideoProcessingSoftLink.h:37 > #define ENABLE_VCP_ENCODER 0 Nit: not new, but there is an extra space before the "&&" in the line above this > Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/VideoProcessingSoftLink.h:39 > #elif (defined(TARGET_OS_IPHONE) && TARGET_OS_IPHONE) Nit: Ditto. > Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/VideoProcessingSoftLink.h:44 > +//#define ENABLE_VCP_ENCODER __MAC_OS_X_VERSION_MIN_REQUIRED >= 101300 > +//#define ENABLE_VCP_VTB_ENCODER __MAC_OS_X_VERSION_MIN_REQUIRED >= 101500 Oops! > Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.mm:321 > + VTCompressionSessionRef _compressionSession; Nit: something like "_vtCompressionSession" would be clearer. > Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.mm:507 > + status = 1; Is it worth logging an error here to make it clear what failed? > Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.mm:876 > + RTC_LOG(LS_ERROR) << "VTSessionSetProperty failed to set: " << kVTCompressionPropertyKey_DataRateLimits Nit: kVTCompressionPropertyKey_DataRateLimits is a constant, why not hard code it in the string? > Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/helpers.cc:38 > +void SetVTSessionProperty(VTCompressionSessionRef session, VCPCompressionSessionRef vcpSession, Nit: something like "vtSession" would be clearer. > Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/helpers.cc:48 > + if (session) > + status = VTSessionSetProperty(session, key, cfNum); > +#if ENABLE_VCP_ENCODER > + else > + status = webrtc::VCPCompressionSessionSetProperty(vcpSession, key, cfNum); If we are going to assume vcpSession is valid when session is NULL, maybe ASSERT(session || vcpSession)? > Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/helpers.cc:70 > + if (session) > + status = VTSessionSetProperty(session, key, cfNum); > +#if ENABLE_VCP_ENCODER > + else > + status = webrtc::VCPCompressionSessionSetProperty(vcpSession, key, cfNum); Ditto. > Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/helpers.cc:88 > + if (session) > + status = VTSessionSetProperty(session, key, cf_bool); > +#if ENABLE_VCP_ENCODER > + else > + status = webrtc::VCPCompressionSessionSetProperty(vcpSession, key, cf_bool); Ditto. > Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/helpers.cc:106 > + if (session) > + status = VTSessionSetProperty(session, key, value); > +#if ENABLE_VCP_ENCODER > + else > + status = webrtc::VCPCompressionSessionSetProperty(vcpSession, key, value); Ditto
Thanks for the review. (In reply to Eric Carlson from comment #25) > Comment on attachment 371408 [details] > Patch > > View in context: > https://bugs.webkit.org/attachment.cgi?id=371408&action=review > > > Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/VideoProcessingSoftLink.h:37 > > #define ENABLE_VCP_ENCODER 0 > > Nit: not new, but there is an extra space before the "&&" in the line above > this OK > > Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/VideoProcessingSoftLink.h:39 > > #elif (defined(TARGET_OS_IPHONE) && TARGET_OS_IPHONE) > > Nit: Ditto. OK > > Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/VideoProcessingSoftLink.h:44 > > +//#define ENABLE_VCP_ENCODER __MAC_OS_X_VERSION_MIN_REQUIRED >= 101300 > > +//#define ENABLE_VCP_VTB_ENCODER __MAC_OS_X_VERSION_MIN_REQUIRED >= 101500 > > Oops! > Thanks! Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.mm:321 > > + VTCompressionSessionRef _compressionSession; > > Nit: something like "_vtCompressionSession" would be clearer. OK > > Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.mm:507 > > + status = 1; > > Is it worth logging an error here to make it clear what failed? It is logged below. > > Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoEncoderH264.mm:876 > > + RTC_LOG(LS_ERROR) << "VTSessionSetProperty failed to set: " << kVTCompressionPropertyKey_DataRateLimits > > Nit: kVTCompressionPropertyKey_DataRateLimits is a constant, why not hard > code it in the string? I prefer keeping it this way to limit the changes compared to upstream. Admittedly, we have already branched quite a bit... > > Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/helpers.cc:38 > > +void SetVTSessionProperty(VTCompressionSessionRef session, VCPCompressionSessionRef vcpSession, > > Nit: something like "vtSession" would be clearer. OK
Created attachment 371533 [details] Patch for landing
Comment on attachment 371533 [details] Patch for landing Clearing flags on attachment: 371533 Committed r246263: <https://trac.webkit.org/changeset/246263>
All reviewed patches have been landed. Closing bug.
Created attachment 375695 [details] global end-to-end sip trace How to read the trace: (WSC-SE-01) is the webrtc gateway. The SDP contained in the first INVITE on the left comes directly from the Iphone 6s. 10.132.4.50 is the Cisco Jabber SIP client SBC-WSC-MILANO is the Oracle SIP SBC Cucm-CC is the Cisco Call Manager, SIP PBX. Jabber is registered here. WSC-ME is the webrtc gateway Media Engine...media flow through here. Calling: uncle@uncle.it // webrtc caller Called: 10090 // Jabber extension number
Hello, even though this bug is FIXED, I am experiencing the exact same problem whenever I try to set up a webrtc call from latest ios/Safari on iPhone 6s. Scenario: Iphone 6s --- Webrtc gateway --- Oracle SBC --- Cisco Call Manager --- Cisco Jabber Problem: Cisco Jabber does not show remote video (audio is fine). Same scenario with Android/Chrome works fine. It works also on Iphone/Safari with 12.1.4 release. In a nutshell, safari ios is sending to Jabber H264 high profile even though the offer/answer negotiation agrees on h264 base profile on both parts. Reasoning: in the attached message-flow.html: SDP INVITE from ios/Safari contains #2 h264 m-lines (high and base profile): a=rtpmap:96 H264/90000 a=fmtp:96 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=640c1f <--- high-profile a=rtpmap:98 H264/90000 a=fmtp:98 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f <--- base profile final 200 OK sent back to ios/Safari contains just one h264 m line: a=rtpmap:126 H264/90000 a=fmtp:126 profile-level-id=42E01F;packetization-mode=1;max-fs=3601;max-rcmd-nalu-size=32000;level-asymmetry-allowed=1 This is ACKed by Safari so they both agree on using h264 base profile. But if you open the attached ios-call-ok-old-iphone.pcapng, isolate the h264 track with this filter: (ip.src==10.133.8.53 && udp.srcport==20530 && ip.dst==10.132.4.59 && udp.dstport==32220 && rtp.ssrc==0x29b6f70a) and look at the very first h264 packet, you'll see: 0110 0100 = Profile_idc: High profile (100)
Created attachment 375696 [details] complete wireshark trace taken on the Jabber laptop it shows the rtp and sip dialog between my sip client and cucm/sbc
Forgot to add iphone user agent string: Mozilla/5.0 (iPhone; CPU iPhone OS 12_4 like Mac OS X) AppleWebKit/605.1.15 /KHTML, like Gecko) Version/12.1.2 Mobile/15E148 Safari/604.1
Hi, any comment on this? Do you need additional information? Thank you
> any comment on this? Do you need additional information? Which OS are you testing on? Have you tried on the latest iOS beta?
> Which OS are you testing on? Mozilla/5.0 (iPhone; CPU iPhone OS 12_4 like Mac OS X) AppleWebKit/605.1.15 /KHTML, like Gecko) Version/12.1.2 Mobile/15E148 Safari/604.1 > Have you tried on the latest iOS beta? Not yet. Doing that asap.
Hi Youenn, with ios 13 beta 7 it is finally working fine. I can see h264 base profile sent by safari! User Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 13_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0 Mobile/15E148 Safari/604.1 Thanks
*** Bug 211862 has been marked as a duplicate of this bug. ***
Hello Youenn, We are seeing exact same problem on iPad Pro 11-inch 2nd generation, OS 14.7, Can you help me check on it, please? user agent says: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.2 Safari/605.1.15 Thanks, Long Cheng
Hello, I am experiencing the exact same problem on macos10.15.7 safari. I think this problem is related to the system version, not the browser version, I reproduced it on two Safari devices with different versions. their system versions are 10.15.7.
Created attachment 459155 [details] Specified profile-level-id, but no working. this is safari webrtc log.
Call was negotiated with H264 Constrained Baseline Profile 42e01f but encoded in Baseline Profile
And Baseline cannot decoded by Constrained Baseline on M1 mac's safari. chrome is work fine.
Has any more work been done on this?