Summary: | REGRESSION (iOS 16 Beta 7) Outgoing video is stopped and do not recover for WebRTC call if the iPhone device is connected with a Bluetooth device (AirPods) and User uses Youtube / Spotify or other app in the middle of the call | ||
---|---|---|---|
Product: | WebKit | Reporter: | Madara Freimane <madara.freimane> |
Component: | WebRTC | Assignee: | Nobody <webkit-unassigned> |
Status: | RESOLVED CONFIGURATION CHANGED | ||
Severity: | Critical | CC: | daginge, ddp.bkdn, webkit-bug-importer, youennf |
Priority: | P2 | Keywords: | InRadar |
Version: | Other | ||
Hardware: | iPhone / iPad | ||
OS: | Other | ||
See Also: | https://bugs.webkit.org/show_bug.cgi?id=243982 |
Description
Madara Freimane
2022-08-29 11:57:46 PDT
Tried reproducing this. I can kinda do it, but it eventually recovers. Tested on iPhone 12 Pro MGMP3QN/A running iOS 16 Beta 8 (20A5358a) using Airpods Pro A2084 running 4E71 (1.3.8). Test procedure: 1. Go to https://webrtc.github.io/samples/src/content/peerconnection/pc1/ and initiate the call. Observe that you can see and hear yourself. 2. Go to the home screen and navigate to Spotify 3. Start playing a random song. Observe that the capture color disappears in the clock. 4. Return to Safari. Observe that both video views are black and you can no longer hear yourself. 5. Wait 3-4 seconds. Observe that the page starts capturing again. I can also reliably reproduce the same behaviour with the YouTube app. Key seems to be starting audio playback in a native app. Expected behaviour (video mutes when Safari is not the active app) happens when I go to home screen and other apps without playing audio, and recovers as expected quickly when Safari is the active app again. Using my trust gUM debugger (https://codepen.io/daginge/pen/ExLxZzN) I see that the video track mutes as expected when going to the home screen. When the audio starts playing, the audio track mutes. Upon returning to the page, the video and audio track unmutes after 2-3 seconds. Logs: Got stream with constraints: {"audio":true,"video":{"facingMode":"user","deviceId":"default","width":1280,"height":720}} Using video device: Front Camera The video track is currently unmuted Using audio device: AirPods The audio track is currently unmuted Muted video track: true Visibilitychange event triggered: Hidden prop is true Muted audio track: true <-- This is when I start playing Spotify audio Visibilitychange event triggered: Hidden prop is false Unmuted audio track: false Unmuted video track: false However, if I continue playing audio in Spotify, neither the video or audio track unmutes. If I stop playback, they instantly recover. In conclusion: I'm able to reproduce this issue on beta 8 if I continuously play audio. It seems to be an OS mute issue, but the capture is luckily never ended, and will recover. (In reply to daginge from comment #1) > Tried reproducing this. I can kinda do it, but it eventually recovers. > Tested on iPhone 12 Pro MGMP3QN/A running iOS 16 Beta 8 (20A5358a) using > Airpods Pro A2084 running 4E71 (1.3.8). > > Test procedure: > 1. Go to https://webrtc.github.io/samples/src/content/peerconnection/pc1/ > and initiate the call. Observe that you can see and hear yourself. > 2. Go to the home screen and navigate to Spotify > 3. Start playing a random song. Observe that the capture color disappears in > the clock. > 4. Return to Safari. Observe that both video views are black and you can no > longer hear yourself. > 5. Wait 3-4 seconds. Observe that the page starts capturing again. @daginge, this is somehow expected behaviour. Is your request that restarting capture should happen more quickly? > > I can also reliably reproduce the same behaviour with the YouTube app. Key > seems to be starting audio playback in a native app. Expected behaviour > (video mutes when Safari is not the active app) happens when I go to home > screen and other apps without playing audio, and recovers as expected > quickly when Safari is the active app again. > > Using my trust gUM debugger (https://codepen.io/daginge/pen/ExLxZzN) I see > that the video track mutes as expected when going to the home screen. When > the audio starts playing, the audio track mutes. Upon returning to the page, > the video and audio track unmutes after 2-3 seconds. > > Logs: > Got stream with constraints: > {"audio":true,"video":{"facingMode":"user","deviceId":"default","width":1280, > "height":720}} > Using video device: Front Camera > The video track is currently unmuted > Using audio device: AirPods > The audio track is currently unmuted > Muted video track: true > Visibilitychange event triggered: Hidden prop is true > Muted audio track: true <-- This is when I start playing Spotify audio > Visibilitychange event triggered: Hidden prop is false > Unmuted audio track: false > Unmuted video track: false > > However, if I continue playing audio in Spotify, neither the video or audio > track unmutes. If I stop playback, they instantly recover. Can you clarify this? I would have thought that, when going back to Safari, Safari would restart capture and would interrupt Spotify. Isn't this happening? (In reply to youenn fablet from comment #2) > (In reply to daginge from comment #1) > > Tried reproducing this. I can kinda do it, but it eventually recovers. > > Tested on iPhone 12 Pro MGMP3QN/A running iOS 16 Beta 8 (20A5358a) using > > Airpods Pro A2084 running 4E71 (1.3.8). > > > > Test procedure: > > 1. Go to https://webrtc.github.io/samples/src/content/peerconnection/pc1/ > > and initiate the call. Observe that you can see and hear yourself. > > 2. Go to the home screen and navigate to Spotify > > 3. Start playing a random song. Observe that the capture color disappears in > > the clock. > > 4. Return to Safari. Observe that both video views are black and you can no > > longer hear yourself. > > 5. Wait 3-4 seconds. Observe that the page starts capturing again. > > @daginge, this is somehow expected behaviour. > Is your request that restarting capture should happen more quickly? Honestly I don't really mind, as long as it reliably restarts. It's riiight at the limit of users being confused I think, but let's see if we get any reports. I think this is such a corner case that we won't hear much. It is faster without the AirPods in though, can't explain that. > > However, if I continue playing audio in Spotify, neither the video or audio > > track unmutes. If I stop playback, they instantly recover. > > Can you clarify this? > I would have thought that, when going back to Safari, Safari would restart > capture and would interrupt Spotify. > Isn't this happening? I'll do you one better and share a video: https://www.icloud.com/photos/#05bDb-5-djR1WZFp3U53sf0Gw (a bit loud due to the music getting picked up). Strangely, with screen recording the behaviour kinda changed, it never recovered. Or rather, the capture seems to recover, but the video went white (which in this context I think means transparent). But if you ignore this fact for now, the behaviour I described is there. The video and audio track doesn't recover until I pause playback. Hope this helps! > Strangely, with screen recording the behaviour kinda changed, it never
> recovered. Or rather, the capture seems to recover, but the video went white
> (which in this context I think means transparent).
We might still have some bad interaction between autoplay and interruptions here.
It might be that calling video.play() will fix the white issue.
> => From iOS 16 Beta 7
> (iOS_16Beta7_sysdiagnose_2022.08.29_20-41-28+0300_iPhone-OS_iPhone_20A5356a.
> tar.gz):
> https://drive.google.com/file/d/15_tTigk6tJUFGzhW10DPK6_Vm5pnVNOA/
> view?usp=sharing
On this sysdiagnose, I do not find anything wrong.
The encoded frame counter is increasing from the beginning of the call 15:20 to the end 15:29. Ditto for audio.
I do not see also where the audio get interrupted. Or maybe I did not look at the right timestamp?
> I'll do you one better and share a video:
> https://www.icloud.com/photos/#05bDb-5-djR1WZFp3U53sf0Gw (a bit loud due to
> the music getting picked up).
It does not seem bad that Spotify is not interrupted by moving Safari to foreground.
It seems good that user does an action so that it happens. In that case it would be unmuting capture using the red camera icon in the menu bar.
I'll have a try with Spotify to verify this is working as expected.
Hi Youenn! I added a new sysdiagnose file: https://drive.google.com/file/d/1Bsq3gIAJOJ7NEK5aBSYBhh8Z9-v5OhXE/view?usp=sharing Bug reproduced on 2022.09.07_10-48-03 Hi, Youenn! After the investigation we see more than one issue: 1. Bluetooth device issue: Use case: Preconditions: iPhone device is connected with AirPods or other Bluetooth device Permissions for camera and microphone are set to "Ask" in browser settings User is in active WebRTC video call Steps: 1. iOS 16 User navigates to Youtube native app or to another native music app and listens music for ~3min 2. iOS 16 User navigates back to the Safari browser 3. iOS 16 User accepts required permissions for audio and video => AirPods option disappears and Microphone device in settings is switching from iPhone mic to "No mic device found". Added sysdiagnose file: https://drive.google.com/file/d/14i4QbOQvr8K3ItZ79S3jaMcP3QUt5X34/view?usp=sharing 2. H264 encoder/decoder issue: Use case: Preconditions: iPhone device is connected with AirPods or other Bluetooth device Permissions for camera and microphone are set to "Ask" in browser settings The WebRTC call is an active, video enabled for other Users in the call Steps: 1. iOS 16 User joins a call 1. iOS 16 User navigates to Youtube native app or to another native music app and listens music for ~3min 2. iOS 16 User navigates back to the Safari browser 3. iOS 16 User accepts required permissions for audio and video => video becomes broken and sometimes also incoming audio disappears + observed that call becomes broken immediately sometimes if iOS User joins an active WebRTC call while cameras enabled for other Users => we don't have incoming video for iOS User in that case. Added sysdiagnose file with this use case: https://drive.google.com/file/d/1oYuYD_mwaIATrpTBcTZjsWWPuFxEwOHo/view?usp=sharing In my opinion there is a higher repro if iPhone device is connected with AirPods, but I was able reproduce defect also with other Bluetooth devices (for example, with Huawei FreeBuds 4i), but with a lower reproducibility. All sysdiagnostics files were generated immediately the defects were reproduced. Tested device: * iPhone 11 Pro Max | iOS 16, build: 20A362 Hi, Youenn! Still can reproduce defects. Added new sysdiagnostics files, recorded logs using iOS 16.1 Beta 2 (Build: 20B5050f) Here using 2nd use case I reproduced this issue: H264 encoder/decoder issue When came back to Safari browser => AirPods was still connected with a device, no audio issue, only outgoing video was broken (Time: 16:46, EET): https://drive.google.com/file/d/1M48-cn5JbkLW57eHzsj_K1ymfZh9gdxW/view?usp=sharing Here using 1st use case use case I reproduced this issue: Bluetooth device issue When came back to Safari browser => AirPods option was not available anymore, didn’t have incoming audio + outgoing video was broken (Time: 17:43, EET): https://drive.google.com/file/d/1rgXXZ6BCZEsLOC7VqVidNkqeb5YabWnf/view?usp=sharing Tested device: * iPhone 11 Pro Max | iOS 16.1 Beta 2, Build: 20B5050f Hi, Youenn! Bug is still reproducible. Tested device: iPhone 11 Pro Max, iOS 16.2 Beta, Build: 20C5032e. Added new sysdiagnostics file (bug reproduced at 11:05, 07.11.2022 (EET time zone): https://drive.google.com/file/d/1Iek-6-IjsruI4UXepEDRgjfWJWyPK9oP/view?usp=share_link I’m not able to reproduce described issue in ticket with iOS 16.2 Beta 2, build: 20C5043e, iPhone 11 Pro Max. @Youenn, should we see the fix with this version? Bug linked with https://bugs.webkit.org/show_bug.cgi?id=243982 Bug is not reproducible anymore. Tested OK on: * iPhone 11 Pro Max | iOS 16.2, build: 20C65 * iPhone 12 Pro Max | iOS 16.2, build: 20C65 * iPhone 14 | iOS 16.2, build: 20C65 Thanks! |