Summary: REGRESSION (iOS 16 Beta 7) Outgoing video is stopped and do not recover for WebRTC call if the iPhone device is connected with a Bluetooth device (AirPods) and User uses Youtube / Spotify or other app in the middle of the call Tested devices: Bug is reproducible on: - iPhone 11 Pro Max | iOS 16 Beta 7 (20A5356a) Bug is not reproducible on: - iPhone 11 | iOS 15.6 - iPhone 11 | iOS 15.6.1 Use case: Preconditions: iPhone device is connected with AirPods or other Bluetooth device User is in active WebRTC video call Permissions for camera and microphone are set to "Ask" in browser settings Steps: 1. iOS 16 User navigates to Youtube native app or to another native music app and listens music for ~3min 2. iOS 16 User navigates back to the Safari browser 3. iOS 16 User accepts required permissions for audio and video Actual result: Outgoing video is stopped and do not recover for WebRTC call if the iPhone device is connected with a Bluetooth device (AirPods) and User uses Youtube / Spotify or other app in the middle of the call - Sometimes also audio is impacted and User does not have incoming audio anymore after the navigation back to the call Expected result: Incoming and outgoing audio, video are available and recovers for WebRTC call if iPhone device connected with Bluetooth device and the call is interrupted by Youtube, Siri, PSTN, Facetime call or other service Reproducibility: 100% Additional information: -The bug is reproducible only in a case if we have pop-ups about the permissions requests when we return to Safari browser. -With a low reproducibility, it is possible to reproduce a bug if we enable Siri in the middle of the call, receive PSTN / FaceTime calls (and talk more than 2mins) + if we have pop-ups which asks again approve permissions when we return to the browser. To be able compare behaviours with iOS 15.6 and iOS 16 Beta 7, added two sysdiagnose files: => From iOS 16 Beta 7 (iOS_16Beta7_sysdiagnose_2022.08.29_20-41-28+0300_iPhone-OS_iPhone_20A5356a.tar.gz): https://drive.google.com/file/d/15_tTigk6tJUFGzhW10DPK6_Vm5pnVNOA/view?usp=sharing => From iOS 15.6 (iOS_15.6_sysdiagnose_2022.08.29_21-32-11+0300_iPhone-OS_iPhone_19G71.tar.gz): https://drive.google.com/file/d/11ENU4kvvSVGQbwpFHl2t-RMulW5TJjvB/view?usp=sharing This bug can be linked with https://bugs.webkit.org/show_bug.cgi?id=243982, I created a new to get a clearer status of the existing regression.
Tried reproducing this. I can kinda do it, but it eventually recovers. Tested on iPhone 12 Pro MGMP3QN/A running iOS 16 Beta 8 (20A5358a) using Airpods Pro A2084 running 4E71 (1.3.8). Test procedure: 1. Go to https://webrtc.github.io/samples/src/content/peerconnection/pc1/ and initiate the call. Observe that you can see and hear yourself. 2. Go to the home screen and navigate to Spotify 3. Start playing a random song. Observe that the capture color disappears in the clock. 4. Return to Safari. Observe that both video views are black and you can no longer hear yourself. 5. Wait 3-4 seconds. Observe that the page starts capturing again. I can also reliably reproduce the same behaviour with the YouTube app. Key seems to be starting audio playback in a native app. Expected behaviour (video mutes when Safari is not the active app) happens when I go to home screen and other apps without playing audio, and recovers as expected quickly when Safari is the active app again. Using my trust gUM debugger (https://codepen.io/daginge/pen/ExLxZzN) I see that the video track mutes as expected when going to the home screen. When the audio starts playing, the audio track mutes. Upon returning to the page, the video and audio track unmutes after 2-3 seconds. Logs: Got stream with constraints: {"audio":true,"video":{"facingMode":"user","deviceId":"default","width":1280,"height":720}} Using video device: Front Camera The video track is currently unmuted Using audio device: AirPods The audio track is currently unmuted Muted video track: true Visibilitychange event triggered: Hidden prop is true Muted audio track: true <-- This is when I start playing Spotify audio Visibilitychange event triggered: Hidden prop is false Unmuted audio track: false Unmuted video track: false However, if I continue playing audio in Spotify, neither the video or audio track unmutes. If I stop playback, they instantly recover. In conclusion: I'm able to reproduce this issue on beta 8 if I continuously play audio. It seems to be an OS mute issue, but the capture is luckily never ended, and will recover.
(In reply to daginge from comment #1) > Tried reproducing this. I can kinda do it, but it eventually recovers. > Tested on iPhone 12 Pro MGMP3QN/A running iOS 16 Beta 8 (20A5358a) using > Airpods Pro A2084 running 4E71 (1.3.8). > > Test procedure: > 1. Go to https://webrtc.github.io/samples/src/content/peerconnection/pc1/ > and initiate the call. Observe that you can see and hear yourself. > 2. Go to the home screen and navigate to Spotify > 3. Start playing a random song. Observe that the capture color disappears in > the clock. > 4. Return to Safari. Observe that both video views are black and you can no > longer hear yourself. > 5. Wait 3-4 seconds. Observe that the page starts capturing again. @daginge, this is somehow expected behaviour. Is your request that restarting capture should happen more quickly? > > I can also reliably reproduce the same behaviour with the YouTube app. Key > seems to be starting audio playback in a native app. Expected behaviour > (video mutes when Safari is not the active app) happens when I go to home > screen and other apps without playing audio, and recovers as expected > quickly when Safari is the active app again. > > Using my trust gUM debugger (https://codepen.io/daginge/pen/ExLxZzN) I see > that the video track mutes as expected when going to the home screen. When > the audio starts playing, the audio track mutes. Upon returning to the page, > the video and audio track unmutes after 2-3 seconds. > > Logs: > Got stream with constraints: > {"audio":true,"video":{"facingMode":"user","deviceId":"default","width":1280, > "height":720}} > Using video device: Front Camera > The video track is currently unmuted > Using audio device: AirPods > The audio track is currently unmuted > Muted video track: true > Visibilitychange event triggered: Hidden prop is true > Muted audio track: true <-- This is when I start playing Spotify audio > Visibilitychange event triggered: Hidden prop is false > Unmuted audio track: false > Unmuted video track: false > > However, if I continue playing audio in Spotify, neither the video or audio > track unmutes. If I stop playback, they instantly recover. Can you clarify this? I would have thought that, when going back to Safari, Safari would restart capture and would interrupt Spotify. Isn't this happening?
(In reply to youenn fablet from comment #2) > (In reply to daginge from comment #1) > > Tried reproducing this. I can kinda do it, but it eventually recovers. > > Tested on iPhone 12 Pro MGMP3QN/A running iOS 16 Beta 8 (20A5358a) using > > Airpods Pro A2084 running 4E71 (1.3.8). > > > > Test procedure: > > 1. Go to https://webrtc.github.io/samples/src/content/peerconnection/pc1/ > > and initiate the call. Observe that you can see and hear yourself. > > 2. Go to the home screen and navigate to Spotify > > 3. Start playing a random song. Observe that the capture color disappears in > > the clock. > > 4. Return to Safari. Observe that both video views are black and you can no > > longer hear yourself. > > 5. Wait 3-4 seconds. Observe that the page starts capturing again. > > @daginge, this is somehow expected behaviour. > Is your request that restarting capture should happen more quickly? Honestly I don't really mind, as long as it reliably restarts. It's riiight at the limit of users being confused I think, but let's see if we get any reports. I think this is such a corner case that we won't hear much. It is faster without the AirPods in though, can't explain that. > > However, if I continue playing audio in Spotify, neither the video or audio > > track unmutes. If I stop playback, they instantly recover. > > Can you clarify this? > I would have thought that, when going back to Safari, Safari would restart > capture and would interrupt Spotify. > Isn't this happening? I'll do you one better and share a video: https://www.icloud.com/photos/#05bDb-5-djR1WZFp3U53sf0Gw (a bit loud due to the music getting picked up). Strangely, with screen recording the behaviour kinda changed, it never recovered. Or rather, the capture seems to recover, but the video went white (which in this context I think means transparent). But if you ignore this fact for now, the behaviour I described is there. The video and audio track doesn't recover until I pause playback. Hope this helps!
> Strangely, with screen recording the behaviour kinda changed, it never > recovered. Or rather, the capture seems to recover, but the video went white > (which in this context I think means transparent). We might still have some bad interaction between autoplay and interruptions here. It might be that calling video.play() will fix the white issue.
<rdar://problem/99574139>
> => From iOS 16 Beta 7 > (iOS_16Beta7_sysdiagnose_2022.08.29_20-41-28+0300_iPhone-OS_iPhone_20A5356a. > tar.gz): > https://drive.google.com/file/d/15_tTigk6tJUFGzhW10DPK6_Vm5pnVNOA/ > view?usp=sharing On this sysdiagnose, I do not find anything wrong. The encoded frame counter is increasing from the beginning of the call 15:20 to the end 15:29. Ditto for audio. I do not see also where the audio get interrupted. Or maybe I did not look at the right timestamp?
> I'll do you one better and share a video: > https://www.icloud.com/photos/#05bDb-5-djR1WZFp3U53sf0Gw (a bit loud due to > the music getting picked up). It does not seem bad that Spotify is not interrupted by moving Safari to foreground. It seems good that user does an action so that it happens. In that case it would be unmuting capture using the red camera icon in the menu bar. I'll have a try with Spotify to verify this is working as expected.
Hi Youenn! I added a new sysdiagnose file: https://drive.google.com/file/d/1Bsq3gIAJOJ7NEK5aBSYBhh8Z9-v5OhXE/view?usp=sharing Bug reproduced on 2022.09.07_10-48-03
Hi, Youenn! After the investigation we see more than one issue: 1. Bluetooth device issue: Use case: Preconditions: iPhone device is connected with AirPods or other Bluetooth device Permissions for camera and microphone are set to "Ask" in browser settings User is in active WebRTC video call Steps: 1. iOS 16 User navigates to Youtube native app or to another native music app and listens music for ~3min 2. iOS 16 User navigates back to the Safari browser 3. iOS 16 User accepts required permissions for audio and video => AirPods option disappears and Microphone device in settings is switching from iPhone mic to "No mic device found". Added sysdiagnose file: https://drive.google.com/file/d/14i4QbOQvr8K3ItZ79S3jaMcP3QUt5X34/view?usp=sharing 2. H264 encoder/decoder issue: Use case: Preconditions: iPhone device is connected with AirPods or other Bluetooth device Permissions for camera and microphone are set to "Ask" in browser settings The WebRTC call is an active, video enabled for other Users in the call Steps: 1. iOS 16 User joins a call 1. iOS 16 User navigates to Youtube native app or to another native music app and listens music for ~3min 2. iOS 16 User navigates back to the Safari browser 3. iOS 16 User accepts required permissions for audio and video => video becomes broken and sometimes also incoming audio disappears + observed that call becomes broken immediately sometimes if iOS User joins an active WebRTC call while cameras enabled for other Users => we don't have incoming video for iOS User in that case. Added sysdiagnose file with this use case: https://drive.google.com/file/d/1oYuYD_mwaIATrpTBcTZjsWWPuFxEwOHo/view?usp=sharing In my opinion there is a higher repro if iPhone device is connected with AirPods, but I was able reproduce defect also with other Bluetooth devices (for example, with Huawei FreeBuds 4i), but with a lower reproducibility. All sysdiagnostics files were generated immediately the defects were reproduced. Tested device: * iPhone 11 Pro Max | iOS 16, build: 20A362
Hi, Youenn! Still can reproduce defects. Added new sysdiagnostics files, recorded logs using iOS 16.1 Beta 2 (Build: 20B5050f) Here using 2nd use case I reproduced this issue: H264 encoder/decoder issue When came back to Safari browser => AirPods was still connected with a device, no audio issue, only outgoing video was broken (Time: 16:46, EET): https://drive.google.com/file/d/1M48-cn5JbkLW57eHzsj_K1ymfZh9gdxW/view?usp=sharing Here using 1st use case use case I reproduced this issue: Bluetooth device issue When came back to Safari browser => AirPods option was not available anymore, didn’t have incoming audio + outgoing video was broken (Time: 17:43, EET): https://drive.google.com/file/d/1rgXXZ6BCZEsLOC7VqVidNkqeb5YabWnf/view?usp=sharing Tested device: * iPhone 11 Pro Max | iOS 16.1 Beta 2, Build: 20B5050f
Hi, Youenn! Bug is still reproducible. Tested device: iPhone 11 Pro Max, iOS 16.2 Beta, Build: 20C5032e. Added new sysdiagnostics file (bug reproduced at 11:05, 07.11.2022 (EET time zone): https://drive.google.com/file/d/1Iek-6-IjsruI4UXepEDRgjfWJWyPK9oP/view?usp=share_link
I’m not able to reproduce described issue in ticket with iOS 16.2 Beta 2, build: 20C5043e, iPhone 11 Pro Max. @Youenn, should we see the fix with this version? Bug linked with https://bugs.webkit.org/show_bug.cgi?id=243982
Bug is not reproducible anymore. Tested OK on: * iPhone 11 Pro Max | iOS 16.2, build: 20C65 * iPhone 12 Pro Max | iOS 16.2, build: 20C65 * iPhone 14 | iOS 16.2, build: 20C65 Thanks!