Bug 269846 - getUserMedia camera stream lost on history pushState in iOS 17.4 Beta 4
Summary: getUserMedia camera stream lost on history pushState in iOS 17.4 Beta 4
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebRTC (show other bugs)
Version: Safari 17
Hardware: iPhone / iPad iOS 17
: P1 Major
Assignee: Andy Estes
URL: https://github.com/pixelverse-llc/gum...
Keywords: InRadar, Regression
: 271127 271637 272704 (view as bug list)
Depends on:
Blocks: 270318
  Show dependency treegraph
 
Reported: 2024-02-21 06:36 PST by Alex Suzuki
Modified: 2024-05-14 14:24 PDT (History)
5 users (show)

See Also:


Attachments
Minimum reproducible sample (1.55 KB, text/html)
2024-02-21 06:39 PST, Alex Suzuki
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Alex Suzuki 2024-02-21 06:36:36 PST
In iOS 17.4 Beta 4 (and Beta 3 also, did not verify with earlier versions), Safari loses access to the camera stream obtained from getUserMedia() on history.pushState().

As history.pushState() is used in web frameworks for client-side routing, this breaks a lot barcode scanning apps and other apps that rely on the camera stream to be present across navigations.

Minimum reproducible sample:
https://github.com/pixelverse-llc/gum-camera-lost-after-pushstate

1) Load reproducer page
2) Allow camera access, camera feed is displayed
3) Press 'Push state' button => camera feed is lsot
Comment 1 Alex Suzuki 2024-02-21 06:39:41 PST
Created attachment 470000 [details]
Minimum reproducible sample
Comment 2 Radar WebKit Bug Importer 2024-02-21 08:13:53 PST
<rdar://problem/123381737>
Comment 3 youenn fablet 2024-02-21 08:15:40 PST
I can reproduce.
I wonder whether this is a regression in iOS 17.4.
Comment 4 youenn fablet 2024-02-21 08:17:28 PST
(In reply to youenn fablet from comment #3)
> I can reproduce.
> I wonder whether this is a regression in iOS 17.4.

Capture is muted and does not recover without a user gesture.
Capture should probably not be muted in that case.
Comment 5 Alex Suzuki 2024-02-21 09:20:46 PST
Hi youenn, thank you for the quick response – much appreciated.
I have tested the same sample on iOS 17.3 and it does not exhibit this behavior.
Comment 6 Alex Suzuki 2024-02-22 02:28:42 PST
Per WebKit bug priority guidelines I'm changing this to P1 (regression from previously released WebKit version)
Comment 7 Alex Suzuki 2024-02-22 06:37:16 PST
Additional information: this behaviour also occurs if the camera permission setting for the website is "Allow", or the blanket setting for all websites is "Allow", so not presumably not a permission issue.
Comment 8 Alex Suzuki 2024-02-22 07:08:15 PST
@youenn fablet: what exactly do you mean by "does not recover without a user gesture"? Does it mean the user tapping the crossed-through camera icon in the address bar and selecting "Start using camera"? Or is there a programmatic way, triggered through a touch event, to unmute the track?

I extended the reproducer repo with two buttons that do the following:
1) try to unmute track by setting track.enabled = true (does not work)
2) re-acquire MediaStream by releasing old stream/track and obtaining new MediaStream by calling getUserMedia() again (this works, but cumbersome and unexpected)
Comment 9 youenn fablet 2024-02-22 09:02:41 PST
(In reply to Alex Suzuki from comment #8)
> @youenn fablet: what exactly do you mean by "does not recover without a user
> gesture"? Does it mean the user tapping the crossed-through camera icon in
> the address bar and selecting "Start using camera"? Or is there a
> programmatic way, triggered through a touch event, to unmute the track?

The former (user tapping the camera icon).
No programmatic way unfortunately.
We are actively working on fixing this.
Comment 10 Andy Estes 2024-02-22 13:28:24 PST
Pull request: https://github.com/WebKit/WebKit/pull/24971
Comment 11 Alex Suzuki 2024-02-23 00:07:26 PST
Thank you so much for the very fast response to this issue.
Comment 12 EWS 2024-02-23 09:54:31 PST
Committed 275244@main (d97f5a09e4f1): <https://commits.webkit.org/275244@main>

Reviewed commits have been landed. Closing PR #24971 and removing active labels.
Comment 13 Alex Suzuki 2024-02-27 23:46:48 PST
I just checked with the iOS 17.4 Public Beta (build 21E217), and the issue still exists. Does this mean that this fix will not make it into iOS 17.4?
Comment 14 Alex Suzuki 2024-03-05 10:30:43 PST
Checked with the now release iOS 17.4, unfortunately this is still present and can be easily reproduced with the linked sample.
Comment 15 Adam 2024-03-18 19:34:43 PDT
Hello - I discovered this issue in iOS 17.4 and also opened a ticket here: https://bugs.webkit.org/show_bug.cgi?id=271127. Sorry for the duplicate as I didn't find this existing bug report. I also added a minimal reproducible sample in the ticket I opened if it helps.

I couldn't tell if the PR to fix this didn't make it with the release even though the bug did or if the PR didn't actually fix the issue.

Is there any idea when this issue will be fixed?
Comment 16 youenn fablet 2024-03-19 03:28:21 PDT
*** Bug 271127 has been marked as a duplicate of this bug. ***
Comment 17 youenn fablet 2024-03-19 03:29:24 PDT
Bug is fixed in WebKit ToT but I do not think a fix was released in any iOS build yet (stable or beta).

I'll try to comment when this gets out for you guys to validate it.
Comment 18 Alex Suzuki 2024-03-19 04:09:36 PDT
The issue is definitely still present in iOS 17.4. I hope it makes it into 17.4.1.
Comment 19 Adam 2024-03-19 06:45:53 PDT
Thanks @youenn - I appreciate the help. I understand this is a P1 already, but if it helps escalate at all, here are some notes on the level of impact I'm experiencing.

Because this is a new regression, introduced in Safari iOS 17.4, I would expect many more reports due to the variety of use cases that could involve a webcam stream and client side navigation. I think the reason there may not be more reports yet is because the issue is difficult to pinpoint in an existing app. Once it is pinpointed however, it is simple to reproduce consistently per Alex's minimum reproducible sample and mine. In these cases, the webcam stream goes completely black, blocking main functionality (like scanning barcodes, taking pictures, etc). I would expect a lot of these cases to involve a highly negative business impact. Speaking for myself, as a software engineer, I found this bug difficult to pinpoint as it was hidden within application code. I'm now faced with the challenge of (temporarily) replacing client side navigation with browser storage functionality, which is a large and delicate task. My workaround is not ideal though as it will degrade the product. And until I'm able to reliably rollout my workaround, large volumes of users are having a poor experience on our website.

Anyways, I don't want to add too much noise, but I hope this gets escalated accordingly and hope that my above experience is useful. I'll stay tuned.

Thanks again for the help!
Comment 20 Alex Suzuki 2024-03-21 14:24:29 PDT
Issue still occurs in iOS 17.4.1
Comment 21 youenn fablet 2024-03-27 08:21:01 PDT
*** Bug 271637 has been marked as a duplicate of this bug. ***
Comment 22 Alex Suzuki 2024-04-03 00:22:10 PDT
Issue seems to be fixed in iOS 17.5 Developer Beta 1
Comment 23 youenn fablet 2024-04-03 05:41:38 PDT
(In reply to Alex Suzuki from comment #22)
> Issue seems to be fixed in iOS 17.5 Developer Beta 1
Comment 24 Ahmad Saleem 2024-04-19 02:58:59 PDT
*** Bug 272704 has been marked as a duplicate of this bug. ***
Comment 25 Adam 2024-05-14 13:14:42 PDT
I'm seeing this issue now in Chrome for iOS version 124.0.6367.111 via this reproduction link: https://get-user-media-sage.vercel.app/

Description of the repro link: https://github.com/adamhenson/get-user-media

Is it known that this exists not just in Safari for iOS but also Chrome for iOS?

Will the PR from this comment (https://bugs.webkit.org/show_bug.cgi?id=269846#c12)in iOS 17.5 stable for all Webkit browsers (not just Safari for iOS)?
Comment 26 Alexey Proskuryakov 2024-05-14 14:24:26 PDT
This fix was in WebKit code, and would apply to all clients. There may be additional issues though.

If you are still seeing similar issues in iOS 17.5, including in Chrome, please file a new bug with an example, steps to reproduce, and anything you know about when this last worked. Ideally, please have everything in the bug report, clicking through to other bugs or GitHub takes time and increases confusion.