Bug 252465 - In PWA, HTML Video Element may be unable to play stream from 'getUserMedia()'
Summary: In PWA, HTML Video Element may be unable to play stream from 'getUserMedia()'
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebRTC (show other bugs)
Version: Safari 16
Hardware: iPhone / iPad iOS 16
: P2 Major
Assignee: youenn fablet
URL:
Keywords: InRadar
: 234509 (view as bug list)
Depends on:
Blocks:
 
Reported: 2023-02-17 00:51 PST by cube
Modified: 2023-10-20 06:04 PDT (History)
20 users (show)

See Also:


Attachments
reproduce the bug (100.79 MB, video/mp4)
2023-02-17 02:16 PST, cube
no flags Details
iPad 6th Gen - iOS 16.3.1 (768.29 KB, text/plain)
2023-02-28 11:26 PST, Chris
no flags Details
Camera on PWA returns black image (288.92 KB, image/png)
2023-05-17 21:12 PDT, Fred S
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description cube 2023-02-17 00:51:55 PST
I get the media stream by 'getUserMedia()' and make it the source of HTML Video Element. Then I play the video after video is loaded.

It works all fine in Safari.

But the problem is that when it runs in PWA, it somestimes doesn't work. HTML Video Element is unable to play the stream, although the stream is active.

It reproducts after closing the PWA, reopen it right away, and repeat the operation above for several times(maybe 10+ times). 'vide.play()' is always in 'pending'. Also, when it reproducted, I find the icon which indicates the camera is in use will appear but disappear soon. If I close the PWA at this time, wait for 5s or so, and then reopen the PWA,it backs to normal. 

Reproduct both in iPhone 12 mini IOS 16.3 and iPad Pro Generation 3 IOS 15.7.2.

Test code:https://github.com/Cube-J/pwa_camera_test/blob/main/helloworld-pwa.html
Run: https://cube-j.github.io/pwa_camera_test/helloworld-pwa.html
Comment 1 cube 2023-02-17 02:16:41 PST
Created attachment 465049 [details]
reproduce the bug
Comment 2 Radar WebKit Bug Importer 2023-02-24 00:52:49 PST
<rdar://problem/105874017>
Comment 3 Chris 2023-02-28 11:05:33 PST
I'm having the EXACT same problem and I've literally been banging my head against a wall for weeks now. The mediaStream shows that it has a track and it's 'live', not 'muted' and 'enabled' but the camera icon on the top-right either doesn't show or it just shows the red icon and not the green 'dot' and the camera definitely does not work (just black).

I'm kind of relying on this working as it's a fundamental part of a web application that forms the crux of my business.
Comment 4 Chris 2023-02-28 11:06:44 PST
It seems to happen randomly as OP mentions but I've also noticed you can instantly 'kill' the camera by simply pulling down the control center from the top of the screen. Probably not a webkit bug but more an iOS problem.
Comment 5 youenn fablet 2023-02-28 11:17:38 PST
> I'm having the EXACT same problem and I've literally been banging my head
> against a wall for weeks now. The mediaStream shows that it has a track and
> it's 'live', not 'muted' and 'enabled' but the camera icon on the top-right
> either doesn't show or it just shows the red icon and not the green 'dot'
> and the camera definitely does not work (just black).

If the camera icon does not show up, the track should be ended.
Can you provide a sysdiagnose (youenn@apple.com) in the case the track is not ended and video element does not display it?

> I've also noticed you can instantly 'kill' the camera by simply pulling down the control center from the top of the screen.

For this one, the track readyState should get to ended, the ended event should happen. You should be able to restart capture.
Comment 6 Chris 2023-02-28 11:26:06 PST
Created attachment 465228 [details]
iPad 6th Gen - iOS 16.3.1
Comment 7 Chris 2023-02-28 11:29:19 PST
> If the camera icon does not show up, the track should be ended.
> Can you provide a sysdiagnose (youenn@apple.com) in the case the track is
> not ended and video element does not display it?

ips file in comment 6 - the camera icon flashes on screen briefly OR it does reappear but without the green dot - the mediastream is showing as active and the track is there and enabled, not muted and with a readystate of 'live', but the camera does not function

> For this one, the track readyState should get to ended, the ended event
> should happen. You should be able to restart capture.

Yes, you're right - this is a different issue
Comment 8 Tim F 2023-03-13 08:16:07 PDT
We are also having this issue using our PWA with the Dynamsoft drivers License scanning tool.  We have spent a few months now working with them to finally come to the conclusion that this bug is the root cause of our issue.  Because this is a core function of our companies PWA is is critical to resolve this bug.  We are willing to provide any feedback necessary to help if needed however it sounds like the information in the bug ticket covers it pretty well. Thank you for escalating this issue and we hope it is resolved soon.
Comment 9 Chris 2023-03-13 08:30:44 PDT
(In reply to Tim F from comment #8)
> We are also having this issue using our PWA with the Dynamsoft drivers
> License scanning tool.  We have spent a few months now working with them to
> finally come to the conclusion that this bug is the root cause of our issue.
> Because this is a core function of our companies PWA is is critical to
> resolve this bug.  We are willing to provide any feedback necessary to help
> if needed however it sounds like the information in the bug ticket covers it
> pretty well. Thank you for escalating this issue and we hope it is resolved
> soon.

It's reassuring to know I'm not the only one on the planet going crazy with this bug!

Unfortunately after a few emails back and forth with @youennfablet he seems to have ghosted me - so not sure if any discovery was made or if this bug is on Apple's radar :(

I made a hilariously poorly shot video of my troubleshooting and evidence of the bug in progress here https://www.icloud.com/photos/#030hsgwXgkDqnhiMINrrxANEA
Comment 10 youenn fablet 2023-03-14 01:56:17 PDT
If someone can easily reproduce the issue, can you check what the video track muted value (https://developer.mozilla.org/en-US/docs/Web/API/MediaStreamTrack/muted) is?
I would expect it to be muted=true.
Comment 11 Chris 2023-03-14 01:59:53 PDT
(In reply to youenn fablet from comment #10)
> If someone can easily reproduce the issue, can you check what the video
> track muted value
> (https://developer.mozilla.org/en-US/docs/Web/API/MediaStreamTrack/muted) is?
> I would expect it to be muted=true.

In my linked video you can see that initially the track is muted, but after restarting the camera it still fails to load even while the track is not muted (the camera indicator is displayed but the green dot isn’t). No amount of relaunching the app fixes this problem and only a reboot of the iPad resolves it
Comment 12 thompsoncwt 2023-03-18 12:43:34 PDT
I wrestled with this bug for many months but I think I finally discovered what triggers it. It seems (to me) that it is triggered when you close the app while a portion of the video is off-screen, or after it has been sent off-screen. 

Two ways that seem to reliably reproduce the bug:
1. Have the video element/recording be bigger than the screen and part of the video is clipped or off-screen, then close the app.
2. Have no part of the video element be off screen, but send it off screen for example by using a text input that brings up the keyboard, then close the app.

After either of those 2 things happen, the app will no longer be able to play the stream from getUserMedia(), even after closing and reopening the app. #2 is especially frustrating for my situation because I prompt users to input a title for their recording, which can trigger the bug if users close the app while on that page. This only happens while the app is in PWA mode on iOS, never while using the Safari browser on iOS, and never on a Mac or other PC with either Safari or Chrome.

The only way I've found to recover functionality is to either restart the device like mentioned by Chris, or what has worked for me is closing the app while the prompt that asks the users to Allow the app to access microphone and camera is visible on screen.
Comment 13 Chris 2023-03-18 12:52:00 PDT
(In reply to thompsoncwt from comment #12)
> I wrestled with this bug for many months but I think I finally discovered
> what triggers it. It seems (to me) that it is triggered when you close the
> app while a portion of the video is off-screen, or after it has been sent
> off-screen. 
> 
> Two ways that seem to reliably reproduce the bug:
> 1. Have the video element/recording be bigger than the screen and part of
> the video is clipped or off-screen, then close the app.
> 2. Have no part of the video element be off screen, but send it off screen
> for example by using a text input that brings up the keyboard, then close
> the app.
> 
> After either of those 2 things happen, the app will no longer be able to
> play the stream from getUserMedia(), even after closing and reopening the
> app. #2 is especially frustrating for my situation because I prompt users to
> input a title for their recording, which can trigger the bug if users close
> the app while on that page. This only happens while the app is in PWA mode
> on iOS, never while using the Safari browser on iOS, and never on a Mac or
> other PC with either Safari or Chrome.
> 
> The only way I've found to recover functionality is to either restart the
> device like mentioned by Chris, or what has worked for me is closing the app
> while the prompt that asks the users to Allow the app to access microphone
> and camera is visible on screen.

Amazing work!

This explanation fits perfectly with my application use case- my app is basically a wizard interface that flips through multiple panels, showing and hiding div elements- one of those div elements is the 'picture taking' stage which contains the canvas and video elements that have the stream attached.. to improve the UX I basically call getUserMedia() before the 'picture taking' stage appears and leave it running when the user proceeds to the next step.

This has to be why the bug is so common for me!

What I will do is test my app by calling getUserMedia() only while the picture taking stage is on screen, I'll then have it stop the stream before the div is hidden- just to see if I can ever trigger the bug.

I would really love to see this fixed so hopefully the more we can keep this thread alive the greater our chances will be...
Comment 14 cube 2023-03-28 23:50:45 PDT
(In reply to thompsoncwt from comment #12)
> I wrestled with this bug for many months but I think I finally discovered
> what triggers it. It seems (to me) that it is triggered when you close the
> app while a portion of the video is off-screen, or after it has been sent
> off-screen. 
> 
> Two ways that seem to reliably reproduce the bug:
> 1. Have the video element/recording be bigger than the screen and part of
> the video is clipped or off-screen, then close the app.
> 2. Have no part of the video element be off screen, but send it off screen
> for example by using a text input that brings up the keyboard, then close
> the app.
> 
> After either of those 2 things happen, the app will no longer be able to
> play the stream from getUserMedia(), even after closing and reopening the
> app. #2 is especially frustrating for my situation because I prompt users to
> input a title for their recording, which can trigger the bug if users close
> the app while on that page. This only happens while the app is in PWA mode
> on iOS, never while using the Safari browser on iOS, and never on a Mac or
> other PC with either Safari or Chrome.
> 
> The only way I've found to recover functionality is to either restart the
> device like mentioned by Chris, or what has worked for me is closing the app
> while the prompt that asks the users to Allow the app to access microphone
> and camera is visible on screen.

After avoiding the two situations mentioned above, my app works fine(in iPhone mini12 IOS 16.3). However, after I upgraded my iPhone to ios 16.4 today, the bug comes back!!!
Comment 15 Chris 2023-03-29 00:12:06 PDT
(In reply to cube from comment #14)
> (In reply to thompsoncwt from comment #12)
> > I wrestled with this bug for many months but I think I finally discovered
> > what triggers it. It seems (to me) that it is triggered when you close the
> > app while a portion of the video is off-screen, or after it has been sent
> > off-screen. 
> > 
> > Two ways that seem to reliably reproduce the bug:
> > 1. Have the video element/recording be bigger than the screen and part of
> > the video is clipped or off-screen, then close the app.
> > 2. Have no part of the video element be off screen, but send it off screen
> > for example by using a text input that brings up the keyboard, then close
> > the app.
> > 
> > After either of those 2 things happen, the app will no longer be able to
> > play the stream from getUserMedia(), even after closing and reopening the
> > app. #2 is especially frustrating for my situation because I prompt users to
> > input a title for their recording, which can trigger the bug if users close
> > the app while on that page. This only happens while the app is in PWA mode
> > on iOS, never while using the Safari browser on iOS, and never on a Mac or
> > other PC with either Safari or Chrome.
> > 
> > The only way I've found to recover functionality is to either restart the
> > device like mentioned by Chris, or what has worked for me is closing the app
> > while the prompt that asks the users to Allow the app to access microphone
> > and camera is visible on screen.
> 
> After avoiding the two situations mentioned above, my app works fine(in
> iPhone mini12 IOS 16.3). However, after I upgraded my iPhone to ios 16.4
> today, the bug comes back!!!

Yep I also changed my app according to these potential triggers but the bug is still triggering, it seems to be after the iPad screen times out and you come back to it later on :(

As a side note it also annoys me that pulling down the control center kills the camera - this doesn't happen in native apps. You can usually restart the camera with another getUserMedia() but there's no way programmatically you can detect when somebody has pulled the control center down.

Bug is still present in 16.4 - come on Apple!!!
Comment 16 Tim F 2023-04-07 08:32:47 PDT
With the new iOS 16.4 update, safari is not setting the correct default camera that corresponds with the hardware and operating system for the SDK scanner tool using a PWA. We tested iOS 16.4 using an iPhone 10 through 12 and it is not defaulting to the correct camera setting which will render the SDK scanner unusable, however, if you use an iPhone 14 Pro it just so happens the default camera setting matches the safari suggested camera for the hardware and operating system so it will work (hilariously enough).  I do not believe it is connected to this bug report but could get convoluted.
Comment 17 Tim F 2023-04-10 04:50:02 PDT
***So far the below issue seems to be resolved in 16.4.1***
 
With the new iOS 16.4 update, safari is not setting the correct default camera that corresponds with the hardware and operating system for the SDK scanner tool using a PWA. We tested iOS 16.4 using an iPhone 10 through 12 and it is not defaulting to the correct camera setting which will render the SDK scanner unusable, however, if you use an iPhone 14 Pro it just so happens the default camera setting matches the safari suggested camera for the hardware and operating system so it will work (hilariously enough).  I do not believe it is connected to this bug report but could get convoluted.
Comment 18 Pasquale 2023-04-21 06:17:16 PDT
We have the exact same issue. 
iOS 16.4.1 / iPhone 12

If we add the PWA to home screen and then relaunch it in standalone mode from the home screen, we're unable to re-access the camera stream. Note: the pill on top left that should notify the user about the use of the camera flash and get stuck in a sort of loop.
Comment 19 Mariano Goni 2023-05-02 08:27:46 PDT
I tried a lot of things to solve this. As of 16.4.1 no simple solution is available, it looks like a problem with how the service worker caches info. 

The only solution I found to "reset" the camera state without restarting the iPhone or completely erasing Safari cache, is to have a simple web page that opens and closes the camera, added to the home screen. 

You have to hit open cam, it will fail, then hit the close button, and then close this mini app. After this, this and the other app camera start working, normally only once, or perhaps 4 times in a row, until something happens and you have to do the reset. 

It's been 2 weeks of trial and error to finally find out it's an iOS bug. Where can we report it?
Comment 20 Chris 2023-05-02 08:37:48 PDT
(In reply to Mariano Goni from comment #19)
> I tried a lot of things to solve this. As of 16.4.1 no simple solution is
> available, it looks like a problem with how the service worker caches info. 
> 
> The only solution I found to "reset" the camera state without restarting the
> iPhone or completely erasing Safari cache, is to have a simple web page that
> opens and closes the camera, added to the home screen. 
> 
> You have to hit open cam, it will fail, then hit the close button, and then
> close this mini app. After this, this and the other app camera start
> working, normally only once, or perhaps 4 times in a row, until something
> happens and you have to do the reset. 
> 
> It's been 2 weeks of trial and error to finally find out it's an iOS bug.
> Where can we report it?

Thanks for testing this out- it seems like this bug has been present for a very long time. I'm not sure where else this can be raised but Apple are present in this thread and almost certainly should be aware of it- whether or not they will be able to or willing to fix it is another matter!
Comment 21 NielsSec 2023-05-09 06:00:34 PDT
Hoping to hear anything from Apple on this. Is this currently being worked on?

There doesn't seem to be an acceptable workaround for now. We use the video element to show users their camera stream for a barcode scanner. Having the video play in a small element without clipping anything doesn't seem to reliably fix the issue. In some cases the camera video still won't play. Very frustrating because this has a big impact on user satisfaction.

Restarting the iOS device fixes the issue temporarily, but this is not a feasible workaround for users.
Comment 22 Fred S 2023-05-17 20:48:43 PDT
For us PWA camera often returns a black feed. Always works in browser but PWA returns black.
Comment 23 Fred S 2023-05-17 21:12:54 PDT
Created attachment 466394 [details]
Camera on PWA returns black image
Comment 24 Chris 2023-05-29 13:06:54 PDT
Can confirm still exact same problem on iOS 16.5, easily replicated as per comment 12
Comment 25 Eric Pascal 2023-06-08 04:41:58 PDT
I have the exact same bug, easy to reproduce on Iphone 11 bug a lot harder on iPhone X, I have a truncated video also running to have it fill the whole screen.
Need to close and reopen several times the application to reproduce but seems to come back after a time. 
Thank you for the feedbacks if I find a solution I will post-it.
Comment 26 Eric Pascal 2023-06-08 07:29:22 PDT
(In reply to youenn fablet from comment #10)
> If someone can easily reproduce the issue, can you check what the video
> track muted value
> (https://developer.mozilla.org/en-US/docs/Web/API/MediaStreamTrack/muted) is?
> I would expect it to be muted=true.

the muted is true for me
Comment 27 Eric Pascal 2023-06-08 07:44:27 PDT
(In reply to youenn fablet from comment #5)
> > I'm having the EXACT same problem and I've literally been banging my head
> > against a wall for weeks now. The mediaStream shows that it has a track and
> > it's 'live', not 'muted' and 'enabled' but the camera icon on the top-right
> > either doesn't show or it just shows the red icon and not the green 'dot'
> > and the camera definitely does not work (just black).
> 
> If the camera icon does not show up, the track should be ended.
> Can you provide a sysdiagnose (youenn@apple.com) in the case the track is
> not ended and video element does not display it?
> 
> > I've also noticed you can instantly 'kill' the camera by simply pulling down the control center from the top of the screen.
> 
> For this one, the track readyState should get to ended, the ended event
> should happen. You should be able to restart capture.

the stream readyState is 0 when the problem occurs
Comment 28 Eric Pascal 2023-06-08 08:36:04 PDT
I noticed that in this state if I call 

stream=await navigator.mediaDevices.getUserMedia({ video: { facingMode: 'environment' } })

in the console, the camera turns on then off and a second time it turns on 

Yet the stream is not displayed in the browser, tried to recreate video tag. 
Did many test, it seems that the browser is in a specific state. 

But now the camera indicator is on.
Comment 29 Eric Pascal 2023-06-08 08:44:21 PDT
I noticed in that case that the indicator is red recording camera indicator, but the green dot is not displayed. When it works, the green dot is there but the red recording camera is not there.
Comment 30 Eric Pascal 2023-06-08 09:13:46 PDT
Good news, I added a <input type="file" name="picture" accept="image/*" capture="user"></input>

in the page and clicked on it once.

And the stream began to work again and now I'm unable to have the problem again, it was persistent.
Comment 31 Chris 2023-06-08 09:52:18 PDT
(In reply to Eric Pascal from comment #30)
> Good news, I added a <input type="file" name="picture" accept="image/*"
> capture="user"></input>
> 
> in the page and clicked on it once.
> 
> And the stream began to work again and now I'm unable to have the problem
> again, it was persistent.

I’ve just tried this and it doesn’t work for me, red icon comes back but the feed stays dead, testing on an iPhone 12.

Either way, it’s definitely not a suitable solution for a production application!

Is anyone besides youenn looking into this problem from Apple?
Comment 32 rooke 2023-06-14 11:27:18 PDT
This can reliably be reproduced with the following device:

Model Number: MXCX2VC/A
iOS Version 16.5

Method: 
1) Open up an PWA that uses the camera.
2) Use the camera (however the PWA uses it, ultimately calling navigator.mediaDevices.getUserMedia()). It works normally.
3) Go to the app selection carousel (via swipe up from bottom, or double-tapping home button)
4) Swipe up on the PWA app's preview to close the app.
5) Open up the PWA app again
6) Go to the camera. It will be black/not showing. The following is returned from a call to navigator.mediaDevices.getUserMedia({ audio: false, video: true }):

MediaStreamTrack: {
contentHint: ""
enabled: true
id: <a-uuid-redacted>
kind: "video"
label: "Front Camera"
muted: true
onconfigurationchange: null
onended: null
onmute: null
onunmute: null
readyState: "live"
}
Comment 33 Eric Pascal 2023-06-14 21:51:37 PDT
(In reply to rooke from comment #32)
> This can reliably be reproduced with the following device:
> 
> Model Number: MXCX2VC/A
> iOS Version 16.5
> 
> Method: 
> 1) Open up an PWA that uses the camera.
> 2) Use the camera (however the PWA uses it, ultimately calling
> navigator.mediaDevices.getUserMedia()). It works normally.
> 3) Go to the app selection carousel (via swipe up from bottom, or
> double-tapping home button)
> 4) Swipe up on the PWA app's preview to close the app.
> 5) Open up the PWA app again
> 6) Go to the camera. It will be black/not showing. The following is returned
> from a call to navigator.mediaDevices.getUserMedia({ audio: false, video:
> true }):
> 
> MediaStreamTrack: {
> contentHint: ""
> enabled: true
> id: <a-uuid-redacted>
> kind: "video"
> label: "Front Camera"
> muted: true
> onconfigurationchange: null
> onended: null
> onmute: null
> onunmute: null
> readyState: "live"
> }

Thank you, I confim the same observations.
I would add that it reproduces very easily the same way on iPhone 11/12, less likely in iPhone X. It seems that when the network connection is fast, that the javascript loads fast (workers) it reproduces faster. If you close completely and reopen rapidly the application, it can be easily reproduced too.
Comment 34 rooke 2023-06-15 11:35:58 PDT
Just to add to the information available, and provide a detection method, you can subscribe to the HTMLMediaElement's events (e.g. <video muted="true" />). In a normal situation, the events follow this order:

play 
waiting 
loadstart 
progress 
suspend 
durationchange 
loadedmetadata 
loadeddata 
canplay 
playing 
canplaythrough
timeupdate (repeatedly for every frame)

When the PWA camera issue rears its head, the events are cut off at 'suspend', i.e.:

play 
waiting 
loadstart 
progress 
suspend 
<then nothing>

This is the method to detect when you're in that situation. Just timeout on any of the missing 'expected' events, e.g. canplay:

let timeoutId: NodeJS.Timeout;

// set timeout of 3 seconds to wait for oncanplay
timeoutId = setTimeout(() => {
  // The video didn't get to 'canplay'
}, yourPreferredTimeoutInMillis);

videoElement.oncanplay = () => {
  // clear the timeout because oncanplay was called
  clearTimeout(timeoutId);
};
Comment 35 Chris 2023-06-15 11:44:11 PDT
(In reply to rooke from comment #34)
> Just to add to the information available, and provide a detection method,
> you can subscribe to the HTMLMediaElement's events (e.g. <video muted="true"
> />). In a normal situation, the events follow this order:
> 
> play 
> waiting 
> loadstart 
> progress 
> suspend 
> durationchange 
> loadedmetadata 
> loadeddata 
> canplay 
> playing 
> canplaythrough
> timeupdate (repeatedly for every frame)
> 
> When the PWA camera issue rears its head, the events are cut off at
> 'suspend', i.e.:
> 
> play 
> waiting 
> loadstart 
> progress 
> suspend 
> <then nothing>
> 
> This is the method to detect when you're in that situation. Just timeout on
> any of the missing 'expected' events, e.g. canplay:
> 
> let timeoutId: NodeJS.Timeout;
> 
> // set timeout of 3 seconds to wait for oncanplay
> timeoutId = setTimeout(() => {
>   // The video didn't get to 'canplay'
> }, yourPreferredTimeoutInMillis);
> 
> videoElement.oncanplay = () => {
>   // clear the timeout because oncanplay was called
>   clearTimeout(timeoutId);
> };



This is really helpful, thank you

I’d been doing something similar already with a chained function call that executes after a timer is set to say the camera couldn’t be loaded. If the function succeeds and it progresses past “suspend” then the timer is cleared and the camera is displayed.

Being able to detect the problem was half the battle but obviously having an actual fix for this is the dream! In the meantime a work around to detect the issue and then get the camera working again without user input would be great, too…
Comment 36 Chris 2023-06-22 13:22:53 PDT
Bug is still present in 16.6, tested on iPad 6th Gen and iPhone 12 Mini.
Comment 37 Fabien R 2023-06-25 02:15:47 PDT
Bug is still present in 17.2 beta, tested on iPhone 12.
Comment 38 Chris 2023-06-25 02:45:02 PDT
For a major bug that's existed since at least iOS 14- I'm shocked this still has no 'Assignee'
Comment 39 Andre 2023-06-28 01:45:15 PDT
(In reply to Chris from comment #35)
> (In reply to rooke from comment #34)
> > Just to add to the information available, and provide a detection method,
> > you can subscribe to the HTMLMediaElement's events (e.g. <video muted="true"
> > />). In a normal situation, the events follow this order:
> > 
> > play 
> > waiting 
> > loadstart 
> > progress 
> > suspend 
> > durationchange 
> > loadedmetadata 
> > loadeddata 
> > canplay 
> > playing 
> > canplaythrough
> > timeupdate (repeatedly for every frame)
> > 
> > When the PWA camera issue rears its head, the events are cut off at
> > 'suspend', i.e.:
> > 
> > play 
> > waiting 
> > loadstart 
> > progress 
> > suspend 
> > <then nothing>
> > 
> > This is the method to detect when you're in that situation. Just timeout on
> > any of the missing 'expected' events, e.g. canplay:
> > 
> > let timeoutId: NodeJS.Timeout;
> > 
> > // set timeout of 3 seconds to wait for oncanplay
> > timeoutId = setTimeout(() => {
> >   // The video didn't get to 'canplay'
> > }, yourPreferredTimeoutInMillis);
> > 
> > videoElement.oncanplay = () => {
> >   // clear the timeout because oncanplay was called
> >   clearTimeout(timeoutId);
> > };
> 
> 
> 
> This is really helpful, thank you
> 
> I’d been doing something similar already with a chained function call that
> executes after a timer is set to say the camera couldn’t be loaded. If the
> function succeeds and it progresses past “suspend” then the timer is cleared
> and the camera is displayed.
> 
> Being able to detect the problem was half the battle but obviously having an
> actual fix for this is the dream! In the meantime a work around to detect
> the issue and then get the camera working again without user input would be
> great, too…

Still not working on 16.5.1 :(

Can you show an example of your code how you fixed it with a workaround? I tried a lot but nothing worked for me.
Comment 40 Chris 2023-06-28 01:52:11 PDT
(In reply to Andre from comment #39)
> (In reply to Chris from comment #35)
> > (In reply to rooke from comment #34)
> > > Just to add to the information available, and provide a detection method,
> > > you can subscribe to the HTMLMediaElement's events (e.g. <video muted="true"
> > > />). In a normal situation, the events follow this order:
> > > 
> > > play 
> > > waiting 
> > > loadstart 
> > > progress 
> > > suspend 
> > > durationchange 
> > > loadedmetadata 
> > > loadeddata 
> > > canplay 
> > > playing 
> > > canplaythrough
> > > timeupdate (repeatedly for every frame)
> > > 
> > > When the PWA camera issue rears its head, the events are cut off at
> > > 'suspend', i.e.:
> > > 
> > > play 
> > > waiting 
> > > loadstart 
> > > progress 
> > > suspend 
> > > <then nothing>
> > > 
> > > This is the method to detect when you're in that situation. Just timeout on
> > > any of the missing 'expected' events, e.g. canplay:
> > > 
> > > let timeoutId: NodeJS.Timeout;
> > > 
> > > // set timeout of 3 seconds to wait for oncanplay
> > > timeoutId = setTimeout(() => {
> > >   // The video didn't get to 'canplay'
> > > }, yourPreferredTimeoutInMillis);
> > > 
> > > videoElement.oncanplay = () => {
> > >   // clear the timeout because oncanplay was called
> > >   clearTimeout(timeoutId);
> > > };
> > 
> > 
> > 
> > This is really helpful, thank you
> > 
> > I’d been doing something similar already with a chained function call that
> > executes after a timer is set to say the camera couldn’t be loaded. If the
> > function succeeds and it progresses past “suspend” then the timer is cleared
> > and the camera is displayed.
> > 
> > Being able to detect the problem was half the battle but obviously having an
> > actual fix for this is the dream! In the meantime a work around to detect
> > the issue and then get the camera working again without user input would be
> > great, too…
> 
> Still not working on 16.5.1 :(
> 
> Can you show an example of your code how you fixed it with a workaround? I
> tried a lot but nothing worked for me.

Hi Andre- there is no fix... currently the only way to get the camera back is to restart the iPad/iPhone or wait (for quite a long time, apparently) and close/re-open the home screen app

If you want to detect when the camera can't be loaded you can do something like this in JS:

======================
  var cameraFailTimer
  cameraStarting = true
  const constraints = {
    audio: false,
    video: { 
      facingMode: 'user'
    }
  }
  navigator.mediaDevices.getUserMedia(constraints)
    .then((mediaStream) => {
      cameraView.srcObject = mediaStream
      clearTimeout(cameraFailTimer)
      cameraFailTimer = setTimeout(function(){
        // Camera isn't working - do something on screen
        cameraStarting = false
        
      }, 12000)
      cameraView.onloadedmetadata = () => {
        cameraView.play().then(function() {
          // Camera is working - continue as normal
          clearTimeout(cameraFailTimer)
          cameraStarting = false
          callback(true)
        })
      }
    })
    .catch((err) => {
      // Camera isn't working - do something on screen
      cameraStarting = false
      callback(false)
    })
======================
Comment 41 youenn fablet 2023-06-28 06:13:44 PDT
Pull request: https://github.com/WebKit/WebKit/pull/15370
Comment 42 ssjblue197 2023-07-06 02:22:28 PDT
if your app doesn't need to use service worker then remove it, and it will work perfectly. I checked and found that even if you register a service worker that does nothing, the same problem will occur. This error is caused by ios operating system.
Comment 43 Mariano Goni 2023-07-06 04:22:52 PDT
That's the whole point of the thread. PWA's use service workers.I had to resort to not using one an degrading the functionality of the app (like not being able to sent instant notifications to the system).
Comment 44 Eric Pascal 2023-07-06 05:10:37 PDT
(In reply to ssjblue197 from comment #42)
> if your app doesn't need to use service worker then remove it, and it will
> work perfectly. I checked and found that even if you register a service
> worker that does nothing, the same problem will occur. This error is caused
> by ios operating system.

I think this may be due to the speed of initialization of PWA that doesn't need to load from network and thus go very fast. My issue appears mainly when I launch camera quickly at start, but not always :(
Comment 45 Eric Pascal 2023-07-06 05:12:37 PDT
Apparently a patch has been submitted, check the description it's kinda suttle, the bugs seems to come from the OS but a workaround has been implemented.
Don't know when it will but present in Safari
Comment 46 Chris 2023-07-06 06:18:49 PDT
It’s been submitted as pull request but that’s one request amongst 590 others so who knows when it could be merged and be present in the live code
Comment 47 EWS 2023-07-12 00:02:35 PDT
Committed 265986@main (bbf602136b81): <https://commits.webkit.org/265986@main>

Reviewed commits have been landed. Closing PR #15370 and removing active labels.
Comment 48 Chris 2023-07-12 00:24:22 PDT
I can see the PR has been merged successfully (thanks youenn!) - do we know how long before this will be included in the safari code in iOS?
Comment 49 Matt 2023-07-17 07:05:07 PDT
I've been struggling with the same issue (iOs 16.5.1 on Iphone 14) and like Tim F this is critical to my business that a solution is quickly deployed for this... 

One question: Comment 42 suggests that if you don't register the service worker then the issue doesn't occur.  That isn't my experience, but maybe I'm doing something wrong, are you sure?
Comment 50 Chris 2023-07-17 07:19:04 PDT
(In reply to Matt from comment #49)
> I've been struggling with the same issue (iOs 16.5.1 on Iphone 14) and like
> Tim F this is critical to my business that a solution is quickly deployed
> for this... 
> 
> One question: Comment 42 suggests that if you don't register the service
> worker then the issue doesn't occur.  That isn't my experience, but maybe
> I'm doing something wrong, are you sure?

I don’t think it has anything to do with service workers tbh. My app doesn’t register one at all and I still have this problem when it’s in the “added to Home Screen” environment
Comment 51 Matt 2023-07-17 07:21:57 PDT
Thanks Chris, aligns with my experience.  I've tried not registering and also removing sw.js completely from the file structure but makes no difference.
Comment 52 Chris 2023-07-21 01:20:04 PDT
Fix didn't make it into 16.6 it seems- bug occurred for me immediately (iPhone 12 Mini)
Comment 53 Eric Pascal 2023-07-21 01:27:20 PDT
(In reply to youenn fablet from comment #41)
> Pull request: https://github.com/WebKit/WebKit/pull/15370

Could you tell us how long it will take to be integrated in safari iOS ?
Comment 54 youenn fablet 2023-07-21 01:35:02 PDT
I'll try to update this thread when this becomes available in iOS beta.
I don't think it is available yet in any iOS beta.
Comment 55 Mariano Goni 2023-07-26 05:22:24 PDT
Don't know why the Status can say that, but here again to communicate that now, it doesn't work even without a service worker.

Without a service worker and just with add to the home screen, the camera was opening fine. Today I just tested and the app is unusable, the camera closes instantly. 

Still works on the Ukrainian browser "Bluefy", so one wonders, if it is better to rely on a software maintained by people that are undergoing a war? . I think yes, it's more reliable than to have Apple make bugs randomly appear.
Comment 56 Chris 2023-07-26 05:47:04 PDT
(In reply to Mariano Goni from comment #55)
> Don't know why the Status can say that, but here again to communicate that
> now, it doesn't work even without a service worker.
> 
> Without a service worker and just with add to the home screen, the camera
> was opening fine. Today I just tested and the app is unusable, the camera
> closes instantly. 
> 
> Still works on the Ukrainian browser "Bluefy", so one wonders, if it is
> better to rely on a software maintained by people that are undergoing a war?
> . I think yes, it's more reliable than to have Apple make bugs randomly
> appear.

this bug has been present at least since iOS14 and perhaps even further back than that- certainly I've experienced it for 2 years now since I first launched my affected product.

The problem (hopefully) has been resolved now, we're just waiting for the code to appear in the OS
Comment 57 Fabien R 2023-07-26 06:30:48 PDT
I just installed iOS 17.4 and the bug no longer seems to be present. I tried to reproduce the bug but I can't anymore.
Comment 58 Tim F 2023-07-26 06:40:04 PDT
How can you download iOS 17.4 when the latest version is iOS 16.6?
Comment 59 Tim F 2023-07-26 06:41:02 PDT
(In reply to Fabien R from comment #57)
> I just installed iOS 17.4 and the bug no longer seems to be present. I tried
> to reproduce the bug but I can't anymore.

How can you download iOS 17.4 when the latest version is iOS 16.6?
Comment 60 Fabien R 2023-07-26 06:43:01 PDT
It's available in beta
Comment 61 Chris 2023-07-26 08:19:30 PDT
(In reply to Fabien R from comment #57)
> I just installed iOS 17.4 and the bug no longer seems to be present. I tried
> to reproduce the bug but I can't anymore.

Just tested this myself and unfortunately the bug is still there.
I’m assuming you mean version 17.0 Beta release 4
Comment 62 Mariano Goni 2023-07-26 08:23:17 PDT
Can you confirm if you are testing with or without service worker installed?
Comment 63 Mariano Goni 2023-07-26 08:32:43 PDT
On iOS 17.0 (beta) still present, and even got worse with a simple page just trying to open the camera only (added to home screen).
Comment 64 Chris 2023-07-26 08:40:51 PDT
(In reply to Mariano Goni from comment #62)
> Can you confirm if you are testing with or without service worker installed?

Yep no service worker at all

Testing on an iPad 7th Generation.

Initially I thought it might be resolved but after exiting the app and then returning to it half an hour or so later (after using the Netflix app) the bug immediately occurred
Comment 65 Mariano Goni 2023-07-26 11:10:52 PDT
Try installing free browser "Bluefy" and see if it works or not. Browser has a fullscreen option too.
Comment 66 poincare 2023-07-27 11:32:18 PDT
I have a PWA where the camera is also used. This problem has been going on for years.
Comment 67 Mariano Goni 2023-08-02 01:42:22 PDT
Update from BETA 17: I cleaned up safari cache (history now) and the camera started working again, with no issues. Haven't tried with a service worker yet. Will post results when checked.
Comment 68 Mariano Goni 2023-08-02 03:06:26 PDT
Checked with service worker installed, it works.
It may only start failing if the camera is incorrectly closed. 
In that case, clearing safari cache will reset it.
Hope this fix survives future updates!!!
Comment 69 Chris 2023-08-02 03:16:05 PDT
Until youenn has confirmed that the code from his fix is present in the iOS build we have to assume this is not fixed still.

I'm testing again now but I did successfully recreate the bug last week when testing on iOS 17.0 beta release 4.
Comment 70 Chris 2023-08-02 04:13:49 PDT
(In reply to Chris from comment #69)
> Until youenn has confirmed that the code from his fix is present in the iOS
> build we have to assume this is not fixed still.
> 
> I'm testing again now but I did successfully recreate the bug last week when
> testing on iOS 17.0 beta release 4.

Again, I was able to recreate the bug very easily by opening the home screen app once and using the camera, closing it (cleanly), using Netflix for 20 mins and then re-opening the home screen app. Camera fails immediately
Comment 71 Andre 2023-08-02 04:52:16 PDT
I tested it on an iPhone Se 2020 with IOS 17 beta 4 and still not working with serviceworker
Comment 72 Mariano Goni 2023-08-02 14:25:46 PDT
Check Settings/Safari/Advanced/Feature Flags to allow service workers and a lot of other flags like permissions api
Comment 73 poincare 2023-08-12 09:28:43 PDT
Is the issue still present in ios 17 beta 5? Anyone testing it?
Comment 74 Andre 2023-08-25 04:50:57 PDT
I am testing the camera with the latest beta update on an iPhone SE 2020 with a pwa and a registered serviceworker. For now it seems to work for me. I am testing further but until now no problem anymore.
Comment 75 youenn fablet 2023-09-19 10:20:42 PDT
*** Bug 234509 has been marked as a duplicate of this bug. ***
Comment 76 Philipp Barth 2023-10-20 06:01:49 PDT
iPhone 12 mini IOS 16.6.1 bug still present for approx 7-10 tries, then gone for some tries (opening and closing standalone webapp) then reappears for approx 10 tries.
SW registered
muted = false

problem appears exactly as described above.

on
iphone SE 1st GEN iOS 15.7.3 
NO SW
muted= false

bug doesn't occur at all

does anyone still have trouble with this?
Comment 77 Chris 2023-10-20 06:04:57 PDT
(In reply to Philipp Barth from comment #76)
> does anyone still have trouble with this?

It's been fixed in iOS 17+