MediaStream canvas.captureStream() fails for WebGL
Produces black or red images
Created attachment 438942 [details]
Comment on attachment 438942 [details]
Nice work Kimmo figuring out how to make this work without the preserveDrawingBuffer hack. Let's see how it looks when the tests are green.
Created attachment 439134 [details]
Created attachment 439141 [details]
Created attachment 439142 [details]
Comment on attachment 439142 [details]
View in context: https://bugs.webkit.org/attachment.cgi?id=439142&action=review
> + flakyness lets rendering happen)
Is there not a way to make a basic test that would:
- check that frames are generated.
- check that frames are not uninitialized (either through video painting in canvas or through internals API, see observeMediaStreamTrack/grabNextMediaStreamTrackFrame).
> + captureStream() and playback implementation is so flaky.
Some tests need rebasing of their expected.txt files
> + return MediaSampleAVFObjC::createImageSample(WTFMove(*pixelBuffer), MediaSampleAVFObjC::VideoRotation::UpsideDown, true);
Why do we get it upside down and mirrored?
Upside down should be correctly handled everywhere, not so sure about mirrored since I am not sure we have sources with mirrored frames.
Created attachment 439597 [details]
Committed r283238 (242280@main): <https://commits.webkit.org/242280@main>
All reviewed patches have been landed. Closing bug and clearing flags on attachment 439597 [details].
*** Bug 229611 has been marked as a duplicate of this bug. ***
(In reply to Jon Lee from comment #10)
> *** Bug 229611 has been marked as a duplicate of this bug. ***
After testing at Monterey, I found that the bug was not fixed and I was wondering if anyone was following up？
From my checks the bug was fixed but only for a moment, before this got broken again.
You can see the canvas to video demo is producing blank square https://webrtc.github.io/samples/src/content/capture/canvas-video/.
Caveat: the demo could work once, I think after a device restart. However if you reload the page, it would no longer work.
(In reply to Alexander Khovansky from comment #12)
> From my checks the bug was fixed but only for a moment, before this got
> broken again.
> You can see the canvas to video demo is producing blank square
> Caveat: the demo could work once, I think after a device restart. However if
> you reload the page, it would no longer work.
The flaky behaviour is due to bug 230623. It will be fixed once that fix ships, unfortunately it is not yet in public releases.
To workaround the flaky behaviour you could try to assign to the srcObject after the video element is created, for example asynchronously in a setTimeout callback.
I still have this issue in Safari 15.3 as well as Technology Preview versions
This issue still not fixed.
1. generate a canvas and a 2d context.
2. fill random number to canvas by context.fillText();
3. canvas.captureStream() and use video tag to play canvas mediaStream
can be reproduced on Mac Safari 15.4
(In reply to rychouwei from comment #15)
> This issue still not fixed.
> 1. generate a canvas and a 2d context.
> 2. fill random number to canvas by context.fillText();
> 3. canvas.captureStream() and use video tag to play canvas mediaStream
> can be reproduced on Mac Safari 15.4
Thanks for the report. This context2d issue is being tracked in bug 230621.
I'll add your bug there.