Bug 231353

Summary: incoming H.264 video is not correctly color managed
Product: WebKit Reporter: Cameron McCormack (:heycam) <heycam>
Component: WebRTCAssignee: Cameron McCormack (:heycam) <heycam>
Status: RESOLVED FIXED    
Severity: Normal CC: eric.carlson, ews-watchlist, glenn, jer.noble, kkinnunen, philipj, sam, sergio, webkit-bug-importer, youennf
Priority: P2 Keywords: InRadar
Version: WebKit Local Build   
Hardware: Unspecified   
OS: Unspecified   
Bug Depends on:    
Bug Blocks: 231645, 229025    
Attachments:
Description Flags
test 1
none
test 2
none
test 3
none
WIP patch
none
patch
none
Patch for landing none

Description Cameron McCormack (:heycam) 2021-10-07 00:05:57 PDT
Attaching a few tests, which use canvas.captureStream(), RTCConnectionPeer, and <video> to convert an sRGB canvas to a video, locally.  In Safari on Monterey, the colors are not quite right.  Debugging showed that the CMSampleBuffers that are provided by the VTDecompressionSession have the VT default guessed color space attachments of Rec.601.  I think these should be sRGB, as that's the default for WebRTC streams.
Comment 1 Cameron McCormack (:heycam) 2021-10-07 00:06:17 PDT
Created attachment 440479 [details]
test 1
Comment 2 Cameron McCormack (:heycam) 2021-10-07 00:06:32 PDT
Created attachment 440480 [details]
test 2
Comment 3 Cameron McCormack (:heycam) 2021-10-07 00:06:48 PDT
Created attachment 440481 [details]
test 3
Comment 4 Radar WebKit Bug Importer 2021-10-07 00:09:10 PDT
<rdar://problem/83969311>
Comment 5 Cameron McCormack (:heycam) 2021-10-07 00:09:35 PDT
Created attachment 440483 [details]
WIP patch
Comment 6 Cameron McCormack (:heycam) 2021-10-07 14:45:56 PDT
Created attachment 440539 [details]
patch
Comment 7 youenn fablet 2021-10-08 00:43:06 PDT
Comment on attachment 440539 [details]
patch

Let's add a test if possible by adding an Internals API method to grab the color space of webrtc tracks.
You can look at Internals.observeMediaStreamTrack and related.
As of the test, you can write your own WebRTC test or update an existing one, for instance webrtc/video.html
Comment 8 Cameron McCormack (:heycam) 2021-10-12 16:08:32 PDT
I wanted to use grabNextMediaStreamTrackFrame() and inspect the converted-to-RGB pixel values, but that doesn't seem to be working when the frame is not already RGB.  Filed bug 231555 for that.  I'll write a test that relies on canvas.drawImage(video), once bug 229025 lands.
Comment 9 Cameron McCormack (:heycam) 2021-10-12 18:38:43 PDT
Created attachment 441028 [details]
Patch for landing
Comment 10 Cameron McCormack (:heycam) 2021-10-12 20:03:08 PDT
Test in bug 231645.
Comment 11 EWS 2021-10-13 00:51:19 PDT
Committed r284082 (242910@main): <https://commits.webkit.org/242910@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 441028 [details].