Bug 231353 - incoming H.264 video is not correctly color managed
Summary: incoming H.264 video is not correctly color managed
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebRTC (show other bugs)
Version: WebKit Local Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Cameron McCormack (:heycam)
URL:
Keywords: InRadar
Depends on:
Blocks: 231645 229025
  Show dependency tree / graph
 
Reported: 2021-10-07 00:05 PDT by Cameron McCormack (:heycam)
Modified: 2021-10-13 00:51 PDT (History)
10 users (show)

See Also:

Attachments
test 1 (1.20 KB, text/html)
2021-10-07 00:06 PDT, Cameron McCormack (:heycam) 		no flags Details
test 2 (1.41 KB, text/html)
2021-10-07 00:06 PDT, Cameron McCormack (:heycam) 		no flags Details
test 3 (1.41 KB, text/html)
2021-10-07 00:06 PDT, Cameron McCormack (:heycam) 		no flags Details
WIP patch (3.27 KB, patch)
2021-10-07 00:09 PDT, Cameron McCormack (:heycam) 		no flags Details | Formatted Diff | Diff
patch (3.43 KB, patch)
2021-10-07 14:45 PDT, Cameron McCormack (:heycam) 		no flags Details | Formatted Diff | Diff
Patch for landing (5.44 KB, patch)
2021-10-12 18:38 PDT, Cameron McCormack (:heycam) 		no flags Details | Formatted Diff | Diff
Show Obsolete (2) View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Cameron McCormack (:heycam) 2021-10-07 00:05:57 PDT 
Attaching a few tests, which use canvas.captureStream(), RTCConnectionPeer, and <video> to convert an sRGB canvas to a video, locally.  In Safari on Monterey, the colors are not quite right.  Debugging showed that the CMSampleBuffers that are provided by the VTDecompressionSession have the VT default guessed color space attachments of Rec.601.  I think these should be sRGB, as that's the default for WebRTC streams.
Comment 1 Cameron McCormack (:heycam) 2021-10-07 00:06:17 PDT 
Created attachment 440479 [details]
test 1
Comment 2 Cameron McCormack (:heycam) 2021-10-07 00:06:32 PDT 
Created attachment 440480 [details]
test 2
Comment 3 Cameron McCormack (:heycam) 2021-10-07 00:06:48 PDT 
Created attachment 440481 [details]
test 3
Comment 4 Radar WebKit Bug Importer 2021-10-07 00:09:10 PDT 
<rdar://problem/83969311>
Comment 5 Cameron McCormack (:heycam) 2021-10-07 00:09:35 PDT 
Created attachment 440483 [details]
WIP patch
Comment 6 Cameron McCormack (:heycam) 2021-10-07 14:45:56 PDT 
Created attachment 440539 [details]
patch
Comment 7 youenn fablet 2021-10-08 00:43:06 PDT 
Comment on attachment 440539 [details]
patch

Let's add a test if possible by adding an Internals API method to grab the color space of webrtc tracks.
You can look at Internals.observeMediaStreamTrack and related.
As of the test, you can write your own WebRTC test or update an existing one, for instance webrtc/video.html
Comment 8 Cameron McCormack (:heycam) 2021-10-12 16:08:32 PDT 
I wanted to use grabNextMediaStreamTrackFrame() and inspect the converted-to-RGB pixel values, but that doesn't seem to be working when the frame is not already RGB.  Filed bug 231555 for that.  I'll write a test that relies on canvas.drawImage(video), once bug 229025 lands.
Comment 9 Cameron McCormack (:heycam) 2021-10-12 18:38:43 PDT 
Created attachment 441028 [details]
Patch for landing
Comment 10 Cameron McCormack (:heycam) 2021-10-12 20:03:08 PDT 
Test in bug 231645.
Comment 11 EWS 2021-10-13 00:51:19 PDT 
Committed r284082 (242910@main): <https://commits.webkit.org/242910@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 441028 [details].