264894 2023-11-15 12:26:50 -0800 VideoEncoder produces no frames with latencyMode "realtime" when framerate/bitrate are not given 2023-11-23 13:10:42 -0800 1 1 1 Unclassified WebKit Media Safari Technology Preview Mac (Apple Silicon) macOS 14 RESOLVED FIXED InRadar P2 Normal --- 1 jacob youennf eric.carlson jer.noble webkit-bug-importer youennf oldest_to_newest 1992863 0 jacob 2023-11-15 12:26:50 -0800 VideoEncoder with `latencyMode: "realtime"` but no `bitrate` or `framerate` specified does not output any frames, nor error messages. Related: https://bugs.webkit.org/show_bug.cgi?id=264893 & https://bugs.webkit.org/show_bug.cgi?id=258669 Steps to reproduce: 1. Visit https://codesandbox.io/s/cocky-ioana-pjyz5v?file=/src/index.mjs 2. Open the console tab and observe the output is "{keyFrames: 0, deltaFrames: 0}" -- i.e. no frames were output whatsoever. No error messages are present. 3. Uncomment `framerate: fps` and `bitrate: 100000` and observe that the output changes to "{keyFrames: 1, deltaFrames: 19}". Expected behavior: If encoding without these parameters is not supported, then the browser should not claim to support the configuration (via isConfigSupported and by not calling the error callback). Full code below: ``` import "./styles.css"; async function main() { const width = 200; const height = 100; const img = new ImageData(width, height); for (let r=0;r<height;r++) { for (let c=0;c<width;c++) { img.data[(r*width+c)*4+0] = Math.floor(r/height*255); img.data[(r*width+c)*4+1] = Math.floor(c/width*255); img.data[(r*width+c)*4+2] = 127; img.data[(r*width+c)*4+3] = 255; } } const bitmap = await createImageBitmap(img); let keyFrames=0; let deltaFrames=0; const encoder = new VideoEncoder({ output: (chunk, metadata) => { if (chunk.type==="key") { ++keyFrames; } else if (chunk.type === "delta") { ++deltaFrames; } console.log('output chunk',chunk,metadata); }, error: (err) => { console.error('encode error', err) } }) encoder.addEventListener("dequeue", (event) => { console.log('dequeued') }) const fps = 30; encoder.configure({ codec: "avc1.42001f", // Baseline profile (42 00) with level 3.1 (1f) width, height, latencyMode: "realtime", // framerate: fps, avc: { format: "annexb" }, // hardwareAcceleration:"prefer-software" // bitrate:100000, // bitrate: 10000000, }); for (let i=0; i < 20; i++) { const frame = new VideoFrame(bitmap, { timestamp:i * 1/fps * 1e6, duration:1/fps * 1e6, }); encoder.encode(frame, {keyFrame: i === 0}); frame.close(); } bitmap.close(); console.log("sent frames"); await encoder.flush(); console.log("flushed", {keyFrames,deltaFrames}) encoder.close(); console.log("closed", {keyFrames,deltaFrames}) } main().catch((e) => console.error(e,e.message)); ``` 1992867 1 jacob 2023-11-15 12:29:09 -0800 Tested in Safari Technology Preview 182 (Safari 17.4, WebKit 19618.1.4.1). Note that due to https://bugs.webkit.org/show_bug.cgi?id=258669, step 3 does not work in Safari 17.0. 1994437 2 webkit-bug-importer 2023-11-22 12:27:13 -0800 <rdar://problem/118725549> 1994441 3 youennf 2023-11-22 13:03:06 -0800 We might need to set a default bitrate allocation in this case, or bail out. I wonder what other browsers are doing here. 1994495 4 youennf 2023-11-23 00:34:00 -0800 Pull request: https://github.com/WebKit/WebKit/pull/20842 1994578 5 ews-feeder 2023-11-23 12:01:18 -0800 Committed 271087@main (ef16bd3f9b1c): <https://commits.webkit.org/271087@main> Reviewed commits have been landed. Closing PR #20842 and removing active labels. 1994581 6 jacob 2023-11-23 13:10:42 -0800 Thanks for your quick response on this ticket. Would there be any way of ensuring that other similar configuration problems in the future produce an isConfigSupported=false response?