Bug 190697

Summary: Add new image type for CSS painting API
Product: WebKit Reporter: Justin Michaud <justin_michaud>
Component: Layout and RenderingAssignee: Justin Michaud <justin_michaud>
Status: RESOLVED FIXED    
Severity: Normal CC: bfulgham, commit-queue, dino, rniwa, simon.fraser, webkit-bug-importer, zalan
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Bug Depends on:    
Bug Blocks: 190217    
Attachments:
Description Flags
Patch
none
Patch none

Description Justin Michaud 2018-10-17 20:10:14 PDT
Add new image type for CSS painting API, and hook it up to draw something.
Comment 1 Justin Michaud 2018-10-17 20:33:15 PDT
Created attachment 352681 [details]
Patch
Comment 2 Justin Michaud 2018-10-18 11:34:53 PDT
Created attachment 352710 [details]
Patch
Comment 3 Dean Jackson 2018-10-18 16:13:33 PDT
Comment on attachment 352710 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=352710&action=review

> Source/WebCore/platform/graphics/CustomPaintImage.cpp:76
> +    auto canvas = OffscreenCanvas::create(*scriptExecutionContext, destSize.width(), destSize.height());
> +    ExceptionOr<OffscreenRenderingContext> contextOrException = canvas->getContext(*execState, OffscreenCanvas::RenderingContextType::Webgl, { });
> +
> +    if (contextOrException.hasException())
> +        return ImageDrawResult::DidNothing;
> +    auto context = contextOrException.releaseReturnValue();
> +
> +    context->clearColor(0, 0, 0, 1.0);
> +    context->clear(GL_COLOR_BUFFER_BIT);
> +
> +    auto result = m_paintCallback->handleEvent(*context);
> +    if (result.type() != CallbackResultType::Success)
> +        return ImageDrawResult::DidNothing;
> +
> +    auto bitmap = canvas->transferToImageBitmap();
> +    if (!bitmap)
> +        return ImageDrawResult::DidNothing;
> +
> +    destContext.drawImage(*bitmap->buffer()->copyImage(), FloatPoint());

This will work fine but will be too slow in practice, since transferToImageBitmap is a GPU readback. However, it's a good place to start.

Ultimately I don't expect we'll want to support WebGL initially - only 2d contexts. It's a bit scary to think that any element on a page could require a WebGL canvas/context.
Comment 4 Justin Michaud 2018-10-18 16:25:12 PDT
(In reply to Dean Jackson from comment #3)
> Comment on attachment 352710 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=352710&action=review
> 
> > Source/WebCore/platform/graphics/CustomPaintImage.cpp:76
> > +    auto canvas = OffscreenCanvas::create(*scriptExecutionContext, destSize.width(), destSize.height());
> > +    ExceptionOr<OffscreenRenderingContext> contextOrException = canvas->getContext(*execState, OffscreenCanvas::RenderingContextType::Webgl, { });
> > +
> > +    if (contextOrException.hasException())
> > +        return ImageDrawResult::DidNothing;
> > +    auto context = contextOrException.releaseReturnValue();
> > +
> > +    context->clearColor(0, 0, 0, 1.0);
> > +    context->clear(GL_COLOR_BUFFER_BIT);
> > +
> > +    auto result = m_paintCallback->handleEvent(*context);
> > +    if (result.type() != CallbackResultType::Success)
> > +        return ImageDrawResult::DidNothing;
> > +
> > +    auto bitmap = canvas->transferToImageBitmap();
> > +    if (!bitmap)
> > +        return ImageDrawResult::DidNothing;
> > +
> > +    destContext.drawImage(*bitmap->buffer()->copyImage(), FloatPoint());
> 
> This will work fine but will be too slow in practice, since
> transferToImageBitmap is a GPU readback. However, it's a good place to start.
> 
> Ultimately I don't expect we'll want to support WebGL initially - only 2d
> contexts. It's a bit scary to think that any element on a page could require
> a WebGL canvas/context.

I did that because it was the only offscreen canvas I could get easily. The spec actually only supports 2d.
Comment 5 WebKit Commit Bot 2018-10-18 16:59:58 PDT
Comment on attachment 352710 [details]
Patch

Clearing flags on attachment: 352710

Committed r237276: <https://trac.webkit.org/changeset/237276>
Comment 6 WebKit Commit Bot 2018-10-18 17:00:00 PDT
All reviewed patches have been landed.  Closing bug.
Comment 7 Radar WebKit Bug Importer 2018-10-18 17:00:50 PDT
<rdar://problem/45390166>
Comment 8 Simon Fraser (smfr) 2018-10-19 21:26:32 PDT
Comment on attachment 352710 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=352710&action=review

In general this approach doesn't seem right. We should not be firing the callback synchronously at paint time. We should be firing the call back at requestAnimationFrame time; the context drawn into should be backed by a display list, and in this doCustomPaint code we should just be playing back that display list.

> Source/WebCore/css/CSSPaintCallback.h:41
> +    virtual CallbackResult<void> handleEvent(WebGLRenderingContext&) = 0;

Why is this using WebGLRenderingContext? The custom paint API is al 2d.

> Source/WebCore/platform/graphics/CustomPaintImage.cpp:59
> +    ExceptionOr<OffscreenRenderingContext> contextOrException = canvas->getContext(*execState, OffscreenCanvas::RenderingContextType::Webgl, { });

Shouldn't be WebGL.