Bug 232551 - [GPU Process] [Filters 5/23] Remove the reference to Filter from FilterEffect
Summary: [GPU Process] [Filters 5/23] Remove the reference to Filter from FilterEffect
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Layout and Rendering (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Said Abou-Hallawa
URL:
Keywords: InRadar
Depends on:
Blocks: 231253
  Show dependency treegraph
 
Reported: 2021-10-31 19:47 PDT by Said Abou-Hallawa
Modified: 2021-11-29 21:38 PST (History)
24 users (show)

See Also:


Attachments
Patch (249.83 KB, patch)
2021-10-31 20:03 PDT, Said Abou-Hallawa
no flags Details | Formatted Diff | Diff
Patch (249.92 KB, patch)
2021-10-31 22:12 PDT, Said Abou-Hallawa
no flags Details | Formatted Diff | Diff
Patch for review (151.87 KB, patch)
2021-10-31 22:19 PDT, Said Abou-Hallawa
no flags Details | Formatted Diff | Diff
Patch (153.42 KB, patch)
2021-11-10 23:07 PST, Said Abou-Hallawa
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch (154.08 KB, patch)
2021-11-10 23:53 PST, Said Abou-Hallawa
no flags Details | Formatted Diff | Diff
Patch (154.55 KB, patch)
2021-11-14 01:47 PST, Said Abou-Hallawa
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Said Abou-Hallawa 2021-10-31 19:47:57 PDT
Instead of having a reference to the Filter in the FilterEffect, we are going to pass a reference to the Filter to the FilterEffect functions only when needed. These are the FilterEffect functions that need have access to the Filter:

determineFilterPrimitiveSubregion()
determineAbsolutePaintRect()
platformApplySoftware()
Comment 1 Said Abou-Hallawa 2021-10-31 20:03:26 PDT
Created attachment 442947 [details]
Patch
Comment 2 Said Abou-Hallawa 2021-10-31 22:12:24 PDT
Created attachment 442955 [details]
Patch
Comment 3 Said Abou-Hallawa 2021-10-31 22:19:22 PDT
Created attachment 442957 [details]
Patch for review
Comment 4 Radar WebKit Bug Importer 2021-11-07 18:48:19 PST
<rdar://problem/85136447>
Comment 5 Said Abou-Hallawa 2021-11-10 23:07:29 PST
Created attachment 443914 [details]
Patch
Comment 6 Said Abou-Hallawa 2021-11-10 23:53:53 PST
Created attachment 443917 [details]
Patch
Comment 7 Cameron McCormack (:heycam) 2021-11-13 20:27:55 PST
Comment on attachment 443917 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=443917&action=review

Refactoring looks OK. A couple of questions below.

> Source/WebCore/ChangeLog:17
> +        Instead of having a reference to the Filter in the FilterEffect, we are
> +        going to pass a reference to the Filter to the FilterEffect functions
> +        only when needed. FilterEffect may need access to the Filter only in two
> +        methods: determineAbsolutePaintRect() and platformApplySoftware().
> +
> +        We need to change all the FilterEffect create methods to not take a Filter
> +        as input. This requires the SVG filter effect elements to change their
> +        build() method. And it requires also CSSFilter to not pass itself to the
> +        FilterEffects its create.

Please mention why we want to do this refactoring. (Mention something about how it's removing an unnecessary dependency of individual FilterEffects on the Filter object, which makes it easier to send the filter description over IPC.)

> Source/WebCore/platform/graphics/filters/FETurbulence.h:96
> +        const Filter* filter;
> +        FETurbulence* effect;

I guess these can't be references because the object is created uninitialized by ParallelJobs?

> Source/WebCore/platform/graphics/filters/FilterEffect.cpp:424
> +    auto buffer = ImageBuffer::create(clampedSize, RenderingMode::Unaccelerated, 1, operatingColorSpace(), PixelFormat::BGRA8);

I suppose this is an intermediate stage, and by the end of the patch queue the page "use accelerated buffers for filters" setting will be honored again?

What will be the behavior for ports that don't use the GPU process? Will it be a regression that they will use an unaccelerated buffer for filters?

> Source/WebCore/rendering/CSSFilter.cpp:240
> +    RefPtr<FilterEffect> previousEffect = SourceGraphic::create();

auto

> Source/WebCore/rendering/CSSFilter.cpp:415
> -    effect->apply();
> +
> +    for (auto& function : m_functions) {

Is this a behavior change, or does it not matter which order you call apply() on the filter functions?

> Source/WebCore/rendering/CSSFilter.cpp:417
> +            downcast<SVGFilter>(function.ptr())->setSourceImage({ sourceImage() });

Maybe downcasting on the reference is better:

```
downcast<SVGFilter>(function.get()).setSourceImage(...);
```

(and the following two downcasts)

> Source/WebCore/rendering/svg/RenderSVGResourceFilter.cpp:239
> -            lastEffect->apply();
> +            filterData.filter->apply();

Same question here: what's the effect of this change?

> Source/WebCore/svg/SVGFEImageElement.cpp:214
> +        return FEImage::create(Ref<Image> { *m_cachedImage->imageForRenderer(renderer()) }, preserveAspectRatio());

These days you can leave the `<Image>` off and just write `Ref { *m_cachedImage ... }`.
Comment 8 Said Abou-Hallawa 2021-11-14 01:46:11 PST
Comment on attachment 443917 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=443917&action=review

>> Source/WebCore/ChangeLog:17
>> +        FilterEffects its create.
> 
> Please mention why we want to do this refactoring. (Mention something about how it's removing an unnecessary dependency of individual FilterEffects on the Filter object, which makes it easier to send the filter description over IPC.)

Done.

>> Source/WebCore/platform/graphics/filters/FETurbulence.h:96
>> +        FETurbulence* effect;
> 
> I guess these can't be references because the object is created uninitialized by ParallelJobs?

Yes this is correct. This code should be cleaned up when the software filter code is moved to separate files.

>> Source/WebCore/platform/graphics/filters/FilterEffect.cpp:424
>> +    auto buffer = ImageBuffer::create(clampedSize, RenderingMode::Unaccelerated, 1, operatingColorSpace(), PixelFormat::BGRA8);
> 
> I suppose this is an intermediate stage, and by the end of the patch queue the page "use accelerated buffers for filters" setting will be honored again?
> 
> What will be the behavior for ports that don't use the GPU process? Will it be a regression that they will use an unaccelerated buffer for filters?

Yes this is correct. I mentioned in the ChangeLog above that this is going to change in the next patch. But I will add a FIXME comment also saying this is intermediate stage and should be fixed soon. But this should not affect the software filter since we always create the result ImageBuffer with RenderingMode::Unaccelerated. The CoreImage filters have to create the intermediate ImageBuffers with RenderingMode::Accelerated. But currently CoreImage filters have different code paths from this one.

>> Source/WebCore/rendering/CSSFilter.cpp:240
>> +    RefPtr<FilterEffect> previousEffect = SourceGraphic::create();
> 
> auto

SourceGraphic::create() returns a Ref<SourceGraphic>. Using auto will set the type of previousEffect to Ref<SourceGraphic>. This will make it difficult to assign it to any other FilterEffect. And more importantly to need to nullify it with the reference filter case.

>> Source/WebCore/rendering/CSSFilter.cpp:415
>> +    for (auto& function : m_functions) {
> 
> Is this a behavior change, or does it not matter which order you call apply() on the filter functions?

No this is not behavior change. Applying the lastEffect will keep applying its inputs recursively before applying itself till we reach to the SourceGraphic which is the first entry in m_functions.

>> Source/WebCore/rendering/CSSFilter.cpp:417
>> +            downcast<SVGFilter>(function.ptr())->setSourceImage({ sourceImage() });
> 
> Maybe downcasting on the reference is better:
> 
> ```
> downcast<SVGFilter>(function.get()).setSourceImage(...);
> ```
> 
> (and the following two downcasts)

Done.

>> Source/WebCore/rendering/svg/RenderSVGResourceFilter.cpp:239
>> +            filterData.filter->apply();
> 
> Same question here: what's the effect of this change?

No it is not a behavior change. This is what SVGFilter::apply will do:

void SVGFilter::apply()
{
    m_lastEffect->apply(*this);
}

The plan is to change this function by applying the first FilterEffect till we reach the lastEffect. The goal is we do not need to store FilterEffect geometry. We always apply the FilterEffect forward. We will do the calculation of the input FilterEffects first and send these calculation to FilterEffect::apply().

>> Source/WebCore/svg/SVGFEImageElement.cpp:214
>> +        return FEImage::create(Ref<Image> { *m_cachedImage->imageForRenderer(renderer()) }, preserveAspectRatio());
> 
> These days you can leave the `<Image>` off and just write `Ref { *m_cachedImage ... }`.

Done.
Comment 9 Said Abou-Hallawa 2021-11-14 01:47:13 PST
Created attachment 444172 [details]
Patch
Comment 10 EWS 2021-11-14 18:45:37 PST
Committed r285796 (244240@main): <https://commits.webkit.org/244240@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 444172 [details].