Bug 268035 - AX: Serve initial AXRelativeFrame for elements without cached frames off of the main thread
Summary: AX: Serve initial AXRelativeFrame for elements without cached frames off of t...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Accessibility (show other bugs)
Version: WebKit Nightly Build
Hardware: All All
: P2 Normal
Assignee: Joshua Hoffman
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2024-01-24 14:39 PST by Joshua Hoffman
Modified: 2024-02-12 10:56 PST (History)
10 users (show)

See Also:


Attachments
Patch (18.50 KB, patch)
2024-01-24 14:48 PST, Joshua Hoffman
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch (18.69 KB, patch)
2024-01-24 14:59 PST, Joshua Hoffman
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch (19.07 KB, patch)
2024-01-24 16:25 PST, Joshua Hoffman
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch (19.17 KB, patch)
2024-01-24 16:52 PST, Joshua Hoffman
no flags Details | Formatted Diff | Diff
Patch (19.87 KB, patch)
2024-01-25 08:56 PST, Joshua Hoffman
no flags Details | Formatted Diff | Diff
Patch (20.07 KB, patch)
2024-02-07 14:51 PST, Joshua Hoffman
no flags Details | Formatted Diff | Diff
Patch (20.03 KB, patch)
2024-02-07 18:52 PST, Joshua Hoffman
no flags Details | Formatted Diff | Diff
Patch (21.53 KB, patch)
2024-02-09 10:42 PST, Joshua Hoffman
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Joshua Hoffman 2024-01-24 14:39:01 PST
We should be serving a rough relative frame from the secondary thread until the frame is cached during painting.
Comment 1 Radar WebKit Bug Importer 2024-01-24 14:39:10 PST
<rdar://problem/121554364>
Comment 2 Joshua Hoffman 2024-01-24 14:39:21 PST
rdar://120764228
Comment 3 Joshua Hoffman 2024-01-24 14:48:13 PST
Created attachment 469535 [details]
Patch
Comment 4 Joshua Hoffman 2024-01-24 14:59:39 PST
Created attachment 469536 [details]
Patch
Comment 5 Joshua Hoffman 2024-01-24 16:25:13 PST
Created attachment 469539 [details]
Patch
Comment 6 Joshua Hoffman 2024-01-24 16:52:03 PST
Created attachment 469540 [details]
Patch
Comment 7 Joshua Hoffman 2024-01-25 08:56:12 PST
Created attachment 469548 [details]
Patch
Comment 8 Joshua Hoffman 2024-02-07 14:51:39 PST
Created attachment 469764 [details]
Patch
Comment 9 Joshua Hoffman 2024-02-07 18:52:33 PST
Created attachment 469765 [details]
Patch
Comment 10 Tyler Wilcock 2024-02-08 08:48:11 PST
Comment on attachment 469765 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=469765&action=review

To reduce memory consumption, when a frame is painted via `AXIsolatedTree::updateFrame`, can we also remove AXPropertyName::InitialFrameRect since it presumably wouldn't be needed anymore?

propertyMap.set(AXPropertyName::RelativeFrame, WTFMove(newFrame));
// Now that we have a cached relative frame, the initial frame rect rough estimate frame is no longer necessary.
propertyMap.set(AXPropertyName::InitialFrameRect, FloatRect());

> Source/WebCore/accessibility/AccessibilityRenderObject.cpp:874
> +bool AccessibilityRenderObject::isSVGRoot() const
> +{
> +    RenderObject* renderObject = renderer();
> +
> +    if (!renderObject)
> +        return false;
> +
> +    return renderObject->isRenderOrLegacyRenderSVGRoot();
> +}

This can be written more simply as:

auto* renderer = this->renderer();
return renderer ? renderer->isRenderOrLegacyRenderSVGRoot() : false;

> Source/WebCore/accessibility/AccessibilityRenderObject.cpp:2736
> +    auto* box = dynamicDowncast<RenderBox>(renderer());
> +    if (!box)
> +        return { };
> +
> +    return convertFrameToSpace(box->frameRect(), AccessibilityConversionSpace::Page);

Can use the same trick as above:

auto* box = dynamicDowncast<RenderBox>(renderer());
return box ? convertFrameToSpace(box->frameRect(), AccessibilityConversionSpace::Page) : FloatRect();

> Source/WebCore/accessibility/AccessibilityRenderObject.cpp:2739
> +

Extra newline here I think

> Source/WebCore/accessibility/AccessibilityRenderObject.h:61
> +    FloatRect frameRect() const override;
> +    bool isSVGRoot() const override;

Can these be final?

> Source/WebCore/accessibility/isolatedtree/AXIsolatedObject.cpp:1248
> +    if (!AXObjectCache::shouldServeInitialCachedFrame() || isMockObject() || isSVGRoot()) {

What about other SVG objects that aren't the SVG root? I don't think we gather frames for those during paint either unless you've found otherwise? Could we add a comment explaining why these ones hit the main-thread?

> Source/WebCore/accessibility/isolatedtree/AXIsolatedObject.cpp:1259
> +    auto* ancestor = Accessibility::findAncestor<AXIsolatedObject>(*this, false, [] (auto& object) {

Can this be `const auto& object`?

> Source/WebCore/accessibility/isolatedtree/AXIsolatedObject.cpp:1263
> +    auto frameRect = rectAttributeValue<FloatRect>(AXPropertyName::InitialFrameRect);
> +    if (ancestor && frameRect.location() == FloatPoint())

If we are only going to use the result of the ancestor traversal if frameRect.location() == FloatPoint(), we also shouldn't spend time doing it if frameRect.location() == FloatPoint().

How often do these frame rects have a 0,0 location vs. not? Why do we decide to "trust" the frame rects location if it happens to not be 0,0? Did this give better results?

> Source/WebCore/accessibility/isolatedtree/AXIsolatedObject.h:82
> +    bool getsGeometryFromChildren() const { return m_getsGeometryFromChildren; }

Is this method used?

> Source/WebCore/accessibility/isolatedtree/AXIsolatedObject.h:83
> +    bool isSVGRoot() const { return boolAttributeValue(AXPropertyName::IsSVGRoot); }

Can this be private instead of public?

> Source/WebCore/accessibility/isolatedtree/AXIsolatedObject.h:221
> +    bool isRelativeFrameCachedByPaint() const { return optionalAttributeValue<IntRect>(AXPropertyName::RelativeFrame).has_value(); }

Maybe hasCachedRelativeFrame?
Comment 11 Joshua Hoffman 2024-02-08 16:51:00 PST
Comment on attachment 469765 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=469765&action=review

>> Source/WebCore/accessibility/AccessibilityRenderObject.h:61
>> +    bool isSVGRoot() const override;
> 
> Can these be final?

Yes, will update that.

>> Source/WebCore/accessibility/isolatedtree/AXIsolatedObject.cpp:1248
>> +    if (!AXObjectCache::shouldServeInitialCachedFrame() || isMockObject() || isSVGRoot()) {
> 
> What about other SVG objects that aren't the SVG root? I don't think we gather frames for those during paint either unless you've found otherwise? Could we add a comment explaining why these ones hit the main-thread?

That is a good point. I will make isSVGRoot more encompassing then (isSVGObject()). 

And yeah we can include a comment, probably even a good place for a FIXME referencing.

>> Source/WebCore/accessibility/isolatedtree/AXIsolatedObject.cpp:1263
>> +    if (ancestor && frameRect.location() == FloatPoint())
> 
> If we are only going to use the result of the ancestor traversal if frameRect.location() == FloatPoint(), we also shouldn't spend time doing it if frameRect.location() == FloatPoint().
> 
> How often do these frame rects have a 0,0 location vs. not? Why do we decide to "trust" the frame rects location if it happens to not be 0,0? Did this give better results?

Frame rects will always be 0,0 before they are painted. So if we are at this point with a 0,0 frame, that can mean two things: (1) the frame is truly at 0,0, but hasn't been painted yet (this is okay because the "offset" from it's ancestor will just be 0) or (2) the frame is unpainted, as all FrameRects are initialized at 0,0.

We "trust" them using the assumption that 0,0 means one of the above. Does that make sense?

>> Source/WebCore/accessibility/isolatedtree/AXIsolatedObject.h:82
>> +    bool getsGeometryFromChildren() const { return m_getsGeometryFromChildren; }
> 
> Is this method used?

Ah this was leftover from an older implementation I had—I will remove it.

>> Source/WebCore/accessibility/isolatedtree/AXIsolatedObject.h:83
>> +    bool isSVGRoot() const { return boolAttributeValue(AXPropertyName::IsSVGRoot); }
> 
> Can this be private instead of public?

Yes

>> Source/WebCore/accessibility/isolatedtree/AXIsolatedObject.h:221
>> +    bool isRelativeFrameCachedByPaint() const { return optionalAttributeValue<IntRect>(AXPropertyName::RelativeFrame).has_value(); }
> 
> Maybe hasCachedRelativeFrame?

I like that—I will change it.
Comment 12 Joshua Hoffman 2024-02-09 10:42:24 PST
Created attachment 469796 [details]
Patch
Comment 13 EWS 2024-02-12 10:56:55 PST
Committed 274472@main (e6c6fde68060): <https://commits.webkit.org/274472@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 469796 [details].