Bug 264642 - AX: Cache accessibilityText to serve Title, Description, Help Text attributes off of the main thread
Summary: AX: Cache accessibilityText to serve Title, Description, Help Text attributes...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Accessibility (show other bugs)
Version: WebKit Nightly Build
Hardware: All All
: P2 Normal
Assignee: Joshua Hoffman
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2023-11-10 15:48 PST by Joshua Hoffman
Modified: 2023-11-16 19:03 PST (History)
10 users (show)

See Also:


Attachments
Patch (28.66 KB, patch)
2023-11-10 16:23 PST, Joshua Hoffman
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch (28.66 KB, patch)
2023-11-10 16:47 PST, Joshua Hoffman
no flags Details | Formatted Diff | Diff
Patch (27.45 KB, patch)
2023-11-15 09:26 PST, Joshua Hoffman
no flags Details | Formatted Diff | Diff
Patch (27.63 KB, patch)
2023-11-15 15:16 PST, Joshua Hoffman
no flags Details | Formatted Diff | Diff
Patch (27.62 KB, patch)
2023-11-15 21:13 PST, Joshua Hoffman
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Joshua Hoffman 2023-11-10 15:48:11 PST
We currently go to the main thread the first time Title/Description/Help text is requested for an element. We should be serving these on of the secondary thread and caching them upfront.
Comment 1 Radar WebKit Bug Importer 2023-11-10 15:48:24 PST
<rdar://problem/118255218>
Comment 2 Joshua Hoffman 2023-11-10 16:23:18 PST
Created attachment 468566 [details]
Patch
Comment 3 Joshua Hoffman 2023-11-10 16:47:43 PST
Created attachment 468567 [details]
Patch
Comment 4 Tyler Wilcock 2023-11-13 10:17:22 PST
Comment on attachment 468567 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=468567&action=review

> Source/WebCore/accessibility/AXCoreObject.h:678
> +    AccessibilityText(const String& t, const AccessibilityTextSource& s)
> +        : text(t)
> +        , textSource(s)

I know you're only copy-pasting this, but maybe we can change it to follow the style guidelines and give these proper names, e.g. `text` and `source`

> Source/WebCore/accessibility/AccessibilityObject.h:121
> +    bool isFileUploadButton() const override;

Can this be final?

> Source/WebCore/accessibility/AccessibilityObject.h:190
> +    bool isMeter() const override;

Can this be final?

> Source/WebCore/accessibility/isolatedtree/mac/AXIsolatedObjectMac.mm:72
> +    Vector<AccessibilityText> texts;
> +    object->accessibilityText(texts);

This may be a good opportunity to make AXCoreObject::accessibilityText return a Vector<AccessibilityText>, rather than returning void and taking in a Vector<AccessibilityText>&. Although that's not related to the intention of your patch, so if you wanted to skip it that's OK too.

> Source/WebCore/accessibility/isolatedtree/mac/AXIsolatedObjectMac.mm:75
> +    auto axTextValue = texts.map([] (const auto& text) -> AccessibilityText {
> +        return { text.text.isolatedCopy(), text.textSource };
> +    });

This is an unresearched idea...but is there some text sources we can filter out before caching AccessibilityText to save memory? We should be caching the minimum to correctly serve title, description, and help text. e.g., if an object computes text with AccessibilityTextSource::Subtitle, is that needed in any way to compute any of the three properties in question?
Comment 5 Joshua Hoffman 2023-11-13 10:24:20 PST
Comment on attachment 468567 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=468567&action=review

>> Source/WebCore/accessibility/AXCoreObject.h:678
>> +        , textSource(s)
> 
> I know you're only copy-pasting this, but maybe we can change it to follow the style guidelines and give these proper names, e.g. `text` and `source`

Yeah, I'll go ahead and update those.

>> Source/WebCore/accessibility/AccessibilityObject.h:190
>> +    bool isMeter() const override;
> 
> Can this be final?

yes to both of these.

>> Source/WebCore/accessibility/isolatedtree/mac/AXIsolatedObjectMac.mm:75
>> +    });
> 
> This is an unresearched idea...but is there some text sources we can filter out before caching AccessibilityText to save memory? We should be caching the minimum to correctly serve title, description, and help text. e.g., if an object computes text with AccessibilityTextSource::Subtitle, is that needed in any way to compute any of the three properties in question?

I think that's a great call out! I will audit what text sources we actually need for those.
Comment 6 Joshua Hoffman 2023-11-15 09:07:25 PST
Comment on attachment 468567 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=468567&action=review

>> Source/WebCore/accessibility/isolatedtree/mac/AXIsolatedObjectMac.mm:72
>> +    object->accessibilityText(texts);
> 
> This may be a good opportunity to make AXCoreObject::accessibilityText return a Vector<AccessibilityText>, rather than returning void and taking in a Vector<AccessibilityText>&. Although that's not related to the intention of your patch, so if you wanted to skip it that's OK too.

Yeah we should definitely do this. I'll break this into a follow-up patch since it will be a significant change (we'll also want to update titleElementText, alternativeText, visibleText, and helpText to return a vector too).

>>> Source/WebCore/accessibility/isolatedtree/mac/AXIsolatedObjectMac.mm:75
>>> +    });
>> 
>> This is an unresearched idea...but is there some text sources we can filter out before caching AccessibilityText to save memory? We should be caching the minimum to correctly serve title, description, and help text. e.g., if an object computes text with AccessibilityTextSource::Subtitle, is that needed in any way to compute any of the three properties in question?
> 
> I think that's a great call out! I will audit what text sources we actually need for those.

Unfortunately we use all of the different sources for calculating Title/Description/Help, so we can't do any filtering here.
Comment 7 Joshua Hoffman 2023-11-15 09:26:01 PST
Created attachment 468604 [details]
Patch
Comment 8 Joshua Hoffman 2023-11-15 15:16:53 PST
Created attachment 468611 [details]
Patch
Comment 9 Joshua Hoffman 2023-11-15 21:13:43 PST
Created attachment 468616 [details]
Patch
Comment 10 EWS 2023-11-16 19:03:13 PST
Committed 270860@main (ef43e42c5970): <https://commits.webkit.org/270860@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 468616 [details].