Bug 221917 - [macOS] Introduce a new context menu item to preview images
Summary: [macOS] Introduce a new context menu item to preview images
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Platform (show other bugs)
Version: WebKit Nightly Build
Hardware: All All
: P2 Normal
Assignee: Wenson Hsieh
URL:
Keywords: InRadar
Depends on: 221869
Blocks:
  Show dependency treegraph
 
Reported: 2021-02-15 13:12 PST by Wenson Hsieh
Modified: 2021-02-17 07:49 PST (History)
9 users (show)

See Also:


Attachments
Patch (32.16 KB, patch)
2021-02-15 16:30 PST, Wenson Hsieh
no flags Details | Formatted Diff | Diff
Patch (28.69 KB, patch)
2021-02-15 21:19 PST, Wenson Hsieh
no flags Details | Formatted Diff | Diff
Rebaseline test (31.11 KB, patch)
2021-02-15 22:35 PST, Wenson Hsieh
thorton: review+
Details | Formatted Diff | Diff
For landing (31.17 KB, patch)
2021-02-16 22:47 PST, Wenson Hsieh
no flags Details | Formatted Diff | Diff
Rebase on trunk (31.46 KB, patch)
2021-02-17 00:00 PST, Wenson Hsieh
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Wenson Hsieh 2021-02-15 13:12:06 PST
SSIA
Comment 1 Radar WebKit Bug Importer 2021-02-15 13:38:46 PST
<rdar://problem/74363578>
Comment 2 Wenson Hsieh 2021-02-15 16:30:51 PST
Created attachment 420392 [details]
Patch
Comment 3 Darin Adler 2021-02-15 17:30:01 PST
Comment on attachment 420392 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=420392&action=review

> Source/WebCore/page/ContextMenuController.cpp:525
> +#endif // ENABLE(IMAGE_EXTRACTION)

These comments on #endif don’t seem too helpful when the code is <10 lines long.

> Source/WebCore/page/ContextMenuController.cpp:869
> +                ContextMenuItem RevealImageItem(ActionType, ContextMenuItemTagRevealImage, contextMenuItemTagRevealImage());

Why is this not at the top of the function with the other ContextMenuItem objects?

> Source/WebCore/platform/ContextMenuItem.h:152
> +#if ENABLE(IMAGE_EXTRACTION)

Very concerned about #if in the definition of this enum since I think we recently learned that binary compatibility with some clients (at least Mail) relies on this enumeration matching the one in WKContextMenuItemTypes.h. I think we need to do something to make this less fragile/dangerous. Maybe we can start by not making new enumeration values conditional?

> Source/WebKit/Configurations/WebKit.xcconfig:118
> +WK_UNIFORM_TYPE_IDENTIFIERS_LDFLAGS_MACOS_SINCE_1200 = -framework UniformTypeIdentifiers;

Does this have a launch time or web process startup time cost?

> Source/WebKit/Configurations/WebKit.xcconfig:135
> +WK_QUARTZ_LDFLAGS_MACOS_SINCE_1200 = -framework Quartz;

Same question?
Comment 4 Wenson Hsieh 2021-02-15 20:28:29 PST
Comment on attachment 420392 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=420392&action=review

Thanks for the review!

>> Source/WebCore/page/ContextMenuController.cpp:525
>> +#endif // ENABLE(IMAGE_EXTRACTION)
> 
> These comments on #endif don’t seem too helpful when the code is <10 lines long.

Fair point — removed all the #endifs on short code snippets like this, in this patch.

>> Source/WebCore/page/ContextMenuController.cpp:869
>> +                ContextMenuItem RevealImageItem(ActionType, ContextMenuItemTagRevealImage, contextMenuItemTagRevealImage());
> 
> Why is this not at the top of the function with the other ContextMenuItem objects?

No reason! Moved to the top, to match the other items.

>> Source/WebCore/platform/ContextMenuItem.h:152
>> +#if ENABLE(IMAGE_EXTRACTION)
> 
> Very concerned about #if in the definition of this enum since I think we recently learned that binary compatibility with some clients (at least Mail) relies on this enumeration matching the one in WKContextMenuItemTypes.h. I think we need to do something to make this less fragile/dangerous. Maybe we can start by not making new enumeration values conditional?

Hm…if I understand correctly, I think it's probably okay for an enum type like this that's internal to the WebKit stack, but dangerous to shift enum values that are exposed via SPI (which may lead to bincompat bugs like https://bugs.webkit.org/show_bug.cgi?id=220745). I can see some value in making these enum values match up with the SPI-exposed ones though, so I'll remove the `ENABLE(IMAGE_EXTRACTION)` guard here.

>> Source/WebKit/Configurations/WebKit.xcconfig:118
>> +WK_UNIFORM_TYPE_IDENTIFIERS_LDFLAGS_MACOS_SINCE_1200 = -framework UniformTypeIdentifiers;
> 
> Does this have a launch time or web process startup time cost?

Using a simple test app, it looks like linking WebKit (indirectly) pulls in the UniformTypeIdentifiers framework on all the WebKit child processes and the UI process in shipping WebKit anyways, so I think this should not have any impact on launch time/startup cost.

>> Source/WebKit/Configurations/WebKit.xcconfig:135
>> +WK_QUARTZ_LDFLAGS_MACOS_SINCE_1200 = -framework Quartz;
> 
> Same question?

Using the (same) simple test app, it looks like the web process and UI process both indirectly pull in Quartz.framework on shipping WebKit — however, the networking process currently doesn't pull in Quartz, so this linking could potentially regress the startup type in the networking process.

I'll change this so that we softlink Quartz instead.
Comment 5 Wenson Hsieh 2021-02-15 21:19:46 PST
Created attachment 420422 [details]
Patch
Comment 6 Wenson Hsieh 2021-02-15 22:35:15 PST
Created attachment 420423 [details]
Rebaseline test
Comment 7 Darin Adler 2021-02-16 09:23:35 PST
Comment on attachment 420392 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=420392&action=review

>>> Source/WebCore/platform/ContextMenuItem.h:152
>>> +#if ENABLE(IMAGE_EXTRACTION)
>> 
>> Very concerned about #if in the definition of this enum since I think we recently learned that binary compatibility with some clients (at least Mail) relies on this enumeration matching the one in WKContextMenuItemTypes.h. I think we need to do something to make this less fragile/dangerous. Maybe we can start by not making new enumeration values conditional?
> 
> Hm…if I understand correctly, I think it's probably okay for an enum type like this that's internal to the WebKit stack, but dangerous to shift enum values that are exposed via SPI (which may lead to bincompat bugs like https://bugs.webkit.org/show_bug.cgi?id=220745). I can see some value in making these enum values match up with the SPI-exposed ones though, so I'll remove the `ENABLE(IMAGE_EXTRACTION)` guard here.

I believe the issue was that these values, intended to be internal only, are indirectly exposed by using them as tags on NSMenuItem objects, so people relied on them matching the actual API or SPI values.

>>> Source/WebKit/Configurations/WebKit.xcconfig:135
>>> +WK_QUARTZ_LDFLAGS_MACOS_SINCE_1200 = -framework Quartz;
>> 
>> Same question?
> 
> Using the (same) simple test app, it looks like the web process and UI process both indirectly pull in Quartz.framework on shipping WebKit — however, the networking process currently doesn't pull in Quartz, so this linking could potentially regress the startup type in the networking process.
> 
> I'll change this so that we softlink Quartz instead.

I believe that in the past people told me this was not an issue for frameworks "in the shared cache" but not 100% sure about that. Hope you didn’t spend too much effort on soft linking based on a shared misunderstanding you and I might have had.
Comment 8 Wenson Hsieh 2021-02-16 09:40:18 PST
> > 
> > Using the (same) simple test app, it looks like the web process and UI process both indirectly pull in Quartz.framework on shipping WebKit — however, the networking process currently doesn't pull in Quartz, so this linking could potentially regress the startup type in the networking process.
> > 
> > I'll change this so that we softlink Quartz instead.
> 
> I believe that in the past people told me this was not an issue for
> frameworks "in the shared cache" but not 100% sure about that. Hope you
> didn’t spend too much effort on soft linking based on a shared
> misunderstanding you and I might have had.

Ah, so it does look like Quartz.framework is in the DYLD shared cache on macOS — at least, according to `dyld_shared_cache_util -list | grep "Quartz.framework"`.

That said, soft-linking QuickLookUI (in Quartz) might still not be a bad idea, given that it makes little sense for, say, the WebKit networking process to pull in Quartz (or any of its subframeworks).
Comment 9 Wenson Hsieh 2021-02-16 11:01:33 PST
(In reply to Wenson Hsieh from comment #8)
> > > 
> > > Using the (same) simple test app, it looks like the web process and UI process both indirectly pull in Quartz.framework on shipping WebKit — however, the networking process currently doesn't pull in Quartz, so this linking could potentially regress the startup type in the networking process.
> > > 
> > > I'll change this so that we softlink Quartz instead.
> > 
> > I believe that in the past people told me this was not an issue for
> > frameworks "in the shared cache" but not 100% sure about that. Hope you
> > didn’t spend too much effort on soft linking based on a shared
> > misunderstanding you and I might have had.
> 
> Ah, so it does look like Quartz.framework is in the DYLD shared cache on
> macOS — at least, according to `dyld_shared_cache_util -list | grep
> "Quartz.framework"`.
> 
> That said, soft-linking QuickLookUI (in Quartz) might still not be a bad
> idea, given that it makes little sense for, say, the WebKit networking
> process to pull in Quartz (or any of its subframeworks).

...Tim Horton just reminded me that QuickLookUI doesn't exist in the base system on macOS, so we'll need to softlink in any case.
Comment 10 Tim Horton 2021-02-16 19:29:19 PST
Comment on attachment 420423 [details]
Rebaseline test

View in context: https://bugs.webkit.org/attachment.cgi?id=420423&action=review

> Source/WebKit/ChangeLog:15
> +        Link against Quartz and UniformTypeIdentifiers.

LIES

> Source/WebKit/ChangeLog:29
> +        Add a non-const version of `webHitTestResultData()`, so that we can generate a `CGImage` using
> +        `WebHitTestResultData`'s bitmap data.

So confused why this needs non-const hit test result data
Comment 11 Wenson Hsieh 2021-02-16 20:04:40 PST
Comment on attachment 420423 [details]
Rebaseline test

View in context: https://bugs.webkit.org/attachment.cgi?id=420423&action=review

Thanks for the review!

>> Source/WebKit/ChangeLog:15
>> +        Link against Quartz and UniformTypeIdentifiers.
> 
> LIES

Whoops, fixed!

>> Source/WebKit/ChangeLog:29
>> +        `WebHitTestResultData`'s bitmap data.
> 
> So confused why this needs non-const hit test result data

This is because ShareableBitmap::makeCGImageCopy() is non-const, due to the context object passed into `CGBitmapContextCreateWithData` being a non-const pointer :/

We use the handler to release (and possible destroy) the bitmap, so perhaps the non-const is warranted? But it's also weird that a method that just copies an image is non-const — I think I'll look into this in a followup.
Comment 12 Wenson Hsieh 2021-02-16 22:47:45 PST
Created attachment 420599 [details]
For landing
Comment 13 Wenson Hsieh 2021-02-17 00:00:53 PST
Created attachment 420606 [details]
Rebase on trunk
Comment 14 EWS 2021-02-17 07:49:54 PST
Committed r272997: <https://commits.webkit.org/r272997>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 420606 [details].