Bug 157101 - All Books quirks are flakily not applied in modern WebKit (Add a preference to allow clients to turn on the "const" JS quirk)
Summary: All Books quirks are flakily not applied in modern WebKit (Add a preference t...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: JavaScriptCore (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Tim Horton
URL:
Keywords: InRadar
: 223628 (view as bug list)
Depends on:
Blocks:
 
Reported: 2016-04-27 14:08 PDT by Jon Lee
Modified: 2021-04-23 15:42 PDT (History)
11 users (show)

See Also:


Attachments
Patch (16.25 KB, patch)
2021-03-23 01:43 PDT, Tim Horton
no flags Details | Formatted Diff | Diff
Patch (34.23 KB, patch)
2021-03-23 13:46 PDT, Tim Horton
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch (34.30 KB, patch)
2021-03-23 14:46 PDT, Tim Horton
no flags Details | Formatted Diff | Diff
Patch (34.23 KB, patch)
2021-03-23 15:13 PDT, Tim Horton
no flags Details | Formatted Diff | Diff
Patch (34.10 KB, patch)
2021-03-24 02:45 PDT, Tim Horton
no flags Details | Formatted Diff | Diff
not for landing, just debugging (36.04 KB, patch)
2021-03-24 14:30 PDT, Tim Horton
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
not for landing, just debugging (35.31 KB, patch)
2021-03-24 14:37 PDT, Tim Horton
no flags Details | Formatted Diff | Diff
Patch (35.18 KB, patch)
2021-03-25 00:13 PDT, Tim Horton
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jon Lee 2016-04-27 14:08:42 PDT
Add a preference to allow clients to turn on the "const" JS quirk

rdar://problem/25965028
Comment 1 Jon Lee 2016-04-27 14:09:48 PDT
This should be coupled with removing the iBooks bundle check.
Comment 2 Tim Horton 2021-03-23 01:41:57 PDT
*** Bug 223628 has been marked as a duplicate of this bug. ***
Comment 3 Tim Horton 2021-03-23 01:43:14 PDT
Created attachment 423996 [details]
Patch
Comment 4 Simon Fraser (smfr) 2021-03-23 09:30:24 PDT
Comment on attachment 423996 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=423996&action=review

> Source/WebCore/platform/RuntimeApplicationChecks.h:50
> +bool isInAuxiliaryProcess();

I don't know what "auxiliary" means in this context.
Comment 5 Tim Horton 2021-03-23 10:13:19 PDT
(In reply to Simon Fraser (smfr) from comment #4)
> Comment on attachment 423996 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=423996&action=review
> 
> > Source/WebCore/platform/RuntimeApplicationChecks.h:50
> > +bool isInAuxiliaryProcess();
> 
> I don't know what "auxiliary" means in this context.

Well, it’s the name used throughout WebKit2 and even in this code already :) “WebKit subprocess”

Looks like the GPUP needs to get the hosting app plumbed to it, too.
Comment 6 Tim Horton 2021-03-23 13:46:30 PDT
Created attachment 424058 [details]
Patch
Comment 7 Chris Dumez 2021-03-23 14:01:48 PDT
Comment on attachment 424058 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=424058&action=review

> Source/WebKit/Shared/EntryPointUtilities/Cocoa/XPCService/XPCServiceEntryPoint.mm:88
> +    if (!ok)

return ok; :)

> Source/WebKit/WebProcess/WebProcess.cpp:488
> +    commonVM().setGlobalConstRedeclarationShouldThrow(parameters.shouldThrowExceptionForGlobalConstantRedeclaration);

Why don't we do the bundle check here? What's the benefit of having a configuration flag set specifically by that app and wire it all the way down to the WebProcess? The previous code seemed simpler.
Comment 8 Darin Adler 2021-03-23 14:09:51 PDT
Comment on attachment 424058 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=424058&action=review

> Source/WebKit/Shared/EntryPointUtilities/Cocoa/XPCService/XPCServiceEntryPoint.mm:82
> +    String clientIdentifierString = xpc_dictionary_get_string(m_initializerMessage, "client-sdk-version");

It’s not necessary to put a string into a WTF::String just to use WTF’s parsing. The charactersToUIntStrict has the same algorithm as String::toUIntStrict, but you can pass it 8-bit characters. Also, it handles null pointers. So this can be:

    auto string = xpc_dictionary_get_string(m_initializerMessage, "client-sdk-version");
    bool ok;
    clientSDKVersion = charactersToUIntStrict(string, string ? std::strlen(string) : 0, &ok);
    return ok;
Comment 9 Darin Adler 2021-03-23 14:10:20 PDT
Might need a reinterpret_cast.
Comment 10 Tim Horton 2021-03-23 14:46:34 PDT
Created attachment 424068 [details]
Patch
Comment 11 Tim Horton 2021-03-23 14:50:46 PDT
(In reply to Chris Dumez from comment #7)
> Comment on attachment 424058 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=424058&action=review
> 
> > Source/WebKit/Shared/EntryPointUtilities/Cocoa/XPCService/XPCServiceEntryPoint.mm:88
> > +    if (!ok)
> 
> return ok; :)

Good point, I stole this from the method below that needs to do one more thing :)

> > Source/WebKit/WebProcess/WebProcess.cpp:488
> > +    commonVM().setGlobalConstRedeclarationShouldThrow(parameters.shouldThrowExceptionForGlobalConstantRedeclaration);
> 
> Why don't we do the bundle check here? What's the benefit of having a
> configuration flag set specifically by that app and wire it all the way down
> to the WebProcess? The previous code seemed simpler.

My understanding is that it's generally considered incorrect to do a bundle check where SPI would do the trick (read: for apps that build alongside WebKit)

(In reply to Darin Adler from comment #8)
> Comment on attachment 424058 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=424058&action=review
> 
> > Source/WebKit/Shared/EntryPointUtilities/Cocoa/XPCService/XPCServiceEntryPoint.mm:82
> > +    String clientIdentifierString = xpc_dictionary_get_string(m_initializerMessage, "client-sdk-version");
> 
> It’s not necessary to put a string into a WTF::String just to use WTF’s
> parsing. The charactersToUIntStrict has the same algorithm as
> String::toUIntStrict, but you can pass it 8-bit characters. Also, it handles
> null pointers. So this can be:
> 
>     auto string = xpc_dictionary_get_string(m_initializerMessage,
> "client-sdk-version");
>     bool ok;
>     clientSDKVersion = charactersToUIntStrict(string, string ?
> std::strlen(string) : 0, &ok);
>     return ok;

Nice!
Comment 12 Tim Horton 2021-03-23 15:13:47 PDT
Created attachment 424073 [details]
Patch
Comment 13 Darin Adler 2021-03-23 15:22:03 PDT
Comment on attachment 424073 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=424073&action=review

> Source/WebKit/ChangeLog:28
> +        Add an assertion, and move the quirk to live as a setting on the process
> +        pool configuration, to be set via SPI by Books, instead of as a bundle check.

I know we have long wanted to use SPI or API for this quirk rather than a bundle ID check. I would slightly prefer API over SPI. Is this OK for iBooks? Do we need a bundle ID check to smooth over a transition period? If so, can we make one that works?

> Source/WebKit/UIProcess/Launcher/mac/ProcessLauncherMac.mm:212
> +    xpc_dictionary_set_string(bootstrapMessage.get(), "client-sdk-version", String::number(applicationSDKVersion()).utf8().data());
>      xpc_dictionary_set_string(bootstrapMessage.get(), "process-identifier", String::number(m_launchOptions.processIdentifier.toUInt64()).utf8().data());

Seems like we ought to make a CString::number function for cases like this.
Comment 14 Tim Horton 2021-03-23 15:36:53 PDT
(In reply to Darin Adler from comment #13)
> Comment on attachment 424073 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=424073&action=review
> 
> > Source/WebKit/ChangeLog:28
> > +        Add an assertion, and move the quirk to live as a setting on the process
> > +        pool configuration, to be set via SPI by Books, instead of as a bundle check.
> 
> I know we have long wanted to use SPI or API for this quirk rather than a
> bundle ID check.

At some point, I'd like to get rid of all of the com.apple.* bundle checks, but this one takes precedence because it's breaking things :)

> I would slightly prefer API over SPI.

I'm pretty sure there is zero precedent for making a nitpicky JavaScript behavior configuration like this API. If we want to change course for the project as a whole, that would need a larger conversation.

> Is this OK for iBooks? Do we need a bundle ID check to smooth over a transition period? If
> so, can we make one that works?

We do not need a temporary check, because the current check is already not working :) I checked with Books folks.
Comment 15 Darin Adler 2021-03-23 15:43:32 PDT
(In reply to Tim Horton from comment #14)
> > I would slightly prefer API over SPI.
>
> I'm pretty sure there is zero precedent for making a nitpicky JavaScript
> behavior configuration like this API. If we want to change course for the
> project as a whole, that would need a larger conversation.

Ideally long term the burden of argument goes the other way for future WebKit. I’d like us to think in terms of API, not SPI, whenever there’s a real requirement. Even a small one. Minimizing SPI.
Comment 16 Maciej Stachowiak 2021-03-23 22:49:38 PDT
An SPI is certainly an upgrade to a bundle check.

If the "const" quirk is for compatibility with some existing books, then other EPUB readers using WebKit might want the same quirk. If it's for something truly specific to only the iBooks app (like script they inject that they didn't want to rewrite it), then maybe not.

In any case, it seems reasonable to land this without having resolved the question.
Comment 17 Tim Horton 2021-03-24 02:27:40 PDT
I have been entirely unable to reproduce the debug bot's Webauthn failures (even with a macOS debug WK2 build on Big Sur, exactly like the bot), but they really do seem unique to this patch.
Comment 18 Tim Horton 2021-03-24 02:45:37 PDT
Created attachment 424115 [details]
Patch
Comment 19 Darin Adler 2021-03-24 10:22:56 PDT
(In reply to Maciej Stachowiak from comment #16)
> In any case, it seems reasonable to land this without having resolved the
> question.

Absolutely.

I said "long term" to try to make it clear that I wasn’t talking about "in this patch", but maybe that wasn’t obvious enough.
Comment 20 Tim Horton 2021-03-24 12:21:52 PDT
I'm about ready to blow past the bot, even if only to have it fail on the post-commit bots and collect more information about the failing configuration, since I am coming up empty with ideas or repro. But let me post one patch to try to catch it first...
Comment 21 Tim Horton 2021-03-24 14:30:50 PDT
Created attachment 424184 [details]
not for landing, just debugging
Comment 22 Tim Horton 2021-03-24 14:37:31 PDT
Created attachment 424185 [details]
not for landing, just debugging
Comment 23 Tim Horton 2021-03-24 19:49:50 PDT
OK, I think the latest test results reveal the problem; in WKTR, we're getting the bundle identifier "WebKitTestRunner" instead of "com.apple.WebKit.WebKitTestRunner" for WKTR

Because _NSGetProgName (which feeds client-identifier, which is what I used in this patch) ≠ [[NSBundle mainBundle] bundleIdentifier] (which is what we used to use) in this case.

(I don't understand why it works locally, but this makes sense).

In the interests of not going crazy I'm going to do one of two things, whichever works:

1) Use XPC API to get the bundle identifier of the UI process instead of shipping it across.

or

2) Send [[NSBundle mainBundle] bundleIdentifier] separately in the bootstrap message.

I don't think it's a good idea to change client-identifier, since it seems to be used for a few things downstream.
Comment 24 Tim Horton 2021-03-24 20:02:44 PDT
Nope, still wrong. clientIdentifier comes from [[NSBundle mainBundle] bundleIdentifier]; only falls back to _NSGetProgname() if it's empty.
Comment 25 Tim Horton 2021-03-24 20:08:53 PDT
OK, we need to plumb it in parallel, and read *from* WebCore::applicationBundleIdentifier(), because of the WebCore::setApplicationBundleIdentifierOverride() that WKTR uses.
Comment 26 Tim Horton 2021-03-25 00:13:25 PDT
Created attachment 424219 [details]
Patch
Comment 27 Tim Horton 2021-03-25 02:45:59 PDT
Comment on attachment 424219 [details]
Patch

Phew! The remaining failures on the debug bot are intermittently happening for other people's patches.
Comment 28 EWS 2021-03-25 03:14:49 PDT
Committed r275013: <https://commits.webkit.org/r275013>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 424219 [details].
Comment 29 Alex Christensen 2021-04-23 15:42:34 PDT
Comment on attachment 424219 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=424219&action=review

> Source/WebKit/Shared/EntryPointUtilities/Cocoa/XPCService/XPCServiceEntryPoint.h:120
> +    if (!delegate.getClientBundleIdentifier(parameters.clientBundleIdentifier))
> +        exit(EXIT_FAILURE);

This makes apps with the empty string as the bundle identifier unable to launch a WKWebView.  I accidentally made such an app and it looked like the web process was in a crash loop.  Might be a compatibility issue, might not.