Bug 214444 – Propagating user gesture through Fetch API

• Status: RESOLVED
• Resolution: FIXED
• Component: WebCore Misc.

[Jiewen Tan]

Propagating user gesture through Fetch API. Initially, we should only consider this for media only.

[Radar WebKit Bug Importer]

<rdar://problem/65980953>

[Jiewen Tan]

Created attachment 405027 [details]
Patch

[Jiewen Tan]

Created attachment 405028 [details]
Patch

[youenn fablet]

Comment on attachment 405028 [details]
Patch

LGTM overall.
This patch addresses most cases except in case the script is getting the body through ReadableStream.
That might be best done as a follow-up.
It should not be too difficult to add in case of ReadableStreamDefaultController::close and ReadableStreamDefaultController::error which are probably the most important cases.
It would require more work to add support for the chunk read promises, not sure whether this is really needed though.

View in context: https://bugs.webkit.org/attachment.cgi?id=405028&action=review

> Source/WebCore/ChangeLog:49
> +        (WebCore::UserGestureToken::setIsPropogatedFromFetch):

s/IsPropogatedFromFetch/IsPropagatedFromFetch

> Source/WebCore/Modules/fetch/DOMWindowFetch.cpp:61
> +    FetchResponse::fetch(*document, fetchRequest, [promise = WTFMove(promise), userGestureToken = UserGestureIndicator::currentUserGesture(), maximumIntervalForUserGestureForwarding = fetchRequest.maximumIntervalForUserGestureForwarding(), currentTime = MonotonicTime::now()] (ExceptionOr<FetchResponse&>&& result) mutable {

s/ExceptionOr<FetchResponse&>/auto/
Let's also keep []() style instead of [] ().

> Source/WebCore/Modules/fetch/DOMWindowFetch.cpp:67
> +        UserGestureIndicator gestureIndicator(userGestureToken, UserGestureToken::GestureScope::MediaOnly, UserGestureToken::IsPropogatedFromFetch::Yes);

To improve readability, we could rewrite it as:
Optional<UserGestureIndicator> gestureIndicator;
if (...)
   gestureIndicator = UserGestureIndicator(...);
promise.settle()

> Source/WebCore/Modules/fetch/FetchBodyConsumer.h:47
> +    FetchBodyConsumer(Type);

explicit would be good

> Source/WebCore/Modules/fetch/FetchBodyConsumer.h:77
> +    void setMaximumIntervalForUserGestureForwarding(double interval) { m_maximumIntervalForUserGestureForwarding = Seconds(interval); }

Let's make interval a Seconds.

> Source/WebCore/Modules/fetch/FetchRequest.h:86
> +    WEBCORE_EXPORT void setMaximumIntervalForUserGestureForwarding(double);

Ditto.

> Source/WebCore/Modules/fetch/FetchRequest.h:105
> +    Seconds m_maximumIntervalForUserGestureForwarding;

Do we need a per instance variable? It seems this could be a static member/per process setting.

> Source/WebCore/bindings/js/JSDOMWindowBase.cpp:221
> +    auto isProcessingUserGestureFromFetch = userGestureToken && userGestureToken->processingUserGestureForMedia() && userGestureToken->isPropogatedFromFetch();

s/auto/bool.
It does not seem we need the processingUserGestureForMedia check.
I would rename isPropogatedFromFetch to something like isPropagatedToNextMicrotask.
That will make it easier to reuse in case we want to propagate user gesture in getUserMedia resolution handler.

> Source/WebCore/bindings/js/JSDOMWindowBase.cpp:223
> +        if (!isProcessingUserGestureFromFetch || userGestureToken->hasExpired(UserGestureToken::maximumIntervalForUserGestureForwardingForFetch)) {

I am wondering whether the hasExpired check is needed.
A micro task is executed right after so if it is not expired at the time we initialise isProcessingUserGestureFromFetch, it seems ok to consider it is not expired when executing the micro task.
Given we are checking for hasExpired in fetch handler to create the user token that can persist next micro task, we might be able to simply remove the hasExpired check in this method.

> Source/WebCore/bindings/js/JSDOMWindowBase.cpp:228
> +        UserGestureIndicator gestureIndicator(userGestureToken, UserGestureToken::GestureScope::MediaOnly, UserGestureToken::IsPropogatedFromFetch::Yes);

We could rewrite it as: Optional<...> gestureIndicator to have just one callback->call().

Can we also try to rewrite it so that it is not specific to UserGestureToken::GestureScope::MediaOnly, but reuses whatever scope was used above?
We probably do not need to have IsPropogatedFromFetch::Yes set to Yes here.

> Source/WebCore/testing/Internals.cpp:5683
> +    request.setMaximumIntervalForUserGestureForwarding(interval);

Usually we do something like fetch(url, ...) so we will not be able to write tests this way with this API.
It might be simpler to just use a global member here.

[Jiewen Tan]

Comment on attachment 405028 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=405028&action=review

Thanks Youenn for reviewing the patch.

>> Source/WebCore/ChangeLog:49
>> +        (WebCore::UserGestureToken::setIsPropogatedFromFetch):
> 
> s/IsPropogatedFromFetch/IsPropagatedFromFetch

Fixed.

>> Source/WebCore/Modules/fetch/DOMWindowFetch.cpp:61
>> +    FetchResponse::fetch(*document, fetchRequest, [promise = WTFMove(promise), userGestureToken = UserGestureIndicator::currentUserGesture(), maximumIntervalForUserGestureForwarding = fetchRequest.maximumIntervalForUserGestureForwarding(), currentTime = MonotonicTime::now()] (ExceptionOr<FetchResponse&>&& result) mutable {
> 
> s/ExceptionOr<FetchResponse&>/auto/
> Let's also keep []() style instead of [] ().

Fixed.

>> Source/WebCore/Modules/fetch/DOMWindowFetch.cpp:67
>> +        UserGestureIndicator gestureIndicator(userGestureToken, UserGestureToken::GestureScope::MediaOnly, UserGestureToken::IsPropogatedFromFetch::Yes);
> 
> To improve readability, we could rewrite it as:
> Optional<UserGestureIndicator> gestureIndicator;
> if (...)
>    gestureIndicator = UserGestureIndicator(...);
> promise.settle()

Fixed.

> Source/WebCore/Modules/fetch/FetchBodyConsumer.cpp:228
> +    if (m_consumePromise) {

Used Optional as well.

>> Source/WebCore/Modules/fetch/FetchBodyConsumer.h:47
>> +    FetchBodyConsumer(Type);
> 
> explicit would be good

Fixed.

>> Source/WebCore/Modules/fetch/FetchBodyConsumer.h:77
>> +    void setMaximumIntervalForUserGestureForwarding(double interval) { m_maximumIntervalForUserGestureForwarding = Seconds(interval); }
> 
> Let's make interval a Seconds.

Fixed.

>> Source/WebCore/Modules/fetch/FetchRequest.h:86
>> +    WEBCORE_EXPORT void setMaximumIntervalForUserGestureForwarding(double);
> 
> Ditto.

Fixed.

>> Source/WebCore/Modules/fetch/FetchRequest.h:105
>> +    Seconds m_maximumIntervalForUserGestureForwarding;
> 
> Do we need a per instance variable? It seems this could be a static member/per process setting.

A good idea! Let me think about it.

>> Source/WebCore/bindings/js/JSDOMWindowBase.cpp:221
>> +    auto isProcessingUserGestureFromFetch = userGestureToken && userGestureToken->processingUserGestureForMedia() && userGestureToken->isPropogatedFromFetch();
> 
> s/auto/bool.
> It does not seem we need the processingUserGestureForMedia check.
> I would rename isPropogatedFromFetch to something like isPropagatedToNextMicrotask.
> That will make it easier to reuse in case we want to propagate user gesture in getUserMedia resolution handler.

Fixed.

>> Source/WebCore/bindings/js/JSDOMWindowBase.cpp:223
>> +        if (!isProcessingUserGestureFromFetch || userGestureToken->hasExpired(UserGestureToken::maximumIntervalForUserGestureForwardingForFetch)) {
> 
> I am wondering whether the hasExpired check is needed.
> A micro task is executed right after so if it is not expired at the time we initialise isProcessingUserGestureFromFetch, it seems ok to consider it is not expired when executing the micro task.
> Given we are checking for hasExpired in fetch handler to create the user token that can persist next micro task, we might be able to simply remove the hasExpired check in this method.

Probably not. I doubt that as well. Removed.

>> Source/WebCore/bindings/js/JSDOMWindowBase.cpp:228
>> +        UserGestureIndicator gestureIndicator(userGestureToken, UserGestureToken::GestureScope::MediaOnly, UserGestureToken::IsPropogatedFromFetch::Yes);
> 
> We could rewrite it as: Optional<...> gestureIndicator to have just one callback->call().
> 
> Can we also try to rewrite it so that it is not specific to UserGestureToken::GestureScope::MediaOnly, but reuses whatever scope was used above?
> We probably do not need to have IsPropogatedFromFetch::Yes set to Yes here.

Fixed.

>> Source/WebCore/testing/Internals.cpp:5683
>> +    request.setMaximumIntervalForUserGestureForwarding(interval);
> 
> Usually we do something like fetch(url, ...) so we will not be able to write tests this way with this API.
> It might be simpler to just use a global member here.

Let me think one.

[Jiewen Tan]

Comment on attachment 405028 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=405028&action=review

>>> Source/WebCore/Modules/fetch/DOMWindowFetch.cpp:67
>>> +        UserGestureIndicator gestureIndicator(userGestureToken, UserGestureToken::GestureScope::MediaOnly, UserGestureToken::IsPropogatedFromFetch::Yes);
>> 
>> To improve readability, we could rewrite it as:
>> Optional<UserGestureIndicator> gestureIndicator;
>> if (...)
>>    gestureIndicator = UserGestureIndicator(...);
>> promise.settle()
> 
> Fixed.

Looks like UserGestureIndicator is non copyable.

[Jiewen Tan]

Created attachment 405130 [details]
Patch

[Jiewen Tan]

Created attachment 405131 [details]
Patch

[youenn fablet]

Comment on attachment 405131 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=405131&action=review

> Source/WebCore/bindings/js/JSDOMWindowBase.cpp:218
> +    // Propogating media only user gesture for Fetch API's promise chain.

s/Propogating/Propagating

> Source/WebCore/bindings/js/JSDOMWindowBase.cpp:221
> +    bool isPropagatedToNextMicrotask = userGestureToken && userGestureToken->isPropagatedFromFetch();

We could do the following:
if (userGestureToken && !userGestureToken->isPropagatedFromFetch())
   userGestureToken = nullptr;
That way, no need to capture isPropagatedToNextMicrotask in the lambda.

> Source/WebCore/bindings/js/JSDOMWindowBase.cpp:228
> +        UserGestureIndicator gestureIndicator(userGestureToken, UserGestureToken::GestureScope::MediaOnly, UserGestureToken::IsPropagatedFromFetch::Yes);

We could potentially WTFMove(userGestureToken).
Can we add a scope getter to UserGestureToken and replace UserGestureToken::GestureScope::MediaOnly by userGesture->scope()?

> Source/WebCore/dom/UserGestureIndicator.cpp:58
> +void UserGestureToken::setMaximumIntervalForUserGestureForwardingForFetchForTesting(Seconds&& value)

s/Seconds&&/Seconds

> Source/WebCore/dom/UserGestureIndicator.cpp:60
> +    maxIntervalForUserGestureForwardingForFetch = WTFMove(value);

No need to move.

> Source/WebCore/dom/UserGestureIndicator.h:91
> +    // Expand the following methods if more progogation sources are added later.

s/progogation/propagation/

> Source/WebCore/dom/UserGestureIndicator.h:95
> +    bool isPropagatedFromFetch() const { return m_isPropagatedFromFetch == IsPropagatedFromFetch::Yes; }

I would rename IsPropagatedFromFetch to ShouldPersistInNextMicrotask, ditto for setter/getter/member.

> Source/WebCore/testing/Internals.cpp:5677
> +    UserGestureToken::setMaximumIntervalForUserGestureForwardingForFetchForTesting(Seconds(interval));

We probably need to set the value back to the regular value in Internals::Internals so that the next test reusing the process will have the default expected value.

> LayoutTests/http/tests/media/user-gesture-preserved-across-xmlhttprequest.html:50
> +            fetch(request, { method: 'GET', cache: "no-cache" }).then(response => {

Can we just do fetch((`/xmlhttprequest/resources/download-header-with-delay.php?delay=${delay}`, ( method...})?
This should be equivalent but it seems good to exercise this.

[Jiewen Tan]

Comment on attachment 405131 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=405131&action=review

Thanks Youenn for r+ this patch.

>> Source/WebCore/bindings/js/JSDOMWindowBase.cpp:218
>> +    // Propogating media only user gesture for Fetch API's promise chain.
> 
> s/Propogating/Propagating

Fixed.

>> Source/WebCore/bindings/js/JSDOMWindowBase.cpp:221
>> +    bool isPropagatedToNextMicrotask = userGestureToken && userGestureToken->isPropagatedFromFetch();
> 
> We could do the following:
> if (userGestureToken && !userGestureToken->isPropagatedFromFetch())
>    userGestureToken = nullptr;
> That way, no need to capture isPropagatedToNextMicrotask in the lambda.

Good idea. Let me do that!

>> Source/WebCore/bindings/js/JSDOMWindowBase.cpp:228
>> +        UserGestureIndicator gestureIndicator(userGestureToken, UserGestureToken::GestureScope::MediaOnly, UserGestureToken::IsPropagatedFromFetch::Yes);
> 
> We could potentially WTFMove(userGestureToken).
> Can we add a scope getter to UserGestureToken and replace UserGestureToken::GestureScope::MediaOnly by userGesture->scope()?

Let's add a rvalue constructor as a fellow up.
No, we can't the scope will be reset at the end of last runloop when the UserGestureIndicator is destroyed.

>> Source/WebCore/dom/UserGestureIndicator.cpp:58
>> +void UserGestureToken::setMaximumIntervalForUserGestureForwardingForFetchForTesting(Seconds&& value)
> 
> s/Seconds&&/Seconds

Fixed.

>> Source/WebCore/dom/UserGestureIndicator.cpp:60
>> +    maxIntervalForUserGestureForwardingForFetch = WTFMove(value);
> 
> No need to move.

Fixed.

>> Source/WebCore/dom/UserGestureIndicator.h:91
>> +    // Expand the following methods if more progogation sources are added later.
> 
> s/progogation/propagation/

Fixed.

>> Source/WebCore/dom/UserGestureIndicator.h:95
>> +    bool isPropagatedFromFetch() const { return m_isPropagatedFromFetch == IsPropagatedFromFetch::Yes; }
> 
> I would rename IsPropagatedFromFetch to ShouldPersistInNextMicrotask, ditto for setter/getter/member.

Since now this is the only api to would set this flag, it may be more appropriate to name it more specific. Also, the current patch is a quick fix to expand user gesture propagation to some important APIs. Ultimately, the way how user gesture is propagated here cannot be scaled. We can't set UserGestureIndicator for every async API. A better way of doing this for all could be User Activation API in the html spec. Doing that now for iOS 14 and macOS Big Sur is too risky though.

>> Source/WebCore/testing/Internals.cpp:5677
>> +    UserGestureToken::setMaximumIntervalForUserGestureForwardingForFetchForTesting(Seconds(interval));
> 
> We probably need to set the value back to the regular value in Internals::Internals so that the next test reusing the process will have the default expected value.

The static variable is per web process. I guess the value will not be carried to the next test given it will load a different page. For different test vectors within the same test file, I think that test file should ultimately control the values.

>> LayoutTests/http/tests/media/user-gesture-preserved-across-xmlhttprequest.html:50
>> +            fetch(request, { method: 'GET', cache: "no-cache" }).then(response => {
> 
> Can we just do fetch((`/xmlhttprequest/resources/download-header-with-delay.php?delay=${delay}`, ( method...})?
> This should be equivalent but it seems good to exercise this.

Yup, I was too lazy to change it.

[Jiewen Tan]

Created attachment 405169 [details]
Patch for landing

[EWS]

Committed r264853: <https://trac.webkit.org/changeset/264853>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 405169 [details].

[youenn fablet]

> No, we can't the scope will be reset at the end of last runloop when the
> UserGestureIndicator is destroyed.

Right, we should capture the scope in the lambda.

> >> Source/WebCore/dom/UserGestureIndicator.h:95
> >> +    bool isPropagatedFromFetch() const { return m_isPropagatedFromFetch == IsPropagatedFromFetch::Yes; }
> > 
> > I would rename IsPropagatedFromFetch to ShouldPersistInNextMicrotask, ditto for setter/getter/member.
> 
> Since now this is the only api to would set this flag, it may be more
> appropriate to name it more specific. Also, the current patch is a quick fix
> to expand user gesture propagation to some important APIs. Ultimately, the
> way how user gesture is propagated here cannot be scaled. We can't set
> UserGestureIndicator for every async API. A better way of doing this for all
> could be User Activation API in the html spec. Doing that now for iOS 14 and
> macOS Big Sur is too risky though.

But the point of IsPropagatedFromFetch flag is solely to know whether this should be used in the next microtask.
If we add another API that needs to be propagated to the next microtask, we can just reuse IsPropagatedFromFetch::Yes.

> >> Source/WebCore/testing/Internals.cpp:5677
> >> +    UserGestureToken::setMaximumIntervalForUserGestureForwardingForFetchForTesting(Seconds(interval));
> > 
> > We probably need to set the value back to the regular value in Internals::Internals so that the next test reusing the process will have the default expected value.
> 
> The static variable is per web process. I guess the value will not be
> carried to the next test given it will load a different page. For different
> test vectors within the same test file, I think that test file should
> ultimately control the values.

I do not think there is a guarantee that a new process will be used for the next test, WK1 for instance.
I think we should reset it in Internals::resetToConsistentState.

[Jiewen Tan]

(In reply to youenn fablet from comment #13)
> > No, we can't the scope will be reset at the end of last runloop when the
> > UserGestureIndicator is destroyed.
> 
> Right, we should capture the scope in the lambda.

Not sure what benefit we get by capturing the scope in the lambda, and use it for the new user gesture indicator since we want the new one to be media only for sure.

> 
> > >> Source/WebCore/dom/UserGestureIndicator.h:95
> > >> +    bool isPropagatedFromFetch() const { return m_isPropagatedFromFetch == IsPropagatedFromFetch::Yes; }
> > > 
> > > I would rename IsPropagatedFromFetch to ShouldPersistInNextMicrotask, ditto for setter/getter/member.
> > 
> > Since now this is the only api to would set this flag, it may be more
> > appropriate to name it more specific. Also, the current patch is a quick fix
> > to expand user gesture propagation to some important APIs. Ultimately, the
> > way how user gesture is propagated here cannot be scaled. We can't set
> > UserGestureIndicator for every async API. A better way of doing this for all
> > could be User Activation API in the html spec. Doing that now for iOS 14 and
> > macOS Big Sur is too risky though.
> 
> But the point of IsPropagatedFromFetch flag is solely to know whether this
> should be used in the next microtask.
> If we add another API that needs to be propagated to the next microtask, we
> can just reuse IsPropagatedFromFetch::Yes.

Got you. Will do a fellow up on Bug 214722.

> 
> > >> Source/WebCore/testing/Internals.cpp:5677
> > >> +    UserGestureToken::setMaximumIntervalForUserGestureForwardingForFetchForTesting(Seconds(interval));
> > > 
> > > We probably need to set the value back to the regular value in Internals::Internals so that the next test reusing the process will have the default expected value.
> > 
> > The static variable is per web process. I guess the value will not be
> > carried to the next test given it will load a different page. For different
> > test vectors within the same test file, I think that test file should
> > ultimately control the values.
> 
> I do not think there is a guarantee that a new process will be used for the
> next test, WK1 for instance.
> I think we should reset it in Internals::resetToConsistentState.

Got you. Will do a fellow up on Bug 214722.

[youenn fablet]

Comment on attachment 405131 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=405131&action=review

> Source/WebCore/Modules/fetch/FetchBodyConsumer.cpp:192
> +    m_userGestureToken = UserGestureIndicator::currentUserGesture();

I am wondering whether this is fine/sufficient?
This token may or may not be the same as the token captured for the fetch promise.
If writing the above, this will be fine since the token will be the same:
const response = await fetch('a');
const buffer = await response.arrayBuffer();

But if doing something like:
const response = await fetch('a');
setTimeout(async () => {
    const buffer = await response.arrayBuffer();
   // user gesture not propagated here, but would have been if we were getting the token of the fetch response since it would probably not be yet expired.
}, 0);

[Jiewen Tan]

Comment on attachment 405131 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=405131&action=review

>> Source/WebCore/Modules/fetch/FetchBodyConsumer.cpp:192
>> +    m_userGestureToken = UserGestureIndicator::currentUserGesture();
> 
> I am wondering whether this is fine/sufficient?
> This token may or may not be the same as the token captured for the fetch promise.
> If writing the above, this will be fine since the token will be the same:
> const response = await fetch('a');
> const buffer = await response.arrayBuffer();
> 
> But if doing something like:
> const response = await fetch('a');
> setTimeout(async () => {
>     const buffer = await response.arrayBuffer();
>    // user gesture not propagated here, but would have been if we were getting the token of the fetch response since it would probably not be yet expired.
> }, 0);

I think setTimeout should have propagated the media only user gesture?