Bug 189952 – Intelligent Tracking Prevention preventing BBC.co.uk sign in

• Component: New Bugs
• Status: RESOLVED CONFIGURATION CHANGED

Created attachment 350755 [details]
Blank cookies

Hi,

This bug is following up on this discussion with John Willander on Twitter: https://twitter.com/Marc_Burrows/status/1043049662603841538

**Overview**
Since the release of Safari 12 we have had lots of feedback from users that can no longer access the parts of the BBC website that require sign in. This is affecting by no means all users of Safari 12, and we are unable to recreate it in house.

**Manifestation**
We have seen a few machines with the issue, and it manifests itself in the following manner:
- User has all the correct cookies set on bbc.co.uk
- Three cookies, which are set whilst on session.bbc.co.uk for bbc.co.uk, but are blank and are now “session” cookies. I believe this is what is meant by the cookies having been “partitioned”. See screenshot.
- Cookies on account.bbc.com and session.bbc.com seem to be set correctly

**Known Fixes**
- Clear entire history
- Go to account.bbc.com/account, quit Safari, open Safari, (maybe click sign in, can’t be sure) user appears signed in on bbc.co.uk

**FLOWS**

*Sign In*
Our flow to get the majority of the users in the UK signed in is as follows:
- User starts on https://www.bbc.co.uk
- Clicks "Sign In"
- User GETs to https://session.bbc.co.uk/session - no relevant cookies being set
- User 302 redirects to https://account.bbc.com/authorise - no cookies being set
- User 302 redirects to https://account.bbc.com/authoriseLogin - no relevant cookies being set
- User 302 redirects to https://account.bbc.com/signin - no relevant cookies being set
- User enters email & password, and clicks submit
- User POSTs to https://account.bbc.com/signin - ckns_session, ckns_jwt being set on .account.bbc.com
- User 302 redirects to https://account.bbc.com/authorise - no cookies being set
- User 302 redirects to https://session.bbc.co.uk/session/callback - ckns_rtkn being set on .session.bbc.co.uk, and ckns_idtkn (SECURE), ckns_atkn (SECURE), ckns_stateless (NOT SECURE), ckns_id (NOT SECURE), ckpf_sylphid (NOT SECURE) being set on .bbc.co.uk
- User 302 redirects back on https://www.bbc.co.uk and is signed in

*Normal Token Refresh when everything works*
- User starts on https://www.bbc.co.uk and needs a token refresh
- In iFrame - User goes to https://session.bbc.co.uk/session - no cookies being set
- iFrame redirected to https://account.bbc.com - This checks if a cookie is set, but no cookies are being set
- iFrame redirected back to https://session.bbc.co.uk/session - ckns_stateless, ckns_idtkn, ckns_atkn, ckns_id being set on .bbc.co.uk

*Stateful Token Refresh (Which is what we believe people are seeing because their cookies are blank)*
- User starts on https://www.bbc.co.uk and clicks sign in
- User GETs to https://session.bbc.co.uk/session - no relevant cookies being set
- User 302 redirects to https://account.bbc.com/authorise - no cookies being set
- User 302 redirects to https://session.bbc.co.uk/session/callback - no cookies being set
- User 302 redirects to https://session.bbc.com/session/affinity - no cookies being set
- User 302 redirects to https://session.bbc.co.uk/session/callback - ckns_rtkn being set on .session.bbc.co.uk, and ckns_idtkn (SECURE), ckns_atkn (SECURE), ckns_stateless (NOT SECURE), ckns_id (NOT SECURE), ckpf_sylphid (NOT SECURE) being set on .bbc.co.uk
- User 302 redirects back on https://www.bbc.co.uk but does *NOT* appear signed in as ckns_stateless, ckpf_sylphid and most importantly ckns_id are blank and are session cookies.

**Questions**
- Are we correct in that the three cookies on bbc.co.uk are being partitioned?
- What we don’t understand is why the three cookies on bbc.co.uk are blank (so are therefore set as tracking cookies?), because users are always interacting with pages on bbc.co.uk (within the UK). Is this due ITP? 
    - Users are unlikely to regularly interact with pages on bbc.com, unless they wish to update something with their account.
- Unless we are misunderstood, we don’t believe that the storage access API will resolve our issues, because this has nothing to do with embedding content from a different domain on our pages.
- Let us know if you need more info on anything.

[Radar WebKit Bug Importer]

<rdar://problem/44771679>

[John Wilander]

(In reply to marc.burrows from comment #0)
> Created attachment 350755 [details]
> Blank cookies
> 
> Hi,

Hi Marc, and thanks for your report! I have a couple of questions.

First, have you tested the flow and repro steps in Safari Technology Preview with ITP Debug Mode? That should tell you whether any of your domains are getting classified by ITP as having cross-site tracking abilities.

Documentation for ITP Debug Mode can be found here: https://webkit.org/blog/8387/itp-debug-mode-in-safari-technology-preview-62/

> This bug is following up on this discussion with John Willander on Twitter:
> https://twitter.com/Marc_Burrows/status/1043049662603841538
> 
> **Overview**
> Since the release of Safari 12 we have had lots of feedback from users that
> can no longer access the parts of the BBC website that require sign in. This
> is affecting by no means all users of Safari 12, and we are unable to
> recreate it in house.

Does this mean the steps below are not reproducing the issue on your machines/devices, only on your users'? If so, please leverage the ability to manually classify domains through ITP Debug Mode and see if the issue reproduces then.

> **Manifestation**
> We have seen a few machines with the issue, and it manifests itself in the
> following manner:
> - User has all the correct cookies set on bbc.co.uk
> - Three cookies, which are set whilst on session.bbc.co.uk for bbc.co.uk,
> but are blank and are now “session” cookies. I believe this is what is meant
> by the cookies having been “partitioned”. See screenshot.
> - Cookies on account.bbc.com and session.bbc.com seem to be set correctly
> 
> **Known Fixes**
> - Clear entire history

This resets ITP, i.e. no domains are classified as having cross-site tracking abilities anymore. From this point, ITP starts to learn again and will soon re-classify based on the user's browsing.

> - Go to account.bbc.com/account, quit Safari, open Safari, (maybe click sign
> in, can’t be sure) user appears signed in on bbc.co.uk
> 
> **FLOWS**
> 
> *Sign In*
> Our flow to get the majority of the users in the UK signed in is as follows:
> - User starts on https://www.bbc.co.uk
> - Clicks "Sign In"
> - User GETs to https://session.bbc.co.uk/session - no relevant cookies being
> set
> - User 302 redirects to https://account.bbc.com/authorise - no cookies being
> set
> - User 302 redirects to https://account.bbc.com/authoriseLogin - no relevant
> cookies being set
> - User 302 redirects to https://account.bbc.com/signin - no relevant cookies
> being set
> - User enters email & password, and clicks submit
> - User POSTs to https://account.bbc.com/signin - ckns_session, ckns_jwt
> being set on .account.bbc.com
> - User 302 redirects to https://account.bbc.com/authorise - no cookies being
> set
> - User 302 redirects to https://session.bbc.co.uk/session/callback -
> ckns_rtkn being set on .session.bbc.co.uk, and ckns_idtkn (SECURE),
> ckns_atkn (SECURE), ckns_stateless (NOT SECURE), ckns_id (NOT SECURE),
> ckpf_sylphid (NOT SECURE) being set on .bbc.co.uk
> - User 302 redirects back on https://www.bbc.co.uk and is signed in
> 
> *Normal Token Refresh when everything works*
> - User starts on https://www.bbc.co.uk and needs a token refresh
> - In iFrame - User goes to https://session.bbc.co.uk/session - no cookies
> being set
> - iFrame redirected to https://account.bbc.com - This checks if a cookie is
> set, but no cookies are being set
> - iFrame redirected back to https://session.bbc.co.uk/session -
> ckns_stateless, ckns_idtkn, ckns_atkn, ckns_id being set on .bbc.co.uk
> 
> *Stateful Token Refresh (Which is what we believe people are seeing because
> their cookies are blank)*
> - User starts on https://www.bbc.co.uk and clicks sign in
> - User GETs to https://session.bbc.co.uk/session - no relevant cookies being
> set
> - User 302 redirects to https://account.bbc.com/authorise - no cookies being
> set
> - User 302 redirects to https://session.bbc.co.uk/session/callback - no
> cookies being set
> - User 302 redirects to https://session.bbc.com/session/affinity - no
> cookies being set
> - User 302 redirects to https://session.bbc.co.uk/session/callback -
> ckns_rtkn being set on .session.bbc.co.uk, and ckns_idtkn (SECURE),
> ckns_atkn (SECURE), ckns_stateless (NOT SECURE), ckns_id (NOT SECURE),
> ckpf_sylphid (NOT SECURE) being set on .bbc.co.uk
> - User 302 redirects back on https://www.bbc.co.uk but does *NOT* appear
> signed in as ckns_stateless, ckpf_sylphid and most importantly ckns_id are
> blank and are session cookies.

For all of the steps and redirects you describe, it's always useful for us to know whether you are referring to the top frame or a sub frame. I think what you're saying with "User 302 redirects to" is that it's a navigational redirect in the top frame, i.e. the URL bar changes, but I don't know.

> **Questions**
> - Are we correct in that the three cookies on bbc.co.uk are being
> partitioned?

We can't tell without repro steps. But if a) the user has interacted with a webpage from bbc.com the last 30 days of Safari use and b) bbc.com has been classified as having cross-site tracking abilities, then bbc.com cookies will be partitioned (double-keyed) when bbc.com is third-party under some non-bbc.com webpage. And partitioned cookies automatically become session cookies.

> - What we don’t understand is why the three cookies on bbc.co.uk are blank
> (so are therefore set as tracking cookies?), because users are always
> interacting with pages on bbc.co.uk (within the UK). Is this due ITP?

I see your screenshot from the Web Inspector. However, what does your server see? Are these cookies seen on requests?

>     - Users are unlikely to regularly interact with pages on bbc.com, unless
> they wish to update something with their account.

If bbc.com is classified as having cross-site tracking abilities and the user doesn't interact with a webpage from bbc.com for 30 days of Safari use, all bbc.com website data, including cookies, is deleted.

> - Unless we are misunderstood, we don’t believe that the storage access API
> will resolve our issues, because this has nothing to do with embedding
> content from a different domain on our pages.

I do see you refer to an iframe from bbc.com under bbc.co.uk—"iFrame redirected to https://account.bbc.com"—which constitutes the exact scenario for when the Storage Access API can be called.

> - Let us know if you need more info on anything.

Please see if you can reproduce with ITP Debug Mode and manual setting of either bbc.com or bbc.co.uk as classified by ITP.

Hi John,

Thanks for the quick response. 

> First, have you tested the flow and repro steps in Safari Technology Preview with ITP Debug Mode?
> That should tell you whether any of your domains are getting classified by ITP as having cross-
> site tracking abilities.

Yes we've tried this out, but after quite a long time of browsing we can't recreate the issue that users are seeing. We believe this issue is only happening to around 5% of Safari 12 users (although that is still potentially of thousands of people!). See below for the details.

> For all of the steps and redirects you describe, it's always useful for us to know whether you are
> referring to the top frame or a sub frame. I think what you're saying with "User 302 redirects to"
> is that it's a navigational redirect in the top frame, i.e. the URL bar changes, but I don't know.

Correct - this will happen in the top frame.

> We can't tell without repro steps. But if a) the user has interacted with a webpage from bbc.com
> the last 30 days of Safari use and b) bbc.com has been classified as having cross-site tracking
> abilities, then bbc.com cookies will be partitioned (double-keyed) when bbc.com is third-party
> under some non-bbc.com webpage. And partitioned cookies automatically become session cookies.

The issue here is that on the two machines we have seen the issue (which unfortunately resolved themselves using the two known fixes), we didn't see any of the cookies on bbc.com being purged at all. If they go to any page on bbc.com, then they appeared signed in. However, on bbc.co.uk they do not, as the the main cookie which we use to see if they are signed in - ckns_id - is blank. This is despite bbc.co.uk being the domain that users will interact with the most. This is the part which doesn't seem to make sense to us!

> We can't tell without repro steps. But if a) the user has interacted with a webpage from bbc.com
> the last 30 days of Safari use and b) bbc.com has been classified as having cross-site tracking
> abilities, then bbc.com cookies will be partitioned (double-keyed) when bbc.com is third-party
> under some non-bbc.com webpage. And partitioned cookies automatically become session cookies.

As per above, if the cookies on bbc.com were being partitioned, then it would make a bit more sense to us, but for cookies on bbc.co.uk to be partitioned seems strange.

> I see your screenshot from the Web Inspector. However, what does your server see? Are these
> cookies seen on requests?

The cookies are seen on the server requests, but are blank. We have implemented some logging to see when people come to us with a blank ckns_id as we extract its value in several places throughout our code. This is how we can see how many people are affected by the issue.

> I do see you refer to an iframe from bbc.com under bbc.co.uk—"iFrame redirected to
> https://account.bbc.com"—which constitutes the exact scenario for when the Storage Access API can
> be called.

So this redirect to account.bbc.com can happen in an iFrame, but it doesn't always do so. The redirect is simply checking if a cookie exists or not on account.bbc.com, and never sets anything. During this particular flow, cookies are only ever set on bbc.co.uk or session.bbc.co.uk, as this is the domain that the user is interacting with. Maybe I've misunderstood, but requesting storage access for the domain that the user is already on (in this case bbc.co.uk) doesn't seem like it should be necessary?

> Please see if you can reproduce with ITP Debug Mode and manual setting of either bbc.com or bbc.co.uk as classified by ITP.

When setting only bbc.com as classified by ITP on a brand new web session (history has been cleared):
- I can sign in and have no issues at all.
- However, when I land and then click anywhere on the account.bbc.com/signin page (to sign in), the debugger does say "About to block cookies in third-party contexts for: bbc.co.uk." and then "Done updating cookie blocking". But this doesn't seem to prevent any cookies that we expect to be set from being set properly (on either .com or .co.uk).  
- I also removed some of the cookies on .co.uk, refreshed the page so the browser thinks I'm signed out, and force a token refresh by clicking "sign in", and it will eventually bring me back to where I was, and show me as signed in without having to enter my credentials at any point - which is correct.

When setting only bbc.co.uk as classified by ITP on a brand new web session:
- Exactly the same behaviour occurs as when blocking bbc.com apart from the other way round with the debugger saying "About to block cookies in third-party contexts for: bbc.com" when interacting with a bbc.co.uk page post sign in. Prior to sign in I browsed numerous pages and this never happened.
- However, I still remain signed in at all times, and the cookies on either domain are never purged or modified to be session cookies and blank.

When not setting any domains as classified by ITP, we are yet to see either bbc.com or bbc.co.uk ever appear in the debugger as "about to be blocked".

Thank you for your help so far!

(In reply to John Wilander from comment #2)

Hi John,

I don't suppose you have had any more thoughts on the above? We are still unable to recreate the issue that some of our users are seeing.

To summarise my questions:
- Why would cookies on bbc.co.uk get made blank if this is the site that is being interacted with?
- Can you confirm what a partitioned cookie looks like? Is it set, but with no value, and as a session cookie?
- The Storage Access API can only really be used when embedding content and trying to set cookies on a page from a different domain (e.g bbc.com), in an iFrame. It shouldn't be necessary to use it when setting cookies on the same domain?
- If I use debug mode, then would it still be possible for the browser to set cookies on a domain even if it is classified? Or will it only happen if I don't interact with the domain?
- Why would some users continuously get the issue again soon after clearing their history, whereas other users (like myself) never see the issue even after days of only browsing on bbc.co.uk pages?

Thanks

Marc

(In reply to John Wilander from comment #2)

Hi again John,

Soon after my last message I think we worked out what is happening. And it seems like it is related to the iFrame as you suggested. I forgot this part of our flows!

*We have this flow:*
- User arrives on a bbc.co.uk page and is in need of a token refresh
- The token refresh is done in an iFrame on a bbc.co.uk page, but in order to make sure that they are a valid user we do a redirect (in the iFrame) and make sure that a cookie exists on account.bbc.com
- If that cookie doesn't exist, then we sign the user out
- This redirects the user (in the iFrame) to a sign out end point on session.bbc.co.uk and updates the various cookies we use to check if the user is signed in, to a date in the past (to expire them)
- The user is then redirected (in the iFrame) to the same sign out end point, but this time on session.bbc.com, the cookies on this domain which do the same as the above are then updated to a date in the past as well (to expire them).

*Questions on this flow:*
- Does it seem like this flow would be affected by ITP because we are updating the cookies from a different domain, from within an iFrame?

- It looks like most of the cookies on bbc.co.uk and session.bbc.co.uk that we expire in this flow are blank and session cookies - hence we assume they are partitioned (please confirm). Does this sound like something ITP would do?

- If we are trying to expire cookies on bbc.com through an iFrame on a bbc.co.uk page, then should ITP:
a) Partition the cookies on bbc.com
b) Just not update (expire) the cookies
c) Purge the cookies

- If we were to implement the Storage Access API do we need to request user interaction with bbc.com? In the docs here - https://webkit.org/blog/8124/introducing-storage-access-api/ - we can see that it says:
"The iframe needs to be processing a user gesture at the time of the API call."

But we do a background client side token refresh in the iFrame and it has no user interaction. Is there a suggested solution for this in our situation, because it seems like the examples require a user interaction (be it with a button or something else) and this is done in the background?

Thanks

Marc

[John Wilander]

(In reply to marc.burrows from comment #5)
> (In reply to John Wilander from comment #2)
> 
> Hi again John,
> 
> Soon after my last message I think we worked out what is happening. And it
> seems like it is related to the iFrame as you suggested. I forgot this part
> of our flows!

Good to hear that we're closer to understanding your users' issues.

> *We have this flow:*
> - User arrives on a bbc.co.uk page and is in need of a token refresh
> - The token refresh is done in an iFrame on a bbc.co.uk page, but in order
> to make sure that they are a valid user we do a redirect (in the iFrame) and
> make sure that a cookie exists on account.bbc.com
> - If that cookie doesn't exist, then we sign the user out
> - This redirects the user (in the iFrame) to a sign out end point on
> session.bbc.co.uk and updates the various cookies we use to check if the
> user is signed in, to a date in the past (to expire them)
> - The user is then redirected (in the iFrame) to the same sign out end
> point, but this time on session.bbc.com, the cookies on this domain which do
> the same as the above are then updated to a date in the past as well (to
> expire them).

Since bbc.com might be classified as having tracking abilities at this point, but the user *has* interacted with a webpage from bbc.com the last 30 days of Safari use, bbc.com's cookies in the iframe will be partitioned. If, as part of the sign-out redirect in the iframe, bbc.com sets any cookies, they will be in the partitioned state and only seen under first-party pages from bbc.co.uk.

Now, if you test this in recent versions of Safari Technology Preview, you will see a different behavior. We've done away with partitioned cookies all together, partly because of developers facing issues like what you're describing. Handling multiple cookie jars is hard. So with recent Safari Technology Preview you either have access to your regular cookies or you don't. This might make it hard for you to reproduce in Safari Technology Preview.

> *Questions on this flow:*
> - Does it seem like this flow would be affected by ITP because we are
> updating the cookies from a different domain, from within an iFrame?

Yes, as described above.

> - It looks like most of the cookies on bbc.co.uk and session.bbc.co.uk that
> we expire in this flow are blank and session cookies - hence we assume they
> are partitioned (please confirm). Does this sound like something ITP would
> do?

If bbc.co.uk is first-party in this context, cookies should not be partitioned.

> - If we are trying to expire cookies on bbc.com through an iFrame on a
> bbc.co.uk page, then should ITP:
> a) Partition the cookies on bbc.com
> b) Just not update (expire) the cookies
> c) Purge the cookies

None of the above. If bbc.com is classified, it cannot get to its cookies in an iframe under bbc.co.uk. Not even to delete them.


> - If we were to implement the Storage Access API do we need to request user
> interaction with bbc.com? In the docs here -
> https://webkit.org/blog/8124/introducing-storage-access-api/ - we can see
> that it says:
> "The iframe needs to be processing a user gesture at the time of the API
> call."

First, we're assuming that bbc.com has been classified by ITP.

Now, for bbc.com to get access to its cookies under bbc.co.uk, it has to call the Storage Access API. A successful call to the Storage Access API requires a user gesture in the bbc.com iframe under bbc.co.uk. Something like a login button. In the event handler of that tap or click, you can call the API.

Looking big picture, it might serve you better to do OAuth login from bbc.com to bbc.co.uk.

> But we do a background client side token refresh in the iFrame and it has no
> user interaction. Is there a suggested solution for this in our situation,
> because it seems like the examples require a user interaction (be it with a
> button or something else) and this is done in the background?

Because ITP is designed to prevent passive tracking, there is no way to passively or invisibly get access to cookies if the domain is classified and the resource is third-party. (This was already the case in ITP 1.0, release about a year ago.)

[Karl Dubost]

This can not be reproduced anymore.