Bug 111650 – REGRESSION (r143931): set-cookie-on-redirect.html still breaking subsequent tests

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Dean Jackson]

After https://bugs.webkit.org/show_bug.cgi?id=110844 we're still seeing flakiness on
http/tests/cookies/third-party-cookie-relaxing.html
http/tests/plugins/third-party-cookie-accept-policy.html

e.g. http://build.webkit.org/results/Apple%20MountainLion%20Debug%20WK1%20(Tests)/r144995%20(6142)/results.html

I'm going to skip them for now because our bot is backed up. Sergio, could you remove the skip if you can fix this? Thanks.

[Dean Jackson]

Flakiness added https://trac.webkit.org/r145010

[Alexey Proskuryakov]

Dean, did you confirm that it's set-cookie-on-redirect.html? I was also seeing this flakiness a few days ago locally, but it seemed to me that this was some other test, possibly from another directory. I did not have an opportunity to investigate.

I don't know why this would affect bot performance though. The failing tests are re-run, but that should be orders of magnitude faster than running all tests. Or is it something in EWS logic that makes us re-run all the tests?

[Sergio Villar Senin]

(In reply to comment #2)
> Dean, did you confirm that it's set-cookie-on-redirect.html? I was also seeing this flakiness a few days ago locally, but it seemed to me that this was some other test, possibly from another directory. I did not have an opportunity to investigate.
> 
> I don't know why this would affect bot performance though. The failing tests are re-run, but that should be orders of magnitude faster than running all tests. Or is it something in EWS logic that makes us re-run all the tests?

As Alexey mentions first of all we must ensure that the new test is causing the flakiness. Looks like the cookies' testing infrastructure easily leads to flaky results.

In any case, if you find that the test I added is indeed the culprit I'll be happy to help.

BTW I have just remembered that the relaxing test did not have the typical cleanAllCookies() at the beginning, so any test leaving a cookie in the jar by mistake could be compromising its behavior.

[Alexey Proskuryakov]

Dean, did the change help? Note that one of the paths you added is non-existent, there is no http/tests/cookies/third-party-cookie-accept-policy.html test.

[Alexey Proskuryakov]

This is actually the case, and the underlying reason is a lower level issue, <rdar://problem/10080130>.

I'm looking into whether I can devise a workaround.

[Alexey Proskuryakov]

Created attachment 209294 [details]
proposed fix

[WebKit Commit Bot]

Comment on attachment 209294 [details]
proposed fix

Clearing flags on attachment: 209294

Committed r154410: <http://trac.webkit.org/changeset/154410>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.