Bug 64001 – REGRESSION(r90212-r90246?): http/tests/cookies/third-party-cookie-relaxing.html fails on Snow Leopard

• Status: RESOLVED
• Resolution: CONFIGURATION CHANGED
• Component: Tools / Tests

[Eric Seidel (no email)]

http/tests/cookies/third-party-cookie-relaxing.html is failing under new-run-webkit-tests on the snow leopard bot

[Eric Seidel (no email)]

Example failure:
http://build.webkit.org/results/SnowLeopard%20Intel%20Release%20(Tests)/r90460%20(31134)/http/tests/cookies/third-party-cookie-relaxing-pretty-diff.html

--- /Volumes/Data/WebKit-BuildSlave/snowleopard-intel-release-tests/build/layout-test-results/http/tests/cookies/third-party-cookie-relaxing-expected.txt 
+++ /Volumes/Data/WebKit-BuildSlave/snowleopard-intel-release-tests/build/layout-test-results/http/tests/cookies/third-party-cookie-relaxing-actual.txt 
@@ -26,7 +26,7 @@
 ALERT: Test stage 5 document.cookie is: 
 ALERT: http://localhost:8000/cookies/resources/cookie-utility.php?queryfunction=setFooCookie
 ALERT: XHR response - Set the foo cookie
-ALERT: Test stage 6 document.cookie is: foo=awesomevalue
+ALERT: Test stage 6 document.cookie is: 
 ALERT: Restricting to first party only cookies
 ALERT: http://localhost:8000/cookies/resources/cookie-utility.php?queryfunction=deleteCookies
 ALERT: XHR response - Deleted all cookies
@@ -39,9 +39,9 @@
 ALERT: Test stage 8 document.cookie is: 
 ALERT: http://localhost:8000/cookies/resources/cookie-utility.php?queryfunction=setFooCookie
 ALERT: XHR response - Set the foo cookie
-ALERT: Test stage 9 document.cookie is: foo=awesomevalue
+ALERT: Test stage 9 document.cookie is: 
 ALERT: Restricting to first party only cookies
 ALERT: http://localhost:8000/cookies/resources/cookie-utility.php?queryfunction=setFooAndBarCookie
 ALERT: XHR response - Set the foo and bar cookies
-ALERT: Test stage 10 document.cookie is: bar=anotherawesomevalue; foo=awesomevalue
+ALERT: Test stage 10 document.cookie is:

[Eric Seidel (no email)]

new-run-webkit-tests http/tests/cookies/third-party-cookie-relaxing.html

fails when run straight on my machine.  Then again, so does:

old-run-webkit-tests http/tests/cookies/third-party-cookie-relaxing.html

[Eric Seidel (no email)]

It also fails for me with:
old-run-webkit-tests http/tests/cookies
and
old-run-webkit-tests http

I'm trying to understand how the test is supposed to work?

http://trac.webkit.org/browser/trunk/LayoutTests/http/tests/cookies/third-party-cookie-relaxing.html

Maybe there was a recent regression in behavior on Snow Leopard which happen to correspond with the NRWT transition?

Maybe NRWT running cookie-tests in parallel manages to permanently "jam" the CF cookie store?  Donno.

[Eric Seidel (no email)]

Qt just added support for third-party cookies, and their results seem to match SL, in at least the 6 and 9 cases:
http://trac.webkit.org/browser/trunk/LayoutTests/platform/qt/http/tests/cookies/third-party-cookie-relaxing-expected.txt

[Eric Seidel (no email)]

Bug 63545 is another cookie-related failure which occurs on my machine.  This one of course occurs on the SL bot as well:
http://build.webkit.org/results/SnowLeopard%20Intel%20Release%20(Tests)/r90460%20(31134)/results.html

[Eric Seidel (no email)]

Even if I move my ~/Library/Cookies/Cookies.plist file out of the way, the test still fails for me.

[Eric Seidel (no email)]

Turns out this has nothing to do with NRWT, now that I look at the bot history (with webkit-patch failure-reason).

[Eric Seidel (no email)]

I don't see any changes in that range which seem particularly related.  Perhaps failure-reason is confused.

[Eric Seidel (no email)]

The only possibly-related changes in that range are to the JIT.  But I don't see how they could affect this test:
http://trac.webkit.org/changeset/90237
http://trac.webkit.org/changeset/90232

I checked the results.  Unless build.webkit.org/results is lying, it's definitely this range.  I've not tried using git-bisect, but I think the build was broken a bunch in there, so it may not answer our question.

[Adam Roben (:aroben)]

TestFailures says this test is flaky:

r90315 (31042) (results.html): fail
r90310 (31041) (results.html): pass
r90308 (31040) (results.html): pass
r90288 (31027) (results.html): fail
r90285 (31026) (results.html): pass
r90284 (31025) (results.html): fail
r90280 (31024) (results.html): pass
r90279 (31023) (results.html): fail
r90273 (31022) (results.html): pass
r90271 (31019) (results.html): fail

It always fails with the same diff: http://build.webkit.org/results/SnowLeopard%20Intel%20Release%20(Tests)/r90315%20(31042)/http/tests/cookies/third-party-cookie-relaxing-pretty-diff.html

[Alexey Proskuryakov]

This is no longer happening. Karl removed the expectation via bug 213620.

[Radar WebKit Bug Importer]

<rdar://problem/64783252>