The following layout test is flaky on Chromium, after 133069. Clearing the cache should isolate this test, but it's making the general problem of state leaking from one test to the next visible. In this case, 'http/tests/security/contentSecurityPolicy/object-src-url-allowed.html' is failing because it's getting a console message generated in 'http/tests/security/contentSecurityPolicy/object-src-none-blocked.html'. I'll skip that test to bring this one back to life. CCing Dirk and Bernhard, as I think we talked about this a while back.
Skipped the leaking test in http://trac.webkit.org/changeset/133168. CCing Tony and Ami in case they have brilliant ideas about why 133069 might have made this leakage so consistent.
TL;DR: see last paragraph for my suspicions. Adam: I think you know about CSP; mind reading through the tome below and seeing what you can see? Fascinatingly, console.log() & alert() bleeding between tests was the original impetus for me looking at the test that ended up triggering the observation that caches weren't cleared between tests and the impetus for r133069. It's circular references all the way down! (background in bug 92824; comment #6 especially about logs bleeding). I can repro the bot's failure for this bug on my linux desktop with: rm -rf /tmp/x2 && ./Tools/Scripts/new-run-webkit-tests --child-processes=1 --results-directory=/tmp/x2 --no-retry http/tests/security/contentSecurityPolicy/object-src-url-allowed.html http/tests/security/contentSecurityPolicy/object-src-none-blocked.html and note that this only happens with Release builds, not Debug (flakiness dashboard concurs). I had assumed that the problem is "log leaking" - that nwrt was receiving/interpreting a console log destined for the first test as having come form the second. But in fact that is not the case: the first test gets its message and passes (correctly), and then the second test also receives the same message (incorrectly). Instrumenting ChromeClientImpl::addMessageToConsole() shows that the same message is in fact delivered twice, not that nrwt is messing up stream flushing or accounting or somesuch. Adding a sleep() & attaching to the second aMTC() call with a debugger shows this backtrace: #0 0x00007f014143d83d in nanosleep () at ../sysdeps/unix/syscall-template.S:82 #1 0x00007f014143d6dc in __sleep (seconds=0) at ../sysdeps/unix/sysv/linux/sleep.c:138 #2 0x00007f014309ce36 in WebKit::ChromeClientImpl::addMessageToConsole (this=0x14e9e68, source=3852167088, type=WebCore::LogMessageType, level=WebCore::ErrorMessageLevel, message=..., lineNumber=0, sourceID=...) at ../../third_party/WebKit/Source/WebKit/chromium/src/ChromeClientImpl.cpp:392 #3 0x00007f0143b1b386 in WebCore::Console::addMessage (this=<optimized out>, source=WebCore::JSMessageSource, type=WebCore::LogMessageType, level=WebCore::ErrorMessageLevel, message=..., url=..., lineNumber=<optimized out>, callStack=..., requestIdentifier=<optimized out>) at ../../third_party/WebKit/Source/WebCore/page/Console.cpp:150 #4 0x00007f014351a876 in WebCore::Document::addMessage (this=<optimized out>, source=WebCore::JSMessageSource, type=WebCore::LogMessageType, level=WebCore::ErrorMessageLevel, message=..., sourceURL=..., lineNumber=<optimized out>, callStack=..., requestIdentifier=<optimized out>) at ../../third_party/WebKit/Source/WebCore/dom/Document.cpp:4882 #5 0x00007f014356d5e3 in WebCore::ScriptExecutionContext::addConsoleMessage (this=<optimized out>, source=<optimized out>, type=<optimized out>, level=<optimized out>, message=..., sourceURL=..., lineNumber=<optimized out>, callStack=..., requestIdentifier=<optimized out>, sourceURL=..., level=<optimized out>, source=<optimized out>, type=<optimized out>, message=..., requestIdentifier=<optimized out>, lineNumber=<optimized out>, this=<optimized out>, callStack=...) at ../../third_party/WebKit/Source/WebCore/dom/ScriptExecutionContext.cpp:316 #6 0x00007f0143b23c19 in WebCore::ContentSecurityPolicy::logToConsole (this=<optimized out>, message=..., contextURL=..., contextLine=..., state=<optimized out>) at ../../third_party/WebKit/Source/WebCore/page/ContentSecurityPolicy.cpp:1667 #7 0x00007f0143b1e8a9 in WebCore::ContentSecurityPolicy::reportViolation (this=0x17cd480, directiveText=..., consoleMessage=..., blockedURL=..., reportURIs=..., header=..., contextURL=..., contextLine=..., state=0x0) at ../../third_party/WebKit/Source/WebCore/page/ContentSecurityPolicy.cpp:1554 #8 0x00007f0143b1e80f in WebCore::CSPDirectiveList::reportViolation (this=<optimized out>, directiveText=..., consoleMessage=..., blockedURL=..., contextURL=..., contextLine=..., state=<optimized out>) at ../../third_party/WebKit/Source/WebCore/page/ContentSecurityPolicy.cpp:887 #9 0x00007f0143b2099a in WebCore::CSPDirectiveList::checkSourceAndReportViolation (this=0x17ad0a0, directive=<optimized out>, url=..., type=...) at ../../third_party/WebKit/Source/WebCore/page/ContentSecurityPolicy.cpp:814 #10 0x00007f0143b219ae in WebCore::CSPDirectiveList::allowObjectFromSource (this=0x17ad0a0, url=..., reportingStatus=<optimized out>) at ../../third_party/WebKit/Source/WebCore/page/ContentSecurityPolicy.cpp:1077 #11 0x00007f0143b23527 in WebCore::isAllowedByAllWithURL<&(WebCore::CSPDirectiveList::allowObjectFromSource(WebCore::KURL const&, WebCore::ContentSecurityPolicy::ReportingStatus) const)> (policies=..., url=..., reportingStatus=WebCore::ContentSecurityPolicy::SendReport) at ../../third_party/WebKit/Source/WebCore/page/ContentSecurityPolicy.cpp:1420 #12 0x00007f0143b2349a in WebCore::ContentSecurityPolicy::allowObjectFromSource (this=<optimized out>, url=..., reportingStatus=WebCore::ContentSecurityPolicy::SendReport) at ../../third_party/WebKit/Source/WebCore/page/ContentSecurityPolicy.cpp:1483 #13 0x00007f0143afdce4 in WebCore::SubframeLoader::pluginIsLoadable (this=0x14ddec0, pluginElement=0x1569780, url=..., mimeType=...) at ../../third_party/WebKit/Source/WebCore/loader/SubframeLoader.cpp:132 #14 0x00007f0143afde55 in WebCore::SubframeLoader::requestPlugin (this=0x14ddec0, ownerElement=0x1569780, url=..., mimeType=..., paramNames=..., paramValues=..., useFallback=252) at ../../third_party/WebKit/Source/WebCore/loader/SubframeLoader.cpp:155 #15 0x00007f0143afe2cb in WebCore::SubframeLoader::requestObject (this=0x14ddec0, ownerElement=0x1569780, url=..., frameName=..., mimeType=..., paramNames=..., paramValues=...) at ../../third_party/WebKit/Source/WebCore/loader/SubframeLoader.cpp:230 #16 0x00007f0143c22c68 in WebCore::HTMLObjectElement::updateWidget (this=0x1569780, pluginCreationOption=<optimized out>) at ../../third_party/WebKit/Source/WebCore/html/HTMLObjectElement.cpp:316 #17 0x00007f0143b55fd7 in WebCore::FrameView::updateWidget (this=<optimized out>, object=0x16e7510) at ../../third_party/WebKit/Source/WebCore/page/FrameView.cpp:2384 #18 0x00007f0143b5610f in WebCore::FrameView::updateWidgets (this=0x179e380) at ../../third_party/WebKit/Source/WebCore/page/FrameView.cpp:2417 #19 0x00007f0143b523ef in WebCore::FrameView::performPostLayoutTasks (this=0x179e380) at ../../third_party/WebKit/Source/WebCore/page/FrameView.cpp:2484 #20 0x00007f0143b51efc in WebCore::FrameView::layout (this=0x179e380, allowSubtree=<optimized out>) at ../../third_party/WebKit/Source/WebCore/page/FrameView.cpp:1241 #21 0x00007f01434d3ac8 in WebCore::RenderView::updateWidgetPositions (this=0x15d3020) at ../../third_party/WebKit/Source/WebCore/rendering/RenderView.cpp:749 #22 0x00007f0143b523c7 in WebCore::FrameView::performPostLayoutTasks (this=0x14ea180) at ../../third_party/WebKit/Source/WebCore/page/FrameView.cpp:2481 #23 0x00007f0143b51efc in WebCore::FrameView::layout (this=0x14ea180, allowSubtree=<optimized out>) at ../../third_party/WebKit/Source/WebCore/page/FrameView.cpp:1241 #24 0x00007f01436f8481 in WebCore::ThreadTimers::sharedTimerFiredInternal (this=0x16b5960) at ../../third_party/WebKit/Source/WebCore/platform/ThreadTimers.cpp:116 #25 0x00007f01447fe794 in Run (this=0x18a85a0) at ../../base/callback.h:391 #26 base::Timer::RunScheduledTask (this=<optimized out>) at ../../base/timer.cc:181 #27 0x00007f01447ca89b in Run (this=<optimized out>) at ../../base/callback.h:391 #28 MessageLoop::RunTask (this=0x14b5600, pending_task=...) at ../../base/message_loop.cc:470 #29 0x00007f01447caa2a in MessageLoop::DeferOrRunPendingTask (this=0x14b5600, pending_task=...) at ../../base/message_loop.cc:482 #30 0x00007f01447caccf in MessageLoop::DoWork (this=0x14b5600) at ../../base/message_loop.cc:665 #31 0x00007f014479c929 in base::MessagePumpGlib::RunWithDispatcher (this=0x14cb770, delegate=<optimized out>, dispatcher=<optimized out>) at ../../base/message_pump_glib.cc:203 #32 0x00007f01447e0191 in base::RunLoop::Run (this=0x7fffe59b74a0) at ../../base/run_loop.cc:45 #33 0x00007f01447c9fa9 in MessageLoop::Run (this=<optimized out>) at ../../base/message_loop.cc:307 #34 0x000000000044e2a5 in TestShell::waitTestFinished (this=0x7fffe59b7f88) at ../../third_party/WebKit/Tools/DumpRenderTree/chromium/TestShellPosix.cpp:66 #35 0x000000000044b3b9 in TestShell::runFileTest (this=0x7fffe59b7f88, params=..., shouldDumpPixels=<optimized out>) at ../../third_party/WebKit/Tools/DumpRenderTree/chromium/TestShell.cpp:280 #36 0x000000000043216d in runTest (shell=..., params=..., inputLine=..., forceDumpPixels=false) at ../../third_party/WebKit/Tools/DumpRenderTree/chromium/DumpRenderTree.cpp:113 #37 0x0000000000431d64 in main (argc=2, argv=0x7fffe59b8408) at ../../third_party/WebKit/Tools/DumpRenderTree/chromium/DumpRenderTree.cpp:258 My interpretation of this is that when DRT kicks off the second test, the first test still has an extant FrameView with a ticking timer; when that timer fires it re-does layout and re-discovers the CSP violation, so re-reports it. In fact instrumenting FrameView's ctor & dtor with print statements including |this| confirms that the backtrace above is using an undestroyed FrameView from the first test. (an alternative interpretation was that this is in fact the FrameView for the second test, and some other facet of the CSP-violation (the old <object>, the old CSP header, etc) is persisting between tests; but I don't think that's the case in light of the previous paragraph) My current suspicion is with TestShell::resetTestController() which includes: WebTestingSupport::resetInternalsObject(webView()->mainFrame()) what about frames which aren't "main" in webView()? Are they reset implicitly by the above call or do they need to be called out?
This nuttiness is a result of plugins loading via the render tree instead of the DOM. Why CSP blocks the load we probably need to do it in a way that we won't attempt to load the plugin again unless something significant changes.
(In reply to comment #3) > This nuttiness is a result of plugins loading via the render tree instead of the DOM. > > Why CSP blocks the load we probably need to do it in a way that we won't attempt to load the plugin again unless something significant changes. I'm happy to take this, but the plugin-loading flow is strange and complex. :) Do you have a suggestion for a blocking mechanism that would be more effective?
You could just force a layout in the test. :) <script>document.body.offsetTop</script> http://trac.webkit.org/browser/trunk/LayoutTests/http/tests/security/contentSecurityPolicy/object-src-none-blocked.html But DRT should be smart enough to have laid out and canceled all loads before moving to the next test? Could you show an example flake? Maybe a link to the flakiness dashboard?
(In reply to comment #5) > But DRT should be smart enough to have laid out and canceled all loads before moving to the next test? Could you show an example flake? Maybe a link to the flakiness dashboard? Back when I wrote comment #2 it was 100% reproable with the instructions there (pasted below). Do they not repro the problem for you? I can repro the bot's failure for this bug on my linux desktop with: rm -rf /tmp/x2 && ./Tools/Scripts/new-run-webkit-tests --child-processes=1 --results-directory=/tmp/x2 --no-retry http/tests/security/contentSecurityPolicy/object-src-url-allowed.html http/tests/security/contentSecurityPolicy/object-src-none-blocked.html and note that this only happens with Release builds, not Debug (flakiness dashboard concurs).
Unassigning myself; let's be realistic about what I'm actually working on. :/
This was never a problem on iOS or OS X, and given the lack of issues seen related to this bug in the last few years I'm going to close this.