Bug 184432 – Fix flakiness in insecure-iframe-in-main-frame.html

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Keith Rollin]

The test LayoutTests/http/tests/security/contentSecurityPolicy/block-all-mixed-content/insecure-iframe-in-main-frame.html is flaky. It enables the option to dump frame load events. It then opens a new window. The result is that the load events of the window-opening-window and the load events of the opened-window are interleaved non-deterministically.

<rdar://problem/39297079>

[Keith Rollin]

data-url-iframe-in-main-frame.html seems like it should have the same issue, in that it enables load event dumping and then calls window.open().

[Keith Rollin]

Also:

* insecure-css-in-main-frame.html
* insecure-image-in-main-frame.html
* insecure-plugin-in-main-frame.html
* insecure-script-in-main-frame.html
* insecure-xhr-asynchronous-in-main-frame.html
* insecure-xhr-synchronous-in-main-frame.html
* about-blank-iframe-in-main-frame.html
* data-url-iframe-in-main-frame.html
* insecure-css-in-main-frame.html
* insecure-css-with-secure-cookies.html
* insecure-executable-css-with-secure-cookies.html
* insecure-form-in-main-frame.html
* insecure-iframe-in-main-frame.html
* javascript-url-form-in-main-frame.html
* redirect-http-to-https-iframe-in-main-frame.html
* redirect-https-to-http-iframe-in-main-frame.html
* redirect-https-to-http-image-secure-cookies-block.html
* redirect-https-to-http-image-secure-cookies.html

[Keith Rollin]

The difference may be related to the following code in insecure-iframe-in-main-frame.html:

    // FIXME: For some reason a SecurityPolicyViolation event is not dispatched in frame-with-insecure-iframe.html (why?).
    // So, dump-securitypolicyviolation-and-notify-done.js loaded by frame-with-insecure-iframe.html will never call
    // testRunner.notifyDone(). For now we do not call testRunner.waitUntilDone() and instead wait a fixed timeout :(
    window.setTimeout(function () {
        if (window.testRunner)
            testRunner.notifyDone();
    }, 500);

I note that, the comment notwithstanding, waitUntilDone is indeed called. I also note that the following tests have similar comments, but that they don't go so far as including the timeout code:

* insecure-iframe-in-iframe.html
* insecure-image-in-javascript-url-iframe-in-iframe.html

[Keith Rollin]

Ultimately, the issue is not with determinism. If the expected results for this test are rebased, we’ll pass the test every time. The issue was that the order in which events of interleaved windows are reported was changed. Most other tests affected by this change had their expected results changed. However, this particular test was not treated in the same way because it was marked as flaky. Since it was expected to occasionally fail, the fact that it now *always* failed was missed.

I'm (a) rebasing the expected results and (b) tweaking the test so that the two windows are not opening at the same time and so aren't having their frame-load events interleaved.

[Keith Rollin]

Created attachment 337632 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 337632 [details]
Patch

Clearing flags on attachment: 337632

Committed r230590: <https://trac.webkit.org/changeset/230590>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Chris Dumez]

Comment on attachment 337632 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=337632&action=review

> LayoutTests/http/tests/security/contentSecurityPolicy/block-all-mixed-content/insecure-iframe-in-main-frame.html:34
> +    window.setTimeout(function() {

setTimeout(0) still does not guarantee the current page will have loaded. We may want to do the setTimeout(0) in in unload = function() { }.