Bug 153551 – LayoutTest fast/loader/stateobjects/replacestate-frequency-iframe.html is flaky on El Cap, always times out on Yosemite.

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Ryan Haddad]

LayoutTest fast/loader/stateobjects/replacestate-frequency-iframe.html is flaky

This test was added with r195625, and it seems to be timing out every 2-3 runs across Mac testers.
<https://build.webkit.org/builders/Apple%20El%20Capitan%20Release%20WK2%20(Tests)/builds/2837>

<https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=fast%2Floader%2Fstateobjects%2Freplacestate-frequency-iframe.html>

[Brady Eidson]

Since the timeout case prints absolutely nothing, I cannot diagnose what may or may not be going wrong unless I can reproduce this locally (which I haven't been able to yet)

[Brady Eidson]

According to https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=fast%2Floader%2Fstateobjects%2Freplacestate-frequency-iframe.html it times out *EVERY* time on Yosemite testers.

W.
T.
F.

[Brady Eidson]

I've never seen a flake locally. I've run just this test 100 times in a row, and the entire directory 20 times in a row. Nothing.

[Brady Eidson]

I did notice something wrong with the test and I fixed it in http://trac.webkit.org/changeset/195687

But I don't see how that have caused the flake, or how it could resolve it.

Still exploring...

[Brady Eidson]

Wow.

The iframe url was: "resources/replaceState-iframe.html", but the file on disk is named "replacestate-iframe.html"

Notice the case sensitivity on the 'S'

Apparently some of our build bots are case sensitive and others are not, hence the flake.  :(

[Brady Eidson]

http://trac.webkit.org/changeset/195688

[Alexey Proskuryakov]

> Apparently some of our build bots are case sensitive and others are not, hence the flake.  :(

Ouch!

But unsure if that's a good or a bad thing, after all.

[Brady Eidson]

(In reply to comment #7)
> > Apparently some of our build bots are case sensitive and others are not, hence the flake.  :(
> 
> Ouch!
> 
> But unsure if that's a good or a bad thing, after all.

I think it's a good thing that we test both case sensitive and case insensitive.

I think it's a bad thing that:
A - The split was apparently accidental and unknown
B - Bots that are one or the other are not identified as such