|Summary:||fast/css/object-fit/object-fit-embed.html isn't testing anything|
|Product:||WebKit||Reporter:||Alexey Proskuryakov <ap>|
|Component:||Tools / Tests||Assignee:||Tim Horton <thorton>|
|Severity:||Normal||CC:||aestes, simon.fraser, thorton|
Description Alexey Proskuryakov 2015-08-26 21:42:22 PDT
Positively bisected the flakiness to http://trac.webkit.org/r188807 - when the test fails, expected result seems to be dumped too early, before load finishes. I can reproduce on a debug build like this: run-webkit-tests fast/css/object-fit/object-fit-embed.html --repeat 50 -f --no-retry --no-show
Comment 1 Tim Horton 2015-08-26 21:53:35 PDT
Interesting! I'll look at this tomorrow.
Comment 2 Tim Horton 2015-08-26 21:58:06 PDT
It's fairly amazing that only one (or three or so) tests would be made flaky by this, if we really are snapshotting at the wrong time...
Comment 3 Tim Horton 2015-08-27 10:56:16 PDT
Even upping it to 500, this doesn't repro for me.
Comment 4 Tim Horton 2015-08-27 15:55:22 PDT
Alexey and I have learned a lot (like, for example, this test was previously not working correctly, but previously both the expected and actual would fail in the same way). It would appear that we are trying to use the "WebKit Test PlugIn" to load our PNG instead of an ImageDocument. And indeed, the plugin claims to load PNGs. Not yet sure why it ever sometimes works correctly, though.
Comment 5 Tim Horton 2015-08-27 15:57:52 PDT
The test similarly behaves wrongly in DRT.
Comment 6 Tim Horton 2015-08-27 16:04:38 PDT
And doing a: testRunner.setPluginsEnabled(false); makes it behave correctly. This does not, however, explain why the test became flaky (it should *always* behave incorrectly, consistently) with my patch on the bots and on Alexey's machine.
Comment 7 Tim Horton 2015-08-27 16:06:34 PDT
Patch came from Blink, smfr merged it. That probably explains how we ended up with a broken test.
Comment 8 Tim Horton 2015-08-27 16:06:54 PDT
(In reply to comment #7) > Patch came from Blink, smfr merged it. That probably explains how we ended > up with a broken test. Also, this is one of the bizarre downsides of reftests.
Comment 9 Tim Horton 2015-08-27 16:41:56 PDT
(In reply to comment #4) > Not yet sure why it ever sometimes works correctly, though. Alexey has another bug about sometimes not loading plugins when we should, and is going to investigate that part separately. I'm going to post a patch to disable plugins in the few tests that are affected by the test plugin's handling of PNGs.