Failed: fast/forms/input-appearance-number-rtl.html fast/forms/input-appearance-spinbutton-disabled-readonly.html fast/forms/input-appearance-spinbutton-layer.html fast/forms/input-appearance-spinbutton-visibility.html Flaky: fast/forms/input-number-large-padding.html fast/forms/input-spinbutton-capturing.html fast/forms/input-number-events.html I'm going to skip them.
Skipped in http://trac.webkit.org/changeset/93968.
The following tests passes, they needed new results: fast/forms/input-appearance-number-rtl.html fast/forms/input-appearance-spinbutton-disabled-readonly.html fast/forms/input-appearance-spinbutton-layer.html fast/forms/input-appearance-spinbutton-visibility.html Patch landed in http://trac.webkit.org/changeset/94072
I'm not able to produce flaky results for any of the three tests, in OS X, Ubuntu 10.04, nor Q-BAT.
Unfortunately these tests only fails when you run all tests with NRWT. (Tools/Scripts/run-webkit-tests)
I'm running NRWT, all tests, in parallel (directly w/ new-run-webkit-tests --platform=qt) and serially (run-webkit-tests) and am consistently not getting failures on these tests. I am however getting an occasional failure in fast/forms/search-event-delay, presumably because of a timing reason but I didn't look into it.
(In reply to comment #5) > I'm running NRWT, all tests, in parallel (directly w/ new-run-webkit-tests --platform=qt) and serially (run-webkit-tests) and am consistently not getting failures on these tests. I am however getting an occasional failure in fast/forms/search-event-delay, presumably because of a timing reason but I didn't look into it. Sounds good, if you have time please remove these from Skipped and give it a try on the bots!
(In reply to comment #6) > Sounds good, if you have time please remove these from Skipped and give it a try on the bots! Not at all, these flakey tests are still flakey with NRWT (Tools/Scripts/run-webkit-tests without any additional option!) without parallelism.
I have been able to reproduce the flaky results once out of 5 runs. I'll submit a patch here and we'll let the bots give it a run.
I have to jump onto other issues, but this flakiness is possibly somewhere in DRT. Running the tests singly or in batches of 50, 100, or even 200 results in successful runs; probably for the same reasons that a minimal run (e.g. only fast/forms or the individual tests) or a parallel NRWT run doesn't produce the flaky results. I am able to produce the flakiness with single-worker NRWT running all fast/ tests. Somehow DRT, when permuted across those fast/ tests, corrupts the result of these spinbutton tests. Specifically, eventSender ends up clicking on the down spin button rather than the top spin button.
Investigating this
I'm unable to reproduce this with a run of NRWT fast/ over 10 runs.
These test work now, unskip landed in r135381.