Bug 67118 – [Qt] REGRESSION(r93937): 4 failing and 3 flaky tests

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Zoltan Horvath]

Failed:

fast/forms/input-appearance-number-rtl.html
fast/forms/input-appearance-spinbutton-disabled-readonly.html
fast/forms/input-appearance-spinbutton-layer.html
fast/forms/input-appearance-spinbutton-visibility.html

Flaky:

fast/forms/input-number-large-padding.html
fast/forms/input-spinbutton-capturing.html
fast/forms/input-number-events.html


I'm going to skip them.

[Zoltan Horvath]

Skipped in http://trac.webkit.org/changeset/93968.

[Csaba Osztrogonác]

The following tests passes, they needed new results:
fast/forms/input-appearance-number-rtl.html
fast/forms/input-appearance-spinbutton-disabled-readonly.html
fast/forms/input-appearance-spinbutton-layer.html
fast/forms/input-appearance-spinbutton-visibility.html

Patch landed in http://trac.webkit.org/changeset/94072

[Jarred Nicholls]

I'm not able to produce flaky results for any of the three tests, in OS X, Ubuntu 10.04, nor Q-BAT.

[Csaba Osztrogonác]

Unfortunately these tests only fails when you run all tests with NRWT.
(Tools/Scripts/run-webkit-tests)

[Jarred Nicholls]

I'm running NRWT, all tests, in parallel (directly w/ new-run-webkit-tests --platform=qt) and serially (run-webkit-tests) and am consistently not getting failures on these tests.  I am however getting an occasional failure in fast/forms/search-event-delay, presumably because of a timing reason but I didn't look into it.

[Zoltan Horvath]

(In reply to comment #5)
> I'm running NRWT, all tests, in parallel (directly w/ new-run-webkit-tests --platform=qt) and serially (run-webkit-tests) and am consistently not getting failures on these tests.  I am however getting an occasional failure in fast/forms/search-event-delay, presumably because of a timing reason but I didn't look into it.

Sounds good, if you have time please remove these from Skipped and give it a try on the bots!

[Csaba Osztrogonác]

(In reply to comment #6)
> Sounds good, if you have time please remove these from Skipped and give it a try on the bots!

Not at all, these flakey tests are still flakey with NRWT (Tools/Scripts/run-webkit-tests without any additional option!)  without parallelism.

[Jarred Nicholls]

I have been able to reproduce the flaky results once out of 5 runs.  I'll submit a patch here and we'll let the bots give it a run.

[Jarred Nicholls]

I have to jump onto other issues, but this flakiness is possibly somewhere in DRT.  Running the tests singly or in batches of 50, 100, or even 200 results in successful runs; probably for the same reasons that a minimal run (e.g. only fast/forms or the individual tests) or a parallel NRWT run doesn't produce the flaky results.

I am able to produce the flakiness with single-worker NRWT running all fast/ tests.  Somehow DRT, when permuted across those fast/ tests, corrupts the result of these spinbutton tests.  Specifically, eventSender ends up clicking on the down spin button rather than the top spin button.

[Tor Arne Vestbø]

Investigating this

[Tor Arne Vestbø]

I'm unable to reproduce this with a run of NRWT fast/ over 10 runs.

[Csaba Osztrogonác]

These test work now, unskip landed in r135381.