Bug 69104 – [Qt] Fix failing tests after migrating to refactored Qt5

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Csaba Osztrogonác]

After migrating there are 10605 failing test. (half of the test cases!!!)
You can find all fails here:
http://build.webkit.sed.hu/results/x86-32%20Linux%20Qt%20Release%20WebKit2/r96349%20%2812586%29.zip
Failing, but undocumented fails before migrating: http://build.webkit.sed.hu/results/x86-32%20Linux%20Qt%20Release%20WebKit2/r96328%20%2812574%29.zip

Now all failing tests are in the Skipped list to let developers go ahead.

But I think we should stop for a minute. Disabling half of the tests 
is a brutal regression, which we should fix as soon as possible.

[Csaba Osztrogonác]

Now we have to check the remainding ~6000 tests manually. :-/
I think most of them pass, but they need a Qt5 specific expected result ...

[Tor Arne Vestbø]

(In reply to comment #1)
> Now we have to check the remainding ~6000 tests manually. :-/
> I think most of them pass, but they need a Qt5 specific expected result ...

We can run them with something similar to the orwt --ignore-metrics, to verify that they at least are metrics-failures only.

Also, we should look at this in relation to switching to Qt 4.8, since some of those 6000 might overlap with 4.8, and 4.8 will be the new platform/qt baseline, and Qt 5 will be then new qt-wk2 baseline.

[Csaba Osztrogonác]

These fails are not WK2 related, but Qt5 related, because they fail with Qt5-WK1 too. I moved them to the qt-5.0/Skipped list: http://trac.webkit.org/changeset/97860

Your idea is good, I ran an old-run-webkit-tests --ignore-metrics test on the new qt-5.0/Skipped list with WK1 and I got the following result:
5878 test cases (84%) succeeded
888 test cases (12%) had incorrect layout
176 test cases (2%) crashed

[Csaba Osztrogonác]

Most of them were unskipped after http://trac.webkit.org/changeset/102776,
so we can close this bug.