After migrating there are 10605 failing test. (half of the test cases!!!) You can find all fails here: http://build.webkit.sed.hu/results/x86-32%20Linux%20Qt%20Release%20WebKit2/r96349%20%2812586%29.zip Failing, but undocumented fails before migrating: http://build.webkit.sed.hu/results/x86-32%20Linux%20Qt%20Release%20WebKit2/r96328%20%2812574%29.zip Now all failing tests are in the Skipped list to let developers go ahead. But I think we should stop for a minute. Disabling half of the tests is a brutal regression, which we should fix as soon as possible.
Now we have to check the remainding ~6000 tests manually. :-/ I think most of them pass, but they need a Qt5 specific expected result ...
(In reply to comment #1) > Now we have to check the remainding ~6000 tests manually. :-/ > I think most of them pass, but they need a Qt5 specific expected result ... We can run them with something similar to the orwt --ignore-metrics, to verify that they at least are metrics-failures only. Also, we should look at this in relation to switching to Qt 4.8, since some of those 6000 might overlap with 4.8, and 4.8 will be the new platform/qt baseline, and Qt 5 will be then new qt-wk2 baseline.
These fails are not WK2 related, but Qt5 related, because they fail with Qt5-WK1 too. I moved them to the qt-5.0/Skipped list: http://trac.webkit.org/changeset/97860 Your idea is good, I ran an old-run-webkit-tests --ignore-metrics test on the new qt-5.0/Skipped list with WK1 and I got the following result: 5878 test cases (84%) succeeded 888 test cases (12%) had incorrect layout 176 test cases (2%) crashed
Most of them were unskipped after http://trac.webkit.org/changeset/102776, so we can close this bug.