Bug 69104

Summary: [Qt] Fix failing tests after migrating to refactored Qt5
Product: WebKit Reporter: Csaba Osztrogonác <ossy>
Component: Tools / TestsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED FIXED    
Severity: Critical CC: abecsi, kbalazs, kling, laszlo.gombos, loki, ossy, vestbo, yael, zeno, zherczeg, zoltan
Priority: P1 Keywords: Qt, QtTriaged
Version: 528+ (Nightly build)   
Hardware: All   
OS: All   
Bug Depends on: 69105, 69129, 69132, 69134    
Bug Blocks:    

Description Csaba Osztrogonác 2011-09-29 14:13:16 PDT
After migrating there are 10605 failing test. (half of the test cases!!!)
You can find all fails here:
http://build.webkit.sed.hu/results/x86-32%20Linux%20Qt%20Release%20WebKit2/r96349%20%2812586%29.zip
Failing, but undocumented fails before migrating: http://build.webkit.sed.hu/results/x86-32%20Linux%20Qt%20Release%20WebKit2/r96328%20%2812574%29.zip

Now all failing tests are in the Skipped list to let developers go ahead.

But I think we should stop for a minute. Disabling half of the tests 
is a brutal regression, which we should fix as soon as possible.
Comment 1 Csaba Osztrogonác 2011-10-05 09:48:06 PDT
Now we have to check the remainding ~6000 tests manually. :-/
I think most of them pass, but they need a Qt5 specific expected result ...
Comment 2 Tor Arne Vestbø 2011-10-05 09:54:40 PDT
(In reply to comment #1)
> Now we have to check the remainding ~6000 tests manually. :-/
> I think most of them pass, but they need a Qt5 specific expected result ...

We can run them with something similar to the orwt --ignore-metrics, to verify that they at least are metrics-failures only.

Also, we should look at this in relation to switching to Qt 4.8, since some of those 6000 might overlap with 4.8, and 4.8 will be the new platform/qt baseline, and Qt 5 will be then new qt-wk2 baseline.
Comment 3 Csaba Osztrogonác 2011-10-19 08:08:42 PDT
These fails are not WK2 related, but Qt5 related, because they fail with Qt5-WK1 too. I moved them to the qt-5.0/Skipped list: http://trac.webkit.org/changeset/97860

Your idea is good, I ran an old-run-webkit-tests --ignore-metrics test on the new qt-5.0/Skipped list with WK1 and I got the following result:
5878 test cases (84%) succeeded
888 test cases (12%) had incorrect layout
176 test cases (2%) crashed
Comment 4 Csaba Osztrogonác 2012-02-02 10:20:56 PST
Most of them were unskipped after http://trac.webkit.org/changeset/102776,
so we can close this bug.