Bug 26886 – [META] qtwebkit layout tests

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Antonio Gomes]

In today's snapshot (http://trac.webkit.org/changeset/45436) qtwebkit has the following layout test results:

agomes@tonix:~/dev/webkit$[layout-tests]-> WEBKITOUTPUTDIR=`pwd`/WebKitBuild/Qt WebKitTools/Scripts/run-webkit-tests --qt --debug LayoutTests/
(...)
4801 test cases (96%) succeeded
152 test cases (3%) had incorrect layout
28 test cases (<1%) were new
1 test case (<1%) timed out
12 test cases (<1%) crashed
8 test cases (<1%) had stderr output

we *have* to improve that in order to get buildbot up again for qt platform.

ps: this is a meta bug, and individual bugs will be filed and marked as dependent for this one as the time goes...

[Antonio Gomes]

find below an evolutive timeline of LayoutTests status during during commits in Jul/07 (from Simon)

BUILD MODE: DEBUG
HASH: 3cad03d906acaa664c8c8266d6590ba98147808e (before Simon's action)
OUTPUT: http://pastebin.com/f1ada9d92
Summary:
4740 test cases (94%) succeeded
216 test cases (4%) had incorrect layout
32 test cases (<1%) were new
2 test cases (<1%) timed out
12 test cases (<1%) crashed
8 test cases (<1%) had stderr output

BUILD MODE: debug
HASH: 42bd5b82471a30d465300476040725dcf82de0cf (after timer fix)
OUTPUT: http://pastebin.com/f8192838
Summary
4772 test cases (95%) succeeded
183 test cases (3%) had incorrect layout
32 test cases (<1%) were new
3 test cases (<1%) timed out
12 test cases (<1%) crashed
8 test cases (<1%) had stderr output

BUILD MODE: debug
HASH: 916ce5e35d089bf5f3097004a81ff8e57eed7d52  (after focus fix)
Summary:
4793 test cases (95%) succeeded
162 test cases (3%) had incorrect layout
32 test cases (<1%) were new
3 test cases (<1%) timed out
12 test cases (<1%) crashed
8 test cases (<1%) had stderr output

BUILD MODE: debug
HASH: 7c0cfcdd04cd924fbaf94af4002bad7b6795555a  (after modifying the skiplist)
OUTPUT: http://pastebin.com/m3f3973ef
Summary:
4777 test cases (99%) succeeded
5 test cases (<1%) had incorrect layout
32 test cases (<1%) were new
2 test cases (<1%) timed out
9 test cases (<1%) crashed
8 test cases (<1%) had stderr output

QT
BUILD MODE: release
HASH: 7c0cfcdd04cd924fbaf94af4002bad7b6795555a  (after modifying the skiplist)
OUTPUT: http://pastebin.com/f60255a3c
Summary:
4790 test cases (99%) succeeded
3 test cases (<1%) had incorrect layout
32 test cases (<1%) were new
3 test cases (<1%) had stderr output

QT
BUILD MODE: release
HASH: 21657b649d4b7cde80d96e3ae221689e54ed2ff4 (ToT)
OUTPUT: http://pastebin.com/fbc5a498
Summary:
4791 test cases (99%) succeeded
3 test cases (<1%) had incorrect layout
32 test cases (<1%) were new
3 test cases (<1%) had stderr output

[Simon Hausmann]

(In reply to comment #1)
> QT
> BUILD MODE: release
> HASH: 21657b649d4b7cde80d96e3ae221689e54ed2ff4 (ToT)
> OUTPUT: http://pastebin.com/fbc5a498
> Summary:
> 4791 test cases (99%) succeeded
> 3 test cases (<1%) had incorrect layout
> 32 test cases (<1%) were new
> 3 test cases (<1%) had stderr output

Interesting stats :)

BTW, the submit-form-while-parsing failure should work in ToT now. The other two failures are probably caused by the lack of phonon support. Can you check if your Qt is built with phonon support? (is phonon in mkspecs/qconfig.pri's QT_CONFIG?)

[Antonio Gomes]

> (In reply to comment #1)
> > QT
> > BUILD MODE: release
> > HASH: 21657b649d4b7cde80d96e3ae221689e54ed2ff4 (ToT)
> > OUTPUT: http://pastebin.com/fbc5a498
> > Summary:
> > 4791 test cases (99%) succeeded
> > 3 test cases (<1%) had incorrect layout
> > 32 test cases (<1%) were new
> > 3 test cases (<1%) had stderr output
> 
> Interesting stats :)
> 
> BTW, the submit-form-while-parsing failure should work in ToT now. The other
> two failures are probably caused by the lack of phonon support. Can you check
> if your Qt is built with phonon support? (is phonon in mkspecs/qconfig.pri's
> QT_CONFIG?)

simon, I built qt myself with:

./configure --prefix=/usr -no-webkit -release -phonon -v

QT_CONFIG +=  minimal-config small-config medium-config large-config full-config qt3support phonon phonon-backend accessi    bility reduce_exports ipv6 clock-gettime clock-monotonic mremap getaddrinfo ipv6ifname getifaddrs inotify system-jpeg sys    tem-png png system-freetype system-zlib nis iconv glib gstreamer dbus openssl x11sm xshape xrender mitshm fontconfig xkb     xmlpatterns svg scripttools release

and webkit in release mode (ToT - ea9f22d06208ea1c639d0c0f1dcff4d05f2c2a0e)

RESULTS: 

4778 test cases (98%) succeeded
9 test cases (<1%) had incorrect layout
34 test cases (<1%) were new
8 test cases (<1%) crashed
18 test cases (<1%) had stderr output

konsole output here: http://pastebin.com/f206d215e

--> Something noticeable and annoying: I got 18 tests w/ "stderr output" but all of them output either

Xlib:  extension "RANDR" missing on display ":0.0".

or 

** Message: NP_Initialize
** Message: NP_Initialize succeeded
** Message: NP_Initialize
** Message: NP_Initialize succeeded
** Message: NP_Initialize
** Message: NP_Initialize succeeded
** Message: NP_Initialize
** Message: NP_Initialize succeeded

which are not problem at all :) Simon, do you know how to suppress those messages ? I could patch it ...

[Simon Hausmann]

(In reply to comment #3)
> --> Something noticeable and annoying: I got 18 tests w/ "stderr output" but
> all of them output either
> 
> Xlib:  extension "RANDR" missing on display ":0.0".

That's not good. Does your x server really support not Xrandr?

> or 
> 
> ** Message: NP_Initialize
> ** Message: NP_Initialize succeeded
> ** Message: NP_Initialize
> ** Message: NP_Initialize succeeded
> ** Message: NP_Initialize
> ** Message: NP_Initialize succeeded
> ** Message: NP_Initialize
> ** Message: NP_Initialize succeeded
> 
> which are not problem at all :) Simon, do you know how to suppress those
> messages ? I could patch it ...

I think those are produced by a netscape plugin. Do you happen to have the diamondx plugin installed?

[Antonio Gomes]

filed bug 27215 about this suckages

> > Xlib:  extension "RANDR" missing on display ":0.0".
> That's not good. Does your x server really support not Xrandr?

it seems so:

$ xdpyinfo
name of display:    :0.0
version number:    11.0
vendor string:    The X.Org Foundation
vendor release number:    10600000
X.Org version: 1.6.0
(...)
number of extensions:    29
    BIG-REQUESTS
    Composite
    DAMAGE
    DOUBLE-BUFFER
    DPMS
    DRI2
    GLX
    Generic Event Extension
    MIT-SCREEN-SAVER
    MIT-SHM
    NV-CONTROL
    NV-GLX
    RANDR

> > ** Message: NP_Initialize
> > ** Message: NP_Initialize succeeded
> > ** Message: NP_Initialize
> > ** Message: NP_Initialize succeeded
> > ** Message: NP_Initialize
> > ** Message: NP_Initialize succeeded
> > ** Message: NP_Initialize
> > ** Message: NP_Initialize succeeded
> > 
> > which are not problem at all :) Simon, do you know how to suppress those
> > messages ? I could patch it ...
> 
> I think those are produced by a netscape plugin. Do you happen to have the
> diamondx plugin installed?


I have not have it ... I will try it out again w/ it installed.

[Antonio Gomes]

this meta has done its job , and should be closed: now buildbot is up and running fine.