Bug 85203 – [Qt] DumpRenderTree does not use 'monospace' font when directed

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Dave Tharp]

Created attachment 139462 [details]
Reference test

Monospace font does not get used when running QT DRT.  Note that the monospace font does get used in the QT test browser.  

I'm attaching several files:

- a reference test that specifies monospace font
- png actual from run on QT Linux bots
- a png expected from the GTK bots (to show that monospace works there properly)
- a screen capture of the reference test running in the QT test browser.

[Dave Tharp]

Created attachment 139463 [details]
actual png generated on QT Linux bot

[Dave Tharp]

Created attachment 139464 [details]
expected png from GTK bot

To demonstrate what monospace should look like.

[Dave Tharp]

Created attachment 139465 [details]
Screen capture of ref test running in QT test browser

[Caio Marcelo de Oliveira Filho]

I'm working on this.

[Caio Marcelo de Oliveira Filho]

Created attachment 144071 [details]
Patch for our testfonts.git repository, not for WebKit

[Caio Marcelo de Oliveira Filho]

Created attachment 144566 [details]
And the complimentary patch for QtWebKit

This patch removes the special case for Qt5 WK1. We used to load another configuration file, but that won't be needed anymore.

[Caio Marcelo de Oliveira Filho]

Created attachment 144567 [details]
List of tests that will need rebaseline

List of the 5347 tests needing rebaseline :P

[Balazs Kelemen]

Nice improvement, now the conf file is much more readable (and I assume more reliable as well).

[Marcelo Lira]

Created attachment 150469 [details]
Patch for our testfonts.git repository, not for WebKit

[Marcelo Lira]

Created attachment 150472 [details]
And the complimentary patch for QtWebKit

[Caio Marcelo de Oliveira Filho]

Comment on attachment 150472 [details]
And the complimentary patch for QtWebKit

Missing ChangeLog.

[Caio Marcelo de Oliveira Filho]

(In reply to comment #11)
> (From update of attachment 150472 [details])
> Missing ChangeLog.

This patch should land with the additions to the Skipped too and mention the SHA-1 of the testfonts.git it depends. Maybe even create another bug with a more descriptive summary to land the WebKit change.

[Csaba Osztrogonác]

Discussion about this bug on QtWebKit ML: http://lists.webkit.org/pipermail/webkit-qt/2012-May/002846.html

The plan in nutshell:
- skip affected tests (~5400 tests) (INdT guys)
- land patches in WebKit trunk and in testsfonts.git (INdT guys)
- unskip and rebase tests in ~500 sized batches (Szeged guys)
  
After rebaselining, we will have up-to-date png results for almost 
all tests. It's good time to try to start run pixel tests on the bots.
(It will be experiment first, because I'm not sure if png results will
be same (pixel by pixel) on 32/64 bit, on release/debug, on wk1/wk2.)

I only worry about the Qt 4.8 results a little bit. It would be great if
we can keep same results for Qt 4.8, Qt 5.0-WK1 and Qt 5.0-WK2 as the
actualities. At least until branching WebKit for productization and
drop supporting Qt 4.8 in trunk. I'm going to check if the new results
are same for these 3 platform today.

How does it sound?

[Luciano Wolf]

Ossy, it sounds like a good plan. Talking to Alexis, there are 2 points that worth taking note:
- we need a proper wikipage describing "how to do" the unskip/rebase process so we can get more people involved;
- we have to keep all relevant comments from this mailing list thread and integrate into commit. It's a huge patch so we should keep as much information as possible without relying on links :)

[Csaba Osztrogonác]

I made a quick experiment with the latest two patches (https://bugs.webkit.org/attachment.cgi?id=150469 and https://bugs.webkit.org/attachment.cgi?id=150472).
I applied them, run tests on qt-5.0-wk1 platform, 5448 tests failed. And then I generated new results for these tests to platform/qt and run tests on qt-5.0-wk2 and qt-4.8 platform. There are only ~400 failing tests on both of them. But maybe some of them are false positive, because there can be not updated results in qt-4.8 and qt-5.0-wk2 platform. So it seems we will be able handle differences.

[Marcelo Lira]

Created attachment 150665 [details]
Patch for our testfonts.git repository, not for WebKit

[Marcelo Lira]

Created attachment 150668 [details]
And the complimentary patch for QtWebKit

[Marcelo Lira]

Created attachment 150669 [details]
And the complimentary patch for QtWebKit

Now with ChangeLogs. :-s

[Rafael Brandao]

Comment on attachment 150669 [details]
And the complimentary patch for QtWebKit

View in context: https://bugs.webkit.org/attachment.cgi?id=150669&action=review

> LayoutTests/ChangeLog:3
> +        The test fonts used for Qt tests moved to the Liberation font family.

Maybe "were moved" instead of "moved"? Or "changed", like the comment you've added on Skipped list.

> LayoutTests/ChangeLog:6
> +        unskiped in batches, ASAP.

"unskipped".

> LayoutTests/ChangeLog:10
> +        https://bugs.webkit.org/show_bug.cgi?id=85203

ChangeLog format is wrong. You should have the bug title followed by the bug url and then the description of your patch.

[Kenneth Rohde Christiansen]

Comment on attachment 150669 [details]
And the complimentary patch for QtWebKit

r- due to comments

[Csaba Osztrogonác]

Created attachment 150927 [details]
failing tests on Qt 4.8 after these change

Here is the list of failing tests on Qt 4.8 with these 
patches. We should add them to platform/qt-4.8/Skipped.

My idea is that it makes unskipping work easily distributable. For example we grab 
500-500 tests per person, check them manually, update/add txt/png baselines to platform/qt.
I think the default platform/qt baseline should be Qt-5.0 WK1 results on Ubuntu 11.10 64bit inside XVFB.
And then other persons can check the few hundred exceptions for platform/qt-4.8 and platform/qt-5.0-wk2.

other pros/cons? maybe objections? :)

[Csaba Osztrogonác]

Created attachment 150928 [details]
more failing tests on Qt 5.0 WK2 after these changes

Here is the failing tests on Qt 5.0 WK2 with the proposed patches and skipped tests.
Could you integrate this list and the Qt 4.8 list to the patch, please?

[Bruno Abinader (history only)]

I can help with Qt5 / Ubuntu 11.10 32bit inside XVFB :)

(In reply to comment #21)
> Created an attachment (id=150927) [details]
> failing tests on Qt 4.8 after these change
> 
> Here is the list of failing tests on Qt 4.8 with these 
> patches. We should add them to platform/qt-4.8/Skipped.
> 
> My idea is that it makes unskipping work easily distributable. For example we grab 
> 500-500 tests per person, check them manually, update/add txt/png baselines to platform/qt.
> I think the default platform/qt baseline should be Qt-5.0 WK1 results on Ubuntu 11.10 64bit inside XVFB.
> And then other persons can check the few hundred exceptions for platform/qt-4.8 and platform/qt-5.0-wk2.
> 
> other pros/cons? maybe objections? :)

[Marcelo Lira]

Created attachment 150948 [details]
And the complimentary patch for QtWebKit

[Marcelo Lira]

Created attachment 150963 [details]
And the complimentary patch for QtWebKit

[Csaba Osztrogonác]

Comment on attachment 150963 [details]
And the complimentary patch for QtWebKit

Can we wait Qt5 update, please? :)

[Csaba Osztrogonác]

Comment on attachment 150963 [details]
And the complimentary patch for QtWebKit

Landed in https://trac.webkit.org/changeset/121971

[Csaba Osztrogonác]

Comment on attachment 150665 [details]
Patch for our testfonts.git repository, not for WebKit

Landed in 558f3e7afab058def053a7149160c027a0e67aeb (git://gitorious.org/qtwebkit/testfonts.git)

[Luciano Wolf]

Created attachment 151146 [details]
Batch of reviewed expected results - animations and compositing

I'm not sure about some files. They are listed inside Changelog file.

[Luciano Wolf]

Created attachment 151246 [details]
Batch of reviewed expected txt results - animations and compositing

I'm not sure about some files. They are listed inside Changelog file. This patch includes only expected txt results, as proposed in mailing list.

[Noam Rosenthal]

Comment on attachment 151246 [details]
Batch of reviewed expected txt results - animations and compositing

LGTM

[Csaba Osztrogonác]

Could you wait a little bit with landing, please? I'm writing a detailed e-mail now.

[Luciano Wolf]

Working on css1 directory.

The compositing directory has some chance of being updated later to make use of new WRT tests thing (new PNGs - discussed at #IRC and in a conversation with Caio). So I'm moving to css1 (next in line).

[Alex Bravo]

I'll take html sub-directory.

[Kristóf Kosztyó]

I'll take the css3 directory

[Szilard Ledan]

I'll take css2.1/20110323 sub-directory (244 tests).

[János Badics]

I'll take the fast/block dir

[Ádám Kallai]

I'm working on editing/deleting test directory (121 tests)

[Balazs Ankes]

I'll take editing/selection directory.

[Kristóf Kosztyó]

I'm going to take svg/custom tests

[Noam Rosenthal]

(In reply to comment #33)
> Working on css1 directory.
> 
> The compositing directory has some chance of being updated later to make use of new WRT tests thing (new PNGs - discussed at #IRC and in a conversation with Caio). So I'm moving to css1 (next in line).

I think we can safely rebase the text results for compositing and animations, and skip the pixel results until we're ready with https://bugs.webkit.org/show_bug.cgi?id=90394

[Ádám Kallai]

I'm going to upadte all of the editing tests.

[Balazs Ankes]

I did the svg/W3C-SVG-1.*.
Now Janos and I are going to take the tables directory.

[Alex Bravo]

Created attachment 151986 [details]
Patch

[Alex Bravo]

Comment on attachment 151986 [details]
Patch

I will create a different bug

[Balazs Kelemen]

(In reply to comment #41)
> (In reply to comment #33)
> > Working on css1 directory.
> > 
> > The compositing directory has some chance of being updated later to make use of new WRT tests thing (new PNGs - discussed at #IRC and in a conversation with Caio). So I'm moving to css1 (next in line).
> 
> I think we can safely rebase the text results for compositing and animations, and skip the pixel results until we're ready with https://bugs.webkit.org/show_bug.cgi?id=90394

I go for compositing. I plan to first rebase text results, than push pixel results that are actually correct so later we can test the snapshot pixel results  with those.

[Luciano Wolf]

Working on scrollbars, transforms, transitions directories

[Balazs Ankes]

I'll take until fast/dom (except fast/block).

[Bruno Abinader (history only)]

I'm taking fast/forms on bug 91504.

[Szilard Ledan]

I'm going to take fast/css tests

[Bruno Abinader (history only)]

Now taking fast/table directory on bug 91621.

[Balazs Ankes]

I'm taking
- fast/dynamic
- fast/encoding
- fast/events
- fast/fast-mobile-scrolling
- fast/flexbox
- fast/frames
- fast/gradients
- fast/history
- fast/html
- fast/images
- fast/inline-block
- fast/inline
- fast/innerHTML
- fast/inspector-support
- fast/invalid
- fast/layers
- fast/line-grid
- fast/lists
- fast/media
- fast/multicol

[Bruno Abinader (history only)]

Now taking fast/selectors directory on bug 91640.

[Caio Marcelo de Oliveira Filho]

Taking svg/text, bug 91762.

[Balazs Ankes]

I'm taking
- fast/overflow
- fast/parser
- fast/reflections
- fast/repaint
- fast/replaced
- fast/ruby
- fast/runin
- fast/spatial-navigation
- fast/text
- fast/tokenizer
- fast/transforms
- fast/writing-mode
- fast/xsl
(all of the remaining fast/* tests)

[Szilard Ledan]

I'm taking
- svg/as-background-image
- svg/as-border-image
- svg/as-image
- svg/as-object
- svg/batik
- svg/carto.net
- svg/clip-path
- svg/css
- svg/custom (Kristof gave the results)
- svg/dom
- svg/filters
- svg/foreignObject
- svg/hixie

( bug 91856 )

[Luciano Wolf]

taking ietestcenter (bug 91886)

[Caio Marcelo de Oliveira Filho]

Taking editing/inserting, bug 91985.

[Luciano Wolf]

taking animations - updating only text results, as previously discussed in this thread (bug 91997)

[Balazs Ankes]

taking css2.1/t* (bug 91999)

[Rafael Brandao]

(In reply to comment #60)
> taking css2.1/t* (bug 91999)

I'm taking editing/pasteboard on bug 91621.

[Rafael Brandao]

(In reply to comment #61)
> (In reply to comment #60)
> > taking css2.1/t* (bug 91999)
> 
> I'm taking editing/pasteboard on bug 91621.

Sorry, I mean I`m taking editing/pasteboard on bug 92007. :P

[Rafael Brandao]

(In reply to comment #62)
> (In reply to comment #61)
> > (In reply to comment #60)
> > > taking css2.1/t* (bug 91999)
> > 
> > I'm taking editing/pasteboard on bug 91621.
> 
> Sorry, I mean I`m taking editing/pasteboard on bug 92007. :P

It conflicted with kadam's work. Then I'll work on svg/zoom on bug 92007 if no one objects.

[Caio Marcelo de Oliveira Filho]

Taking tables/mozilla_expected_failures, bug 92013.

[Luciano Wolf]

taking platform, plugins and printing directories (bug 92016)

[Rafael Brandao]

Taking the remaining svg: svg/{in-html,overflow,repaint,wicd} on bug 92023.

[Caio Marcelo de Oliveira Filho]

Taking remaining dom/, bug 92028.

[Caio Marcelo de Oliveira Filho]

Taking css3/selectors, bug 92038.

[Alexis Menard (darktears)]

(In reply to comment #68)
> Taking css3/selectors, bug 92038.

Taking fast/borders

[Luciano Wolf]

taking fast/block/basic directory

[Luciano Wolf]

taking fast/block/float directory

[Luciano Wolf]

(In reply to comment #69)
> (In reply to comment #68)
> > Taking css3/selectors, bug 92038.
> 
> Taking fast/borders

I'll continue the work on this one (fast/borders)

[Caio Marcelo de Oliveira Filho]

Taking fast/block/positioning, bug 92174.

[Bruno Abinader (history only)]

Taking editing/selection, bug 92186.

[Luciano Wolf]

taking fast/block/margin-collapse (bug #92274)

[Luciano Wolf]

taking editing/undo, fast/block/lineboxcontain and fast/js directories

[Luciano Wolf]

(In reply to comment #76)
> taking editing/undo, fast/block/lineboxcontain and fast/js directories

also taking css3/filters and tables/layering (bug 92292)

[Noam Rosenthal]

(In reply to comment #77)
> (In reply to comment #76)
> > taking editing/undo, fast/block/lineboxcontain and fast/js directories
> 
> also taking css3/filters and tables/layering (bug 92292)
For  now, don't rebaseline pixel results for anything under css3/filteres/*-hw.html. It's going to have different pixel results when switching to window snapshots.

[Luciano Wolf]

(In reply to comment #78)
> (In reply to comment #77)
> > (In reply to comment #76)
> > > taking editing/undo, fast/block/lineboxcontain and fast/js directories
> > 
> > also taking css3/filters and tables/layering (bug 92292)
> For  now, don't rebaseline pixel results for anything under css3/filteres/*-hw.html. It's going to have different pixel results when switching to window snapshots.

Ok! Only updating a few text results (3) for css3/filters.

[WebKit Review Bot]

Comment on attachment 151246 [details]
Batch of reviewed expected txt results - animations and compositing

Cleared Noam Rosenthal's review+ from obsolete attachment 151246 [details] so that this bug does not appear in http://webkit.org/pending-commit.

[Caio Marcelo de Oliveira Filho]

DumpRenderTree and WebKitTestRunner now use monospace fonts when asked to. So I'll close this as fixed.

There's still a few tests that need to be triaged and bug 91610 was created to track them. They are identified in Qt's Skipped files.