Running the following, and looking at image_diffs in the output shows that the completely wrong font is loaded: WebKitTools/Scripts/run-webkit-tests --qt --platform mac --tolerance 2.0 -p fast/block/basic/019.html Running the following shows the font in WEBKIT_TESTFONTS getting selected: export FC_DEBUG=1 WebKitBuild/unskip1/Release/bin/DumpRenderTree -v LayoutTests/fast/block/basic/019.html I don't understand enough about the fontconfig files in WEBKIT_TESTFONTS to know how they should be changed to load a font that looks like the one expected by Mac but presumably there must be something straightforward to be done there. Tor Arne, you seem to have set some of this up originally. Why does the font loaded by the Qt DRT not resemble the one used by the Mac pixel tests? robert@mwenge:~/Development/WebKit$ WebKitBuild/unskip1/Release/bin/DumpRenderTree -v LayoutTests/fast/block/basic/019.html FC_DEBUG=1 FC_DEBUG=1 Match Pattern has 15 elts (size 16) family: "Arial"(s) "sans-serif"(w) "Helvetica"(w) "Nimbus Sans L"(w) "sans-serif"(w) slant: 0(i)(s) 0(i)(s) weight: 100(i)(s) 100(i)(s) width: 100(i)(s) 100(i)(s) pixelsize: 16(f)(s) 16(f)(s) hintstyle: 3(i)(s) hinting: FcTrue(s) verticallayout: FcFalse(s) autohint: FcFalse(s) globaladvance: FcTrue(s) outline: FcTrue(s) lang: "en-IE"(s) fontversion: 2147483647(i)(s) embeddedbitmap: FcTrue(s) decorative: FcFalse(s) Best score 0 0 100 100 3 0 0 0 0 2000 0 0 0 0 0 2.14748e+11Pattern has 15 elts (size 15) family: "Nimbus Sans L"(w) style: "Regular"(w) slant: 0(i)(w) weight: 80(i)(w) width: 100(i)(w) foundry: "urw"(w) file: "/home/robert/Development/webkit/testfonts//n019003l.pfb"(w) index: 0(i)(w) outline: FcTrue(w) scalable: FcTrue(w) charset: 0000: 00000000 ffffffff ffffffff 7fffffff 00000000 ffffffff ffffffff ffffffff 0001: ffffffff ffffffff ffffffff ffffffff 00040000 00000000 00000000 00000000 0002: 0f000000 00000000 00000000 00000000 00000000 00000000 3f0002c0 00000000 0003: 00000000 00000000 00000000 00000000 00100000 10000000 00000000 00000000 0004: ffffffff ffffffff ffffffff 00000000 fffff000 ffffffff ffff199f 033fffff 0020: 77180000 06010047 00000010 00000000 00000000 00001000 00000000 00000000 0021: 00400000 00000004 00000000 00000000 00000000 00000000 00000000 00000000 0022: 46260044 00000000 00000000 00000031 00000000 00000000 00000000 00000000 0025: 00000000 00000000 00000000 00000000 00000000 00000000 00000400 00000000 00f6: 00000000 00000000 00000000 00000000 00000000 00000000 00000008 00000000 00fb: 00000006 00000000 00000000 00000000 00000000 00000000 00000000 00000000 (w) lang: aa|ab|af|ast|ava|ay|ba|be|bg|bi|br|bs|bua|ca|ce|ch|chm|co|cs| cv|da|de|en|eo|es|et|eu|fi|fj|fo|fr|fur|fy|gd|gl|gv|ho|hr|hu|ia|id|ie|ik| io|is|it|kaa|ki|kk|kl|ku|kum|kv|ky|la|lb|lez|lt|lv|mg|mh|mk|mo|mt|nb|nds| nl|nn|no|nr|nso|ny|oc|om|os|pl|pt|rm|ro|ru|sah|se|sel|sh|sk|sl|sma|smj|smn| so|sq|sr|ss|st|sv|sw|tg|tk|tn|tr|ts|tt|tyv|uk|uz|vo|vot|wa|wen|wo|xh|yap| zu(w) fontversion: 0(i)(w) fontformat: "Type 1"(w) decorative: FcFalse(w) Match Pattern has 15 elts (size 16) family: "Arial"(s) "sans-serif"(w) "Helvetica"(w) "Nimbus Sans L"(w) "sans-serif"(w) slant: 0(i)(s) 0(i)(s) weight: 200(i)(s) 200(i)(s) width: 100(i)(s) 100(i)(s) pixelsize: 32(f)(s) 32(f)(s) hintstyle: 3(i)(s) hinting: FcTrue(s) verticallayout: FcFalse(s) autohint: FcFalse(s) globaladvance: FcTrue(s) outline: FcTrue(s) lang: "en-IE"(s) fontversion: 2147483647(i)(s) embeddedbitmap: FcTrue(s) decorative: FcFalse(s) Best score 0 0 100 100 3 0 0 0 0 0 0 0 0 0 0 2.14748e+11Pattern has 15 elts (size 15) family: "Nimbus Sans L"(w) style: "Bold"(w) slant: 0(i)(w) weight: 200(i)(w) width: 100(i)(w) foundry: "urw"(w) file: "/home/robert/Development/webkit/testfonts//n019004l.pfb"(w) index: 0(i)(w) outline: FcTrue(w) scalable: FcTrue(w) charset: 0000: 00000000 ffffffff ffffffff 7fffffff 00000000 ffffffff ffffffff ffffffff 0001: ffffffff ffffffff ffffffff ffffffff 00040000 00000000 00000000 00000000 0002: 0f000000 00000000 00000000 00000000 00000000 00000000 3f0002c0 00000000 0003: 00000000 00000000 00000000 00000000 00100000 10000000 00000000 00000000 0004: ffffffff ffffffff ffffffff 00000000 fffff000 ffffffff ffff199f 033fffff 0020: 77180000 06010047 00000010 00000000 00000000 00001000 00000000 00000000 0021: 00000000 00000004 00000000 00000000 00000000 00000000 00000000 00000000 0022: 06260044 00000000 00000000 00000031 00000000 00000000 00000000 00000000 0025: 00000000 00000000 00000000 00000000 00000000 00000000 00000400 00000000 00f6: 00000000 00000000 00000000 00000000 00000000 00000000 00000008 00000000 00fb: 00000006 00000000 00000000 00000000 00000000 00000000 00000000 00000000 (w) lang: aa|ab|af|ast|ava|ay|ba|be|bg|bi|br|bs|bua|ca|ce|ch|chm|co|cs| cv|da|de|en|eo|es|et|eu|fi|fj|fo|fr|fur|fy|gd|gl|gv|ho|hr|hu|ia|id|ie|ik| io|is|it|kaa|ki|kk|kl|ku|kum|kv|ky|la|lb|lez|lt|lv|mg|mh|mk|mo|mt|nb|nds| nl|nn|no|nr|nso|ny|oc|om|os|pl|pt|rm|ro|ru|sah|se|sel|sh|sk|sl|sma|smj|smn| so|sq|sr|ss|st|sv|sw|tg|tk|tn|tr|ts|tt|tyv|uk|uz|vo|vot|wa|wen|wo|xh|yap| zu(w) fontversion: 0(i)(w) fontformat: "Type 1"(w) decorative: FcFalse(w) Source: <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"><html lang="en-au"><head> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> <title></title> <style> h1 { border: 3px solid red; } </style> </head><body> <p>These <h1>s should all be left-aligned:</p> <h1 style="width: auto;">sample</h1> <h1 style="width: 10em;">sample</h1> <p>These <p>s should be (and are) left-aligned:</p> <p style="width: auto;">sample</p> <p style="width: 10em;">sample</p> </body></html> layer at (0,0) size 800x600 RenderView at (0,0) size 800x600 layer at (0,0) size 800x301 RenderBlock {HTML} at (0,0) size 800x301 RenderBody {BODY} at (8,16) size 784x269 RenderBlock {P} at (0,0) size 784x20 RenderText {#text} at (0,0) size 276x20 text run at (0,0) width 276: "These <h1>s should all be left- aligned:" RenderBlock {H1} at (0,41) size 784x47 [border: (3px solid #FF0000)] RenderText {#text} at (3,3) size 111x41 text run at (3,3) width 111: "sample" RenderBlock {H1} at (0,109) size 326x47 [border: (3px solid #FF0000)] RenderText {#text} at (3,3) size 111x41 text run at (3,3) width 111: "sample" RenderBlock {P} at (0,177) size 784x20 RenderText {#text} at (0,0) size 314x20 text run at (0,0) width 314: "These <p>s should be (and are) left-aligned:" RenderBlock {P} at (0,213) size 784x20 RenderText {#text} at (0,0) size 52x20 text run at (0,0) width 52: "sample" RenderBlock {P} at (0,249) size 160x20 RenderText {#text} at (0,0) size 52x20 text run at (0,0) width 52: "sample" #EOF #EOF #EOF
(In reply to comment #0) > Running the following, and looking at image_diffs in the output shows that > the completely wrong font is loaded: > > WebKitTools/Scripts/run-webkit-tests --qt --platform mac --tolerance 2.0 -p > fast/block/basic/019.html You're running this on Linux right, and want to compare with the pixel and/or render-tree results for mac? That's not going to work, as the code-paths for drawing the text are different on those two platforms. On Linux we use Freetype, on OS X we use ATSUI (carbon) or CorteText (Cocoa). They way these underlying frameworks handle metrics vary, for example because they use different algorithms for grid-fitting, so the pixel-results and render-tree-dumps will be different. The WEBKIT_TESTFONTS we use for the Qt port is not meant to match the Mac results pixel by pixel, but to have a consistent set of fonts on Linux. I think that's what you're asking? :)
(In reply to comment #1) > (In reply to comment #0) > > Running the following, and looking at image_diffs in the output shows that > > the completely wrong font is loaded: > > > > WebKitTools/Scripts/run-webkit-tests --qt --platform mac --tolerance 2.0 -p > > fast/block/basic/019.html > > You're running this on Linux right, and want to compare with the pixel and/or > render-tree results for mac? > > That's not going to work, as the code-paths for drawing the text are different > on those two platforms. On Linux we use Freetype, on OS X we use ATSUI (carbon) > or CorteText (Cocoa). They way these underlying frameworks handle metrics vary, > for example because they use different algorithms for grid-fitting, so the > pixel-results and render-tree-dumps will be different. > > The WEBKIT_TESTFONTS we use for the Qt port is not meant to match the Mac > results pixel by pixel, but to have a consistent set of fonts on Linux. > > I think that's what you're asking? :) It is indeed, and thanks for the clarification! So I think this means we have to create Qt-specific expected results for all tests where x/y rendertree values differ from Mac. I've started this effort at https://bugs.webkit.org/show_bug.cgi?id=38437.
(In reply to comment #2) > > I think that's what you're asking? :) > > It is indeed, and thanks for the clarification! > > So I think this means we have to create Qt-specific expected results for all > tests where x/y rendertree values differ from Mac. I've started this effort at > https://bugs.webkit.org/show_bug.cgi?id=38437. Sadly it's not that "simple". Due to the underlying platform layers in Qt we will have different results for Qt on Linux, Mac, and Windows. The Qt mac results will not match the Safari mac results. Even on Linux the result will vary depending on the freetype and fontconfig version you have (that's why you'll see different results than the bot). So in practice you would have to generate, verify, _and maintain_ at least 3 sets of expected files. There have been discussions on how to solve this on a more general level. Some options are: - Convert tests to dumpAsText() or dumpAsMarkup() - Run DRT (on all platforms) with fake hard-coded font-metrics - Only dump part of the test as render tree - Use ref-tests This is still under investigation though. Feel free to jump in if you have any ideas. Oh, and we have the same problem for theming, not just fonts, since each port renders input elements differently, and those input elements are used in tests that don't really test input elements.
(In reply to comment #3) > (In reply to comment #2) > > > I think that's what you're asking? :) > > > > It is indeed, and thanks for the clarification! > > > > So I think this means we have to create Qt-specific expected results for all > > tests where x/y rendertree values differ from Mac. I've started this effort at > > https://bugs.webkit.org/show_bug.cgi?id=38437. > > Sadly it's not that "simple". Due to the underlying platform layers in Qt we > will have different results for Qt on Linux, Mac, and Windows. The Qt mac > results will not match the Safari mac results. Even on Linux the result will > vary depending on the freetype and fontconfig version you have (that's why > you'll see different results than the bot). > > So in practice you would have to generate, verify, _and maintain_ at least 3 > sets of expected files. > > There have been discussions on how to solve this on a more general level. Some > options are: > > - Convert tests to dumpAsText() or dumpAsMarkup() > - Run DRT (on all platforms) with fake hard-coded font-metrics > - Only dump part of the test as render tree > - Use ref-tests > > This is still under investigation though. Feel free to jump in if you have any > ideas. > > Oh, and we have the same problem for theming, not just fonts, since each port > renders input elements differently, and those input elements are used in tests > that don't really test input elements. So do you think I should refrain from committing the patch in https://bugs.webkit.org/show_bug.cgi?id=38437 ? Although I didn't submit it in the belief that it would solve things forever, or that it won't require maintenance the next time the bot upgrades, it was certainly a useful exercise to single out the bugs in the skipped editing/pasteboard tests. Unskipping them also ensures QtWebKit is protected against regressions. As you say, the price will be maintaining them - especially when the bot upgrades to a newer version of Qt, which will happen quite soon.
Updating Qt on bot is absolutely unrelated the patch you mentioned. Now we use Qt-4.6.0 and we will update to Qt-4.7.0 when it is released. We have bot for Qt-4.7.0 branch and it seems we can update quickly: http://webkit.sed.hu/buildbot/builders/x86-32%20Linux%20Qt-4.7.0%20Release
(In reply to comment #4) > (In reply to comment #3) > > (In reply to comment #2) > > > > I think that's what you're asking? :) > > > > > > It is indeed, and thanks for the clarification! > > > > > > So I think this means we have to create Qt-specific expected results for all > > > tests where x/y rendertree values differ from Mac. I've started this effort at > > > https://bugs.webkit.org/show_bug.cgi?id=38437. > > > > Sadly it's not that "simple". Due to the underlying platform layers in Qt we > > will have different results for Qt on Linux, Mac, and Windows. The Qt mac > > results will not match the Safari mac results. Even on Linux the result will > > vary depending on the freetype and fontconfig version you have (that's why > > you'll see different results than the bot). > > > > So in practice you would have to generate, verify, _and maintain_ at least 3 > > sets of expected files. > > > > There have been discussions on how to solve this on a more general level. Some > > options are: > > > > - Convert tests to dumpAsText() or dumpAsMarkup() > > - Run DRT (on all platforms) with fake hard-coded font-metrics > > - Only dump part of the test as render tree > > - Use ref-tests > > > > This is still under investigation though. Feel free to jump in if you have any > > ideas. > > > > Oh, and we have the same problem for theming, not just fonts, since each port > > renders input elements differently, and those input elements are used in tests > > that don't really test input elements. > > So do you think I should refrain from committing the patch in > https://bugs.webkit.org/show_bug.cgi?id=38437 ? Hmm, can't see any patch in that bug. Also, our tab/whitespace behavior is a bit busted so if you're seeing different results that could be it. But in general, adding new results for Qt is great. I was just saying that long term we want to find something that scales better than adding new results for x number of platforms.
(In reply to comment #6) > > Hmm, can't see any patch in that bug. Also, our tab/whitespace behavior is a > bit busted so if you're seeing different results that could be it. > > But in general, adding new results for Qt is great. > > I was just saying that long term we want to find something that scales better > than adding new results for x number of platforms. Silly me - I meant https://bugs.webkit.org/show_bug.cgi?id=38435. Thanks for the clarification anyway.