Bug 21821

Summary:

Redo all reference images

Product:

WebKit

Reporter:

Pierre-Olivier Latour <pol>

Component:

Tools / Tests

Assignee:

Nobody <webkit-unassigned>

Status:

RESOLVED FIXED

Severity:

Normal

CC:

mitz, simon.fraser

Priority:

Version:

528+ (Nightly build)

Hardware:

Mac

OS:

OS X 10.5

Bug Depends on:

21322

Bug Blocks:

21988

Attachments:

Description	Flags
Clean up of reference images	none
Fixed reference images on Tiger	none
Other Tiger patch	none
Revised other Tiger patch	none
Updated reference images for Leopard	none
Updated reference images for Tiger	none

Pierre-Olivier Latour

Reported 2008-10-22 19:48:03 PDT

Now that DumpRenderTree has been updated to really be efficient in dealing with pixel tests (see https://bugs.webkit.org/show_bug.cgi?id=21322), we need to redo all reference images.

Attachments
Clean up of reference images (96.47 KB, patch) 2008-10-28 10:07 PDT, Pierre-Olivier Latour	no flags	Details Formatted Diff Diff
Fixed reference images on Tiger (268.59 KB, patch) 2008-11-03 19:07 PST, Pierre-Olivier Latour	no flags	Details Formatted Diff Diff
Other Tiger patch (deleted) 2008-11-05 18:21 PST, Pierre-Olivier Latour	no flags	Details Formatted Diff Diff
Revised other Tiger patch (1.84 MB, patch) 2008-11-10 19:22 PST, Pierre-Olivier Latour	no flags	Details Formatted Diff Diff
Updated reference images for Leopard (774.89 KB, patch) 2008-11-11 17:26 PST, Pierre-Olivier Latour	no flags	Details Formatted Diff Diff
Updated reference images for Tiger (1.63 MB, patch) 2008-11-12 17:38 PST, Pierre-Olivier Latour	no flags	Details Formatted Diff Diff
Show Obsolete (1) View All Add attachment proposed patch, testcase, etc.

Pierre-Olivier Latour

Comment 1 2008-10-22 19:49:00 PDT

There is some useful info here: https://bugs.webkit.org/show_bug.cgi?id=9496

Pierre-Olivier Latour

Comment 2 2008-10-27 18:09:45 PDT

Setup: - iMac X86 (iMac8,1) + ATI HD 2600 Pro w/ stock 10.5.5 + QT 7.5.5 - WebKit SVN 37896 - Clean build in Release configuration WebKitTools/Scripts/run-webkit-tests --pixel --tolerance 0.0 LayoutTests/ 1017.00s total testing time 5826 test cases (58%) succeeded 4102 test cases (41%) had incorrect layout 2 test cases (<1%) crashed 28 test cases (<1%) had stderr output WebKitTools/Scripts/run-webkit-tests --pixel --tolerance 1.0 LayoutTests/ 954.02s total testing time 9712 test cases (97%) succeeded 216 test cases (2%) had incorrect layout 2 test cases (<1%) crashed 28 test cases (<1%) had stderr output Non pixel test failures: dom/html/level2/html/HTMLDocument12.html dom/xhtml/level2/html/HTMLDocument12.xhtml fast/cookies/local-file-can-set-cookies.html fast/profiler/anonymous-event-handler.html fast/profiler/anonymous-function-called-from-different-contexts.html fast/profiler/anonymous-function-calls-built-in-functions.html fast/profiler/anonymous-function-calls-eval.html fast/profiler/apply.html fast/profiler/built-in-function-calls-anonymous.html fast/profiler/built-in-function-calls-user-defined-function.html fast/profiler/call-nodelist-as-function.html fast/profiler/call.html fast/profiler/calling-the-function-that-started-the-profiler-from-another-scope.html fast/profiler/compare-multiple-profiles.html fast/profiler/constructor.html fast/profiler/dead-time.html fast/profiler/document-dot-write.html fast/profiler/event-handler.html fast/profiler/execution-context-and-eval-on-same-line.html fast/profiler/heavy-view.html fast/profiler/inline-event-handler.html fast/profiler/many-calls-in-the-same-scope.html fast/profiler/multiple-and-different-scoped-anonymous-function-calls.html fast/profiler/multiple-and-different-scoped-function-calls.html fast/profiler/multiple-frames.html fast/profiler/nested-anonymous-functon.html fast/profiler/nested-start-and-stop-profiler.html fast/profiler/one-execution-context.html fast/profiler/profile-calls-in-included-file.html fast/profiler/profiling-from-a-nested-location-but-stop-profiling-outside-the-nesting.html fast/profiler/profiling-from-a-nested-location.html fast/profiler/simple-event-call.html fast/profiler/simple-no-level-change.html fast/profiler/start-and-stop-profiler-multiple-times.html fast/profiler/stop-profiling-after-setTimeout.html fast/profiler/stop-then-function-call.html fast/profiler/throw-exception-from-eval.html fast/profiler/two-execution-contexts.html fast/profiler/user-defined-function-calls-built-in-functions.html fast/profiler/window-dot-eval.html Font rendering differences (acceptable failures): editing/pasteboard/block-wrappers-necessary.html fast/backgrounds/repeat/negative-offset-repeat-transformed.html fast/block/float/editable-text-overlapping-float.html fast/css/css3-nth-child.html fast/css/font-face-multiple-faces.html fast/dynamic/insert-before-table-part-in-continuation.html fast/frames/frame-scrolling-attribute.html fast/frames/iframe-scrolling-attribute.html fast/inline/inline-borders-with-bidi-override.html fast/inline/inline-text-quirk-bpm.html fast/repaint/line-flow-with-floats-1.html fast/repaint/line-flow-with-floats-2.html fast/repaint/line-flow-with-floats-3.html fast/repaint/line-flow-with-floats-4.html fast/repaint/line-flow-with-floats-5.html fast/repaint/line-flow-with-floats-8.html fast/repaint/line-flow-with-floats-9.html fast/repaint/line-flow-with-floats-10.html fast/table/frame-and-rules.html fast/text/atsui-spacing-features.html fast/text/capitalize-boundaries.html fast/text/trailing-white-space-2.html fast/text/trailing-white-space.html fast/text/international/complex-character-based-fallback.html platform/mac/fast/text/international/Geeza-Pro-vertical-metrics-adjustment.html scrollbars/overflow-scrollbar-combinations.html svg/W3C-SVG-1.1/animate-elem-04-t.svg svg/W3C-SVG-1.1/animate-elem-05-t.svg svg/W3C-SVG-1.1/animate-elem-06-t.svg svg/W3C-SVG-1.1/animate-elem-07-t.svg svg/W3C-SVG-1.1/animate-elem-09-t.svg svg/W3C-SVG-1.1/animate-elem-10-t.svg svg/W3C-SVG-1.1/animate-elem-11-t.svg svg/W3C-SVG-1.1/animate-elem-12-t.svg svg/W3C-SVG-1.1/animate-elem-14-t.svg svg/W3C-SVG-1.1/animate-elem-15-t.svg svg/W3C-SVG-1.1/animate-elem-16-t.svg svg/W3C-SVG-1.1/animate-elem-17-t.svg svg/W3C-SVG-1.1/animate-elem-18-t.svg svg/W3C-SVG-1.1/animate-elem-19-t.svg svg/W3C-SVG-1.1/animate-elem-31-t.svg svg/W3C-SVG-1.1/animate-elem-33-t.svg svg/W3C-SVG-1.1/animate-elem-34-t.svg svg/W3C-SVG-1.1/animate-elem-40-t.svg svg/W3C-SVG-1.1/animate-elem-44-t.svg svg/W3C-SVG-1.1/animate-elem-52-t.svg svg/W3C-SVG-1.1/animate-elem-80-t.svg svg/W3C-SVG-1.1/animate-elem-81-t.svg svg/W3C-SVG-1.1/animate-elem-82-t.svg svg/W3C-SVG-1.1/animate-elem-83-t.svg svg/W3C-SVG-1.1/coords-units-01-b.svg svg/W3C-SVG-1.1/coords-units-02-b.svg svg/W3C-SVG-1.1/coords-units-03-b.svg svg/W3C-SVG-1.1/coords-viewattr-03-b.svg svg/W3C-SVG-1.1/filters-color-01-b.svg svg/W3C-SVG-1.1/filters-composite-02-b.svg svg/W3C-SVG-1.1/filters-comptran-01-b.svg svg/W3C-SVG-1.1/filters-diffuse-01-f.svg svg/W3C-SVG-1.1/filters-light-01-f.svg svg/W3C-SVG-1.1/filters-specular-01-f.svg svg/W3C-SVG-1.1/filters-turb-01-f.svg svg/W3C-SVG-1.1/fonts-kern-01-t.svg svg/W3C-SVG-1.1/interact-events-01-b.svg svg/W3C-SVG-1.1/interact-order-01-b.svg svg/W3C-SVG-1.1/interact-order-02-b.svg svg/W3C-SVG-1.1/interact-order-03-b.svg svg/W3C-SVG-1.1/linking-uri-01-b.svg svg/W3C-SVG-1.1/linking-uri-02-b.svg svg/W3C-SVG-1.1/masking-opacity-01-b.svg svg/W3C-SVG-1.1/masking-path-01-b.svg svg/W3C-SVG-1.1/masking-path-02-b.svg svg/W3C-SVG-1.1/masking-path-03-b.svg svg/W3C-SVG-1.1/painting-fill-01-t.svg svg/W3C-SVG-1.1/painting-fill-02-t.svg svg/W3C-SVG-1.1/painting-fill-03-t.svg svg/W3C-SVG-1.1/painting-stroke-01-t.svg svg/W3C-SVG-1.1/painting-stroke-02-t.svg svg/W3C-SVG-1.1/painting-stroke-03-t.svg svg/W3C-SVG-1.1/painting-stroke-04-t.svg svg/W3C-SVG-1.1/paths-data-10-t.svg svg/W3C-SVG-1.1/pservers-grad-04-b.svg svg/W3C-SVG-1.1/pservers-grad-06-b.svg svg/W3C-SVG-1.1/pservers-grad-07-b.svg svg/W3C-SVG-1.1/pservers-grad-08-b.svg svg/W3C-SVG-1.1/pservers-grad-09-b.svg svg/W3C-SVG-1.1/pservers-grad-12-b.svg svg/W3C-SVG-1.1/pservers-grad-14-b.svg svg/W3C-SVG-1.1/script-handle-02-b.svg svg/W3C-SVG-1.1/script-handle-03-b.svg svg/W3C-SVG-1.1/script-handle-04-b.svg svg/W3C-SVG-1.1/struct-dom-03-b.svg svg/W3C-SVG-1.1/struct-dom-04-b.svg svg/W3C-SVG-1.1/struct-dom-05-b.svg svg/W3C-SVG-1.1/struct-group-03-t.svg svg/W3C-SVG-1.1/struct-use-01-t.svg svg/W3C-SVG-1.1/styling-css-02-b.svg svg/W3C-SVG-1.1/text-align-01-b.svg svg/W3C-SVG-1.1/text-align-02-b.svg svg/W3C-SVG-1.1/text-align-03-b.svg svg/W3C-SVG-1.1/text-align-04-b.svg svg/W3C-SVG-1.1/text-align-05-b.svg svg/W3C-SVG-1.1/text-align-06-b.svg svg/W3C-SVG-1.1/text-altglyph-01-b.svg svg/W3C-SVG-1.1/text-fonts-01-t.svg svg/W3C-SVG-1.1/text-fonts-02-t.svg svg/W3C-SVG-1.1/text-intro-05-t.svg svg/W3C-SVG-1.1/text-path-01-b.svg svg/W3C-SVG-1.1/text-spacing-01-b.svg svg/W3C-SVG-1.1/text-text-01-b.svg svg/W3C-SVG-1.1/text-text-03-b.svg svg/W3C-SVG-1.1/text-tselect-01-b.svg svg/W3C-SVG-1.1/text-tselect-02-f.svg svg/W3C-SVG-1.1/text-tspan-01-b.svg svg/W3C-SVG-1.1/text-ws-01-t.svg svg/W3C-SVG-1.1/text-ws-02-t.svg svg/W3C-SVG-1.1/types-basicDOM-01-b.svg svg/batik/text/longTextOnPath.svg svg/batik/text/textProperties.svg svg/batik/text/textStyles.svg svg/batik/text/xmlSpace.svg svg/custom/altglyph.svg svg/custom/dynamic-svg-document-creation.svg svg/custom/pointer-events-path.svg svg/custom/scrolling-embedded-svg-file-image-repaint-problem.html svg/custom/text-whitespace-handling.svg svg/text/kerning.svg svg/text/multichar-glyph.svg svg/text/text-align-02-b.svg svg/text/text-fonts-01-t.svg svg/text/text-path-01-b.svg svg/text/text-spacing-01-b.svg svg/text/text-text-01-b.svg svg/text/text-text-03-b.svg svg/text/text-tselect-01-b.svg svg/text/text-tspan-01-b.svg svg/text/text-ws-01-t.svg svg/text/text-ws-02-t.svg tables/mozilla/core/bloomberg.html tables/mozilla_expected_failures/bugs/bug2479-5.html Reference image at wrong location (move to platform/mac): invalid-percentage-property-expected.png lang-inheritance-expected.png lang-inheritance2-expected.png lang-vs-xml-lang-expected.png lang-vs-xml-lang-xhtml-expected.png text-align-expected.png paste-text-013-expected.png Reference images needs to be deleted: 5607069-1-expected.png Reference image not up-to-date: css2.1/t1202-counter-09-b.html css2.1/t1202-counters-09-b.html fast/flexbox/016.html fast/images/pdf-as-background.html editing/style/5279521.html fast/frames/001.html fast/inline/continuation-outlines-with-layers.html fast/inline/continuation-outlines.html fast/parser/broken-comments-vs-parsing-mode.html fast/text/international/thai-line-breaks.html fast/text/whitespace/013.html fast/xsl/document-function.xml fast/xsl/xslt-relative-path.xml tables/mozilla/bugs/bug2973.html SVG not rendering (https://bugs.webkit.org/show_bug.cgi?id=21910): fast/backgrounds/svg-as-background-1.html fast/backgrounds/svg-as-background-2.html fast/backgrounds/svg-as-background-3.html fast/backgrounds/svg-as-background-5.html fast/backgrounds/svg-as-background-6.html fast/backgrounds/svg-as-mask.html fast/borders/svg-as-border-image-2.html fast/borders/svg-as-border-image.html fast/images/svg-as-background.html fast/images/svg-as-image.html fast/images/svg-as-relative-image.html fast/images/svg-as-tiled-background.html No video images (https://bugs.webkit.org/show_bug.cgi?id=21911): media/video-aspect-ratio.html media/video-controls-rendering.html media/video-display-toggle.html media/video-layer-crash.html media/video-transformed.html Possible Flash failure (https://bugs.webkit.org/show_bug.cgi?id=21912): fast/dynamic/flash-replacement-test.html Repaint failure (https://bugs.webkit.org/show_bug.cgi?id=21913): fast/repaint/body-background-image.html Excessive repainting (https://bugs.webkit.org/show_bug.cgi?id=21914): fast/repaint/float-overflow-right.html fast/repaint/float-overflow.html Invalid image tiling (https://bugs.webkit.org/show_bug.cgi?id=21915): fast/backgrounds/size/backgroundSize19.html Test doesn't repaint entire view so result is corrupted by previous test (https://bugs.webkit.org/show_bug.cgi?id=21916): tables/mozilla_expected_failures/bugs/bug178855.xml

Pierre-Olivier Latour

Comment 3 2008-10-27 18:37:31 PDT

(In reply to comment #2) > Reference image at wrong location (move to platform/mac): > invalid-percentage-property-expected.png > lang-inheritance-expected.png > lang-inheritance2-expected.png > lang-vs-xml-lang-expected.png > lang-vs-xml-lang-xhtml-expected.png > text-align-expected.png > paste-text-013-expected.png Actually, "paste-text-013-expected.png" is in 2 places, so the one not in platform simply needs to be deleted

Pierre-Olivier Latour

Comment 4 2008-10-28 10:07:38 PDT

Created attachment 24720 [details] Clean up of reference images This patch takes care of the first set of issues: ensure all bases are in the right location and delete the ones that aren't needed.

mitz

Comment 5 2008-10-28 10:11:19 PDT

Comment on attachment 24720 [details] Clean up of reference images r=me There's a tab character in the change log but I will fix that when committing the patch.

mitz

Comment 6 2008-10-28 10:17:42 PDT

Comment on attachment 24720 [details] Clean up of reference images Clearing the review flag after committing this in r37927.

Pierre-Olivier Latour

Comment 7 2008-10-30 14:37:37 PDT

First set of new image bases was committed here: http://trac.webkit.org/changeset/38008

Pierre-Olivier Latour

Comment 8 2008-10-30 14:43:41 PDT

(In reply to comment #7) > First set of new image bases was committed here: > http://trac.webkit.org/changeset/38008 The re-generated bases come from the same iMac using a release build of WebKit #r38003: LayoutTests/svg/custom/ LayoutTests/fast/borders/ LayoutTests/fast/css/

Pierre-Olivier Latour

Comment 9 2008-10-31 16:17:45 PDT

Since the bases were generated on 37896, there was one new base checked in independently: http://trac.webkit.org/changeset/37967/trunk/LayoutTests

Pierre-Olivier Latour

Comment 10 2008-11-03 18:55:10 PST

Important note on tolerances: to avoid failures due to slight font rendering differences or CG dithering on various HW, pixel tests are expected to be run with a small tolerance: - Leopard: 0.1% - Tiger: 1.0%

Pierre-Olivier Latour

Comment 11 2008-11-03 19:07:16 PST

Created attachment 24880 [details] Fixed reference images on Tiger Fixed reference images on Tiger for: LayoutTests/svg/custom/ LayoutTests/fast/borders/ LayoutTests/fast/css/

mitz

Comment 12 2008-11-04 10:48:02 PST

Comment on attachment 24880 [details] Fixed reference images on Tiger r=me I am going to tweak the change log a bit when landing this.

mitz

Comment 13 2008-11-04 10:58:39 PST

Comment on attachment 24880 [details] Fixed reference images on Tiger Landed as <http://trac.webkit.org/changeset/38103>. Clearing the review flag.

mitz

Comment 14 2008-11-05 01:35:29 PST

More updated expected results landed in <http://trac.webkit.org/changeset/38121> and <http://trac.webkit.org/changeset/38122>, including accessibility/, animations/, css1/, css2.1/, css3/, dom/, editing/, fast/ and svg/.

Pierre-Olivier Latour

Comment 15 2008-11-05 18:21:42 PST

Created attachment 24931 [details] Other Tiger patch Fixed more reference images on Tiger

Pierre-Olivier Latour

Comment 16 2008-11-05 18:23:32 PST

Note that once all new bases are checked in and both Tiger and Leopard are running fine, we need to check in "exception" bases for tests that fail on Tiger with a difference > 1.0% but are still valid.

mitz

Comment 17 2008-11-09 22:18:42 PST

Comment on attachment 24931 [details] Other Tiger patch + - part of https://bugs.webkit.org/show_bug.cgi?id=21821 + Removed pixel results in platform/mac-tiger that were only slightly different + (within 1% tolerance) from the platform/mac results. + Removed results in platform/mac-tiger where the results in platform/mac were + also correct for Tiger. + The patch also modifies several files (I think just updating images to match the correct render trees already checked in) and you should mention that in the change log. It also adds files. I don't understand why: A LayoutTests/platform/mac-tiger/fast/text/international/complex-character-based-fallback-expected.txt A LayoutTests/platform/mac-tiger/fast/text/international/hindi-spacing-expected.txt

Pierre-Olivier Latour

Comment 18 2008-11-10 19:07:04 PST

(In reply to comment #9) > Since the bases were generated on 37896, there was one new base checked in > independently: > http://trac.webkit.org/changeset/37967/trunk/LayoutTests http://trac.webkit.org/changeset/38237/trunk/LayoutTests http://trac.webkit.org/changeset/38273/trunk/LayoutTests

Pierre-Olivier Latour

Comment 19 2008-11-10 19:22:58 PST

Created attachment 25040 [details] Revised other Tiger patch Updated patch following discussion with Dan

mitz

Comment 20 2008-11-11 09:33:34 PST

Comment on attachment 25040 [details] Revised other Tiger patch rs=me

Pierre-Olivier Latour

Comment 21 2008-11-11 10:19:31 PST

Last round of check in for new bases: http://trac.webkit.org/changeset/38297/trunk/LayoutTests

mitz

Comment 22 2008-11-11 10:27:45 PST

Comment on attachment 25040 [details] Revised other Tiger patch Landed this in r38305.

Pierre-Olivier Latour

Comment 23 2008-11-11 17:26:39 PST

Created attachment 25080 [details] Updated reference images for Leopard

mitz

Comment 24 2008-11-12 10:34:42 PST

Comment on attachment 25080 [details] Updated reference images for Leopard r=me!

mitz

Comment 25 2008-11-12 10:35:53 PST

Comment on attachment 25080 [details] Updated reference images for Leopard Landed in r38335

Pierre-Olivier Latour

Comment 26 2008-11-12 17:38:01 PST

Created attachment 25110 [details] Updated reference images for Tiger

mitz

Comment 27 2008-11-12 17:40:41 PST

Comment on attachment 25110 [details] Updated reference images for Tiger r=me

mitz

Comment 28 2008-11-12 17:41:41 PST

Comment on attachment 25110 [details] Updated reference images for Tiger Landed in r38354.

Pierre-Olivier Latour

Comment 29 2008-11-12 17:50:47 PST

We're finally done here!

Note You need to log in before you can comment on or make changes to this bug.