We seem to have a bunch of .checksum expectation files that don't have a .png (and vice-versa). This seems like a bad idea since it can lead to inconsistent/confusing results (e.g. .checksum mismatch even through the .pngs would match). In directory LayoutTests/platform/chromium-mac/animations No PNG: state-at-end-event-expected In directory LayoutTests/platform/chromium-mac/css2.1 No checksum: t0905-c414-flt-01-d-g-expected t100801-c544-valgn-03-d-agi-expected t100801-c544-valgn-02-d-agi-expected t100801-c544-valgn-04-d-agi-expected In directory LayoutTests/platform/chromium-mac/editing/deleting No PNG: 5390681-2-expected In directory LayoutTests/platform/chromium-mac/editing/execCommand No PNG: 5481523-expected indent-selection-expected 5138441-expected In directory LayoutTests/platform/chromium-mac/editing/selection No checksum: replaced-boundaries-1-expected image-before-linebreak-expected extend-by-word-002-expected replaced-boundaries-2-expected 4402375-expected No PNG: select-text-overflow-ellipsis-expected 4960116-expected In directory LayoutTests/platform/chromium-mac/fast/backgrounds No PNG: quirks-mode-line-box-backgrounds-expected svg-as-background-2-expected In directory LayoutTests/platform/chromium-mac/fast/block/positioning No PNG: absolute-positioning-no-scrollbar-expected In directory LayoutTests/platform/chromium-mac/fast/blockflow No checksum: background-vertical-rl-expected english-lr-text-expected japanese-rl-text-expected In directory LayoutTests/platform/chromium-mac/fast/borders No PNG: borderRadiusSolid04-expected borderRadiusSolid03-expected borderRadiusGroove02-expected borderRadiusGroove01-expected borderRadiusOutset01-expected fieldsetBorderRadius-expected borderRadiusSolid01-expected borderRadiusDouble02-expected borderRadiusDouble03-expected borderRadiusArcs01-expected borderRadiusInset01-expected borderRadiusSolid02-expected borderRadiusRidge01-expected In directory LayoutTests/platform/chromium-mac/fast/css No PNG: font-face-woff-expected css3-space-in-nth-and-lang-expected compare-content-style-expected apple-prefix-expected In directory LayoutTests/platform/chromium-mac/fast/css/namespaces No PNG: namespaces-comments-expected namespaces-invalid-at-expected namespaces-empty-expected namespaces-escapes-expected In directory LayoutTests/platform/chromium-mac/fast/dom/HTMLMeterElement No checksum: meter-appearances-capacity-expected meter-styles-expected meter-styles-changing-pseudo-expected meter-optimums-expected meter-element-expected In directory LayoutTests/platform/chromium-mac/fast/forms No checksum: preserveFormDuringResidualStyle-expected input-appearance-bkcolor-expected input-align-image-expected input-type-change-expected No PNG: textarea-scrollbar-expected In directory LayoutTests/platform/chromium-mac/fast/images No checksum: icon-decoding-expected imagemap-focus-ring-expected In directory LayoutTests/platform/chromium-mac/fast/js No PNG: missing-title-end-tag-js-expected In directory LayoutTests/platform/chromium-mac/fast/lists No PNG: anonymous-items-expected marker-before-empty-inline-expected In directory LayoutTests/platform/chromium-mac/fast/media No checksum: media-query-invalid-value-expected In directory LayoutTests/platform/chromium-mac/fast/multicol No PNG: column-count-with-rules-expected In directory LayoutTests/platform/chromium-mac/fast/overflow No PNG: line-clamp-expected In directory LayoutTests/platform/chromium-mac/fast/reflections No PNG: opacity-reflection-transform-expected In directory LayoutTests/platform/chromium-mac/fast/replaced No checksum: image-tag-expected In directory LayoutTests/platform/chromium-mac/fast/runin No PNG: generated3-expected generated4-expected generated2-expected In directory LayoutTests/platform/chromium-mac/fast/text No PNG: stripNullFromText-expected In directory LayoutTests/platform/chromium-mac/http/tests/misc No PNG: slow-loading-mask-expected In directory LayoutTests/platform/chromium-mac/media No PNG: video-transformed-expected In directory LayoutTests/platform/chromium-mac/scrollbars No checksum: overflow-scrollbar-combinations-expected listbox-scrollbar-combinations-expected In directory LayoutTests/platform/chromium-mac/svg/batik/text No PNG: textProperties-expected In directory LayoutTests/platform/chromium-mac/svg/css No checksum: text-gradient-shadow-expected No PNG: css-box-min-width-expected In directory LayoutTests/platform/chromium-mac/svg/custom No PNG: fill-opacity-rgba-expected gradient-stop-corner-cases-expected container-opacity-clip-viewBox-expected fill-opacity-hsl-expected mask-invalidation-expected massive-coordinates-expected transformedMaskFails-expected In directory LayoutTests/platform/chromium-mac/svg/filters No checksum: filterRes-expected filteredImage-expected No PNG: feColorMatrix-values-expected In directory LayoutTests/platform/chromium-mac/svg/text No checksum: foreignObject-text-clipping-bug-expected In directory LayoutTests/platform/chromium-mac/svg/W3C-SVG-1.1 No checksum: struct-image-01-t-expected In directory LayoutTests/platform/chromium-mac/tables/mozilla/bugs No checksum: bug12908-1-expected bug56563-expected bug10565-expected bug13169-expected bug4527-expected bug50695-2-expected bug4284-expected bug17138-expected bug120107-expected bug196870-expected bug12908-2-expected bug15544-expected bug1271-expected bug2981-2-expected bug6404-expected bug86708-expected bug4093-expected bug11026-expected bug1188-expected bug1318-expected No PNG: bug6674-expected In directory LayoutTests/platform/chromium-mac/tables/mozilla/core No checksum: misc-expected col_widths_auto_autoFix-expected In directory LayoutTests/platform/chromium-mac/tables/mozilla/marvin No checksum: th_valign_middle-expected thead_valign_baseline-expected tbody_valign_middle-expected tfoot_valign_baseline-expected thead_valign_middle-expected tr_valign_top-expected tbody_valign_bottom-expected td_valign_baseline-expected tr_valign_middle-expected tbody_valign_top-expected tbody_valign_baseline-expected th_valign_top-expected thead_valign_top-expected th_valign_baseline-expected td_valign_top-expected td_valign_middle-expected tfoot_valign_bottom-expected th_valign_bottom-expected td_valign_bottom-expected tr_valign_bottom-expected tfoot_valign_middle-expected tr_valign_baseline-expected thead_valign_bottom-expected tfoot_valign_top-expected In directory LayoutTests/platform/chromium-mac/tables/mozilla_expected_failures/bugs No checksum: bug6933-expected 97619-expected In directory LayoutTests/platform/chromium-mac/transitions No PNG: transition-end-event-rendering-expected In directory LayoutTests/platform/chromium-mac/webarchive No PNG: test-link-rel-icon-expected In directory LayoutTests/platform/chromium-win/compositing/repaint No checksum: become-overlay-composited-layer-expected In directory LayoutTests/platform/chromium-win/fast/backgrounds/size No PNG: backgroundSize03-expected In directory LayoutTests/platform/chromium-win/fast/block/float No PNG: float-on-zero-height-line-expected br-with-clear-expected negative-margin-clear-expected 009-expected overhanging-after-height-decrease-offsets-expected nowrap-clear-min-width-expected 034-expected narrow-after-wide-expected 4145535Crash-expected In directory LayoutTests/platform/chromium-win/fast/block/positioning No PNG: 010-expected 003-expected 038-expected 015-expected 018-expected 049-expected 029-expected 030-expected relative-overconstrained-expected 028-expected 021-expected 017-expected 013-expected 006-expected 035-expected 005-expected 042-expected 039-expected 041-expected 024-expected move-with-auto-width-expected 044-expected 033-expected 023-expected 025-expected 027-expected 020-expected 032-expected 036-expected 016-expected 050-expected 009-expected 045-expected 008-expected 007-expected 043-expected 048-expected 062-expected 019-expected 014-expected 004-expected 012-expected 022-expected 040-expected 037-expected 026-expected 034-expected 011-expected 031-expected 046-expected In directory LayoutTests/platform/chromium-win/fast/borders No PNG: outline-offset-min-assert-expected In directory LayoutTests/platform/chromium-win/fast/canvas No PNG: canvas-transform-identity-expected canvas-transform-infinity-expected canvas-transform-multiply-expected canvas-transform-nan-expected canvas-transform-non-invertible-expected In directory LayoutTests/platform/chromium-win/fast/clip No PNG: 017-expected In directory LayoutTests/platform/chromium-win/fast/css No PNG: last-child-style-sharing-expected transition-color-unspecified-expected content-dynamic-expected affected-by-hover-after-style-change-expected 006-expected In directory LayoutTests/platform/chromium-win/fast/dom No PNG: css-cached-import-rule-expected delete-contents-expected css-insert-import-rule-expected In directory LayoutTests/platform/chromium-win/fast/dom/HTMLStyleElement No PNG: insert-parser-generated-expected In directory LayoutTests/platform/chromium-win/fast/dynamic No PNG: staticY-expected positioned-movement-with-positioned-children-expected subtree-boundary-percent-height-expected 001-expected floating-to-positioned-expected subtree-table-cell-height-expected 005-expected float-withdrawal-2-expected float-no-longer-overhanging-expected genContentDestroyChildren-expected floating-to-positioned-2-expected subtree-no-common-root-static-y-expected In directory LayoutTests/platform/chromium-win/fast/events No PNG: onloadFrameCrash-expected In directory LayoutTests/platform/chromium-win/fast/forms No checksum: searchfield-heights-expected In directory LayoutTests/platform/chromium-win/fast/frames No PNG: calculate-relative-expected empty-cols-attribute-expected empty-frame-src-expected 002-expected frame-length-fractional-expected calculate-fixed-expected calculate-percentage-expected calculate-order-expected no-frame-borders-expected calculate-round-expected frame-set-whitespace-attributes-expected In directory LayoutTests/platform/chromium-win/fast/images No PNG: pdf-as-tiled-background-expected svg-width-100p-as-background-expected svg-as-image-expected svg-as-relative-image-expected svg-as-background-expected image-in-map-expected In directory LayoutTests/platform/chromium-win/fast/inline-block No PNG: 005-expected 004-expected In directory LayoutTests/platform/chromium-win/fast/invalid No PNG: table-residual-style-crash-expected 015-expected In directory LayoutTests/platform/chromium-win/fast/js No PNG: missing-title-end-tag-js-expected missing-style-end-tag-js-expected exception-linenums-in-html-3-expected In directory LayoutTests/platform/chromium-win/fast/layers No PNG: positioned-inside-root-with-margins-expected zindex-ridonkulous-expected In directory LayoutTests/platform/chromium-win/fast/lists No PNG: outSideListMarkers-expected In directory LayoutTests/platform/chromium-win/fast/media No checksum: media-query-invalid-value-expected In directory LayoutTests/platform/chromium-win/fast/overflow No PNG: position-relative-expected unreachable-content-test-expected In directory LayoutTests/platform/chromium-win/fast/parser No PNG: style-script-head-test-expected In directory LayoutTests/platform/chromium-win/fast/reflections No PNG: table-cell-expected In directory LayoutTests/platform/chromium-win/fast/repaint No PNG: button-spurious-layout-hint-expected float-in-new-block-with-layout-delta-expected In directory LayoutTests/platform/chromium-win/fast/replaced No PNG: 008-expected In directory LayoutTests/platform/chromium-win/fast/table No PNG: vertical-align-baseline-readjust-expected growCellForImageQuirk-expected 024-expected max-width-integer-overflow-expected percent-widths-stretch-expected giantCellspacing-expected 031-expected caption-relayout-expected div-as-col-span-expected In directory LayoutTests/platform/chromium-win/fast/text No PNG: align-center-rtl-spill-expected apply-start-width-after-skipped-text-expected selection-painted-separately-expected soft-hyphen-3-expected stripNullFromText-expected fixed-pitch-control-characters-expected In directory LayoutTests/platform/chromium-win/fast/tokenizer No PNG: missing-style-end-tag-2-expected missing-style-end-tag-1-expected script-after-frameset-expected In directory LayoutTests/platform/chromium-win/fast/xsl No PNG: xslt-mismatched-tags-in-xslt-expected In directory LayoutTests/platform/chromium-win/printing No checksum: return-from-printing-mode-expected In directory LayoutTests/platform/chromium-win/svg/custom No PNG: transform-ignore-after-invalid-expected use-recalcStyle-crash-expected inner-svg-hit-test-expected fill-update-expected inner-percent-expected transformedMaskFails-expected use-empty-reference-expected invalid-transforms-expected gradient-add-stops-expected clip-path-with-transform-expected create-metadata-element-expected percentage-rect2-expected visibility-override-expected polyline-invalid-points-expected resource-client-removal-expected visibility-override-mask-expected evt-onload-expected transform-scale-parse-expected image-with-aspect-ratio-stretch-expected viewport-update2-expected dynamic-viewBox-expected transform-with-ending-space-expected repaint-on-image-bounds-change-expected SVGRect-interface-expected mask-excessive-malloc-expected pattern-in-defs-expected viewport-no-width-height-expected viewport-clip-expected svg-absolute-children-expected scroll-hit-test-expected xhtml-no-svg-renderer-expected SVGNumber-interface-expected clip-path-href-changes-expected polyline-setattribute-points-null-expected SVGPoint-interface-expected xml-stylesheet-expected css-pixels-dpi-expected polyline-hittest-expected rgbcolor-syntax-expected gradient-attr-update-expected visibility-override-clip-expected viewport-update-expected hover-default-fill-expected clip-path-child-changes-expected rootelement-expected class-selector-expected transform-removeAttributeNS-expected gradient-stop-style-change-expected text-gradient-no-content-expected transform-invalid-expected display-none-expected prevent-default-expected sheet-title-expected filter-source-alpha-expected mask-inside-defs-expected createelement-expected conditional-processing-outside-switch-expected clip-path-units-changes-expected poly-identify-expected attribute-namespace-check-expected use-symbol-overflow-expected In directory LayoutTests/platform/chromium-win/svg/hixie/dynamic No PNG: 003-expected 004-expected 006-expected In directory LayoutTests/platform/chromium-win/svg/hixie/error No PNG: 007-expected 001-expected 015-expected 014-test-expected 016-expected 006-expected 009-expected 008-expected 014-expected In directory LayoutTests/platform/chromium-win/svg/hixie/painting No PNG: 001-expected In directory LayoutTests/platform/chromium-win/svg/hixie/processing-model No PNG: 005-expected In directory LayoutTests/platform/chromium-win/svg/hixie/rendering-model No PNG: 003a-expected In directory LayoutTests/platform/chromium-win/svg/hixie/use No PNG: 002-expected 002-test-expected 001-expected In directory LayoutTests/platform/chromium-win/svg/hixie/viewbox No PNG: 003-expected 002-expected 004-expected 001-expected In directory LayoutTests/platform/chromium-win/svg/W3C-SVG-1.1 No PNG: struct-frag-01-t-expected In directory LayoutTests/platform/chromium-win/svg/webarchive No PNG: svg-script-subresouces-expected In directory LayoutTests/platform/chromium-win/tables/mozilla/bugs No PNG: bug222336-expected bug222467-expected bug8032-2-expected bug9024-expected bug269566-expected adforce_imgis_com-expected bug1818-6-expected bug277062-expected bug103533-expected bug2954-expected bug28933-expected In directory LayoutTests/platform/chromium-win/tables/mozilla/core No PNG: col_widths_fix_autoFixPer-expected In directory LayoutTests/platform/chromium-win/tables/mozilla_expected_failures/bugs No PNG: bug3517-expected bug128876-expected In directory LayoutTests/platform/chromium-win/transitions No PNG: transition-end-event-rendering-expected In directory LayoutTests/platform/chromium-win/webarchive No PNG: test-td-background-expected test-img-src-expected test-link-rel-icon-expected test-body-background-expected test-input-src-expected test-script-src-expected test-object-data-expected archive-with-unencoded-url-expected test-xml-stylesheet-expected test-table-background-expected In directory LayoutTests/platform/chromium-linux/animations No PNG: suspend-transform-animation-expected In directory LayoutTests/platform/chromium-linux/fast/block/float No checksum: clear-element-too-wide-for-containing-block-expected In directory LayoutTests/platform/chromium-linux/fast/blockflow No checksum: english-bt-text-expected english-rl-text-expected english-lr-text-expected In directory LayoutTests/platform/chromium-linux/fast/lists No PNG: outSideListMarkers-expected In directory LayoutTests/platform/chromium-linux/fast/multicol No checksum: margin-collapse-expected border-padding-pagination-expected In directory LayoutTests/platform/chromium-linux/svg/text No PNG: text-hkern-expected text-vkern-expected In directory LayoutTests/platform/chromium-linux/tables/mozilla/bugs No checksum: bug16012-expected In directory LayoutTests/platform/chromium-linux/transitions No PNG: suspend-transform-transition-expected
Created attachment 83842 [details] find-mismatched-results.py Script that I used to find these
I have a script that generates .checksum from .png files, but it's probably not useful. Odds are if there's a png and no checksum, we should delete the png. If there's a checksum and no png, we can probably run the rebaseline script to grab the pngs. If there's a mismatch, we probably want to manually verify the differences. The rebaseline script can still help with this. A pre-submit hook would be useful for catching the missing file cases.
Created attachment 83846 [details] generate checksums from pngs For reference, here's how to calculate the checksum in python. I think it only works for win/linux since mac does something weird with opacity.
Created attachment 84809 [details] Patch
(In reply to comment #4) > Created an attachment (id=84809) [details] > Patch May as well get find-mismatched-results.py checked in. This is based on Mihai's code, but I wanted something I could pipe to xargs rm.
Comment on attachment 84809 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=84809&action=review Looks fine otherwise. > Tools/Scripts/find-mismatched-results:51 > + option_parser.error("Please --missing-pngs or --missing-checksums.") Nit. "Please specify either ..." > Tools/Scripts/find-mismatched-results:55 > + '..', '..', 'LayoutTests')] Nit. Should replace this with a webkitpy routine to find the top of the tree. Unfortunately, common.checkout.scm.find_checkout_root() doesn't really do the right thing. layout_tests.port.get().path_from_webkit_base('LayoutTests') should, I think. Maybe add a FIXME?
Created attachment 84985 [details] Patch
Comment on attachment 84985 [details] Patch Nit: The name find-mismatched-results is rather generic, maybe find-mismatched-layout-test-results?
Comment on attachment 84985 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=84985&action=review > Tools/ChangeLog:5 > + [Chromium] .checksum files without .pngs and vice-versa Nit (if it's not too late): "Find .checksum files ..." ?
Committed r80504: <http://trac.webkit.org/changeset/80504>
I'm going to go ahead and delete pngs without checksums (all ports) and try grabbing pngs from the bots for checksums without pngs (chromium ports only). We should probably also have a presubmit check for this, but that can be tracked in a different bug.
(In reply to comment #11) > I'm going to go ahead and delete pngs without checksums (all ports) and try grabbing pngs from the bots for checksums without pngs (chromium ports only). I deleted pngs without checksums, so `find-mismatched-layout-test-results --missing-checksums` should return nothing now. I was unable to grab pngs from the bots because passing tests don't have archived results. I went ahead and landed pngs for chromium-linux, but most of the missing pngs are in chromium-mac.