Bug 55236 - [Chromium] .checksum files without .pngs and vice-versa
Summary: [Chromium] .checksum files without .pngs and vice-versa
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: All All
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-02-25 11:05 PST by Mihai Parparita
Modified: 2011-03-08 11:43 PST (History)
5 users (show)

See Also:


Attachments
find-mismatched-results.py (702 bytes, text/x-python-script)
2011-02-25 11:06 PST, Mihai Parparita
no flags Details
generate checksums from pngs (1.74 KB, text/plain)
2011-02-25 11:19 PST, Tony Chang
no flags Details
Patch (4.86 KB, patch)
2011-03-04 15:09 PST, Tony Chang
no flags Details | Formatted Diff | Diff
Patch (4.89 KB, patch)
2011-03-07 15:02 PST, Tony Chang
mihai: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Mihai Parparita 2011-02-25 11:05:03 PST
We seem to have a bunch of .checksum expectation files that don't have a .png (and vice-versa). This seems like a bad idea since it can lead to inconsistent/confusing results (e.g. .checksum mismatch even through the .pngs would match).

In directory LayoutTests/platform/chromium-mac/animations
  No PNG:
    state-at-end-event-expected
In directory LayoutTests/platform/chromium-mac/css2.1
  No checksum:
    t0905-c414-flt-01-d-g-expected
    t100801-c544-valgn-03-d-agi-expected
    t100801-c544-valgn-02-d-agi-expected
    t100801-c544-valgn-04-d-agi-expected
In directory LayoutTests/platform/chromium-mac/editing/deleting
  No PNG:
    5390681-2-expected
In directory LayoutTests/platform/chromium-mac/editing/execCommand
  No PNG:
    5481523-expected
    indent-selection-expected
    5138441-expected
In directory LayoutTests/platform/chromium-mac/editing/selection
  No checksum:
    replaced-boundaries-1-expected
    image-before-linebreak-expected
    extend-by-word-002-expected
    replaced-boundaries-2-expected
    4402375-expected
  No PNG:
    select-text-overflow-ellipsis-expected
    4960116-expected
In directory LayoutTests/platform/chromium-mac/fast/backgrounds
  No PNG:
    quirks-mode-line-box-backgrounds-expected
    svg-as-background-2-expected
In directory LayoutTests/platform/chromium-mac/fast/block/positioning
  No PNG:
    absolute-positioning-no-scrollbar-expected
In directory LayoutTests/platform/chromium-mac/fast/blockflow
  No checksum:
    background-vertical-rl-expected
    english-lr-text-expected
    japanese-rl-text-expected
In directory LayoutTests/platform/chromium-mac/fast/borders
  No PNG:
    borderRadiusSolid04-expected
    borderRadiusSolid03-expected
    borderRadiusGroove02-expected
    borderRadiusGroove01-expected
    borderRadiusOutset01-expected
    fieldsetBorderRadius-expected
    borderRadiusSolid01-expected
    borderRadiusDouble02-expected
    borderRadiusDouble03-expected
    borderRadiusArcs01-expected
    borderRadiusInset01-expected
    borderRadiusSolid02-expected
    borderRadiusRidge01-expected
In directory LayoutTests/platform/chromium-mac/fast/css
  No PNG:
    font-face-woff-expected
    css3-space-in-nth-and-lang-expected
    compare-content-style-expected
    apple-prefix-expected
In directory LayoutTests/platform/chromium-mac/fast/css/namespaces
  No PNG:
    namespaces-comments-expected
    namespaces-invalid-at-expected
    namespaces-empty-expected
    namespaces-escapes-expected
In directory LayoutTests/platform/chromium-mac/fast/dom/HTMLMeterElement
  No checksum:
    meter-appearances-capacity-expected
    meter-styles-expected
    meter-styles-changing-pseudo-expected
    meter-optimums-expected
    meter-element-expected
In directory LayoutTests/platform/chromium-mac/fast/forms
  No checksum:
    preserveFormDuringResidualStyle-expected
    input-appearance-bkcolor-expected
    input-align-image-expected
    input-type-change-expected
  No PNG:
    textarea-scrollbar-expected
In directory LayoutTests/platform/chromium-mac/fast/images
  No checksum:
    icon-decoding-expected
    imagemap-focus-ring-expected
In directory LayoutTests/platform/chromium-mac/fast/js
  No PNG:
    missing-title-end-tag-js-expected
In directory LayoutTests/platform/chromium-mac/fast/lists
  No PNG:
    anonymous-items-expected
    marker-before-empty-inline-expected
In directory LayoutTests/platform/chromium-mac/fast/media
  No checksum:
    media-query-invalid-value-expected
In directory LayoutTests/platform/chromium-mac/fast/multicol
  No PNG:
    column-count-with-rules-expected
In directory LayoutTests/platform/chromium-mac/fast/overflow
  No PNG:
    line-clamp-expected
In directory LayoutTests/platform/chromium-mac/fast/reflections
  No PNG:
    opacity-reflection-transform-expected
In directory LayoutTests/platform/chromium-mac/fast/replaced
  No checksum:
    image-tag-expected
In directory LayoutTests/platform/chromium-mac/fast/runin
  No PNG:
    generated3-expected
    generated4-expected
    generated2-expected
In directory LayoutTests/platform/chromium-mac/fast/text
  No PNG:
    stripNullFromText-expected
In directory LayoutTests/platform/chromium-mac/http/tests/misc
  No PNG:
    slow-loading-mask-expected
In directory LayoutTests/platform/chromium-mac/media
  No PNG:
    video-transformed-expected
In directory LayoutTests/platform/chromium-mac/scrollbars
  No checksum:
    overflow-scrollbar-combinations-expected
    listbox-scrollbar-combinations-expected
In directory LayoutTests/platform/chromium-mac/svg/batik/text
  No PNG:
    textProperties-expected
In directory LayoutTests/platform/chromium-mac/svg/css
  No checksum:
    text-gradient-shadow-expected
  No PNG:
    css-box-min-width-expected
In directory LayoutTests/platform/chromium-mac/svg/custom
  No PNG:
    fill-opacity-rgba-expected
    gradient-stop-corner-cases-expected
    container-opacity-clip-viewBox-expected
    fill-opacity-hsl-expected
    mask-invalidation-expected
    massive-coordinates-expected
    transformedMaskFails-expected
In directory LayoutTests/platform/chromium-mac/svg/filters
  No checksum:
    filterRes-expected
    filteredImage-expected
  No PNG:
    feColorMatrix-values-expected
In directory LayoutTests/platform/chromium-mac/svg/text
  No checksum:
    foreignObject-text-clipping-bug-expected
In directory LayoutTests/platform/chromium-mac/svg/W3C-SVG-1.1
  No checksum:
    struct-image-01-t-expected
In directory LayoutTests/platform/chromium-mac/tables/mozilla/bugs
  No checksum:
    bug12908-1-expected
    bug56563-expected
    bug10565-expected
    bug13169-expected
    bug4527-expected
    bug50695-2-expected
    bug4284-expected
    bug17138-expected
    bug120107-expected
    bug196870-expected
    bug12908-2-expected
    bug15544-expected
    bug1271-expected
    bug2981-2-expected
    bug6404-expected
    bug86708-expected
    bug4093-expected
    bug11026-expected
    bug1188-expected
    bug1318-expected
  No PNG:
    bug6674-expected
In directory LayoutTests/platform/chromium-mac/tables/mozilla/core
  No checksum:
    misc-expected
    col_widths_auto_autoFix-expected
In directory LayoutTests/platform/chromium-mac/tables/mozilla/marvin
  No checksum:
    th_valign_middle-expected
    thead_valign_baseline-expected
    tbody_valign_middle-expected
    tfoot_valign_baseline-expected
    thead_valign_middle-expected
    tr_valign_top-expected
    tbody_valign_bottom-expected
    td_valign_baseline-expected
    tr_valign_middle-expected
    tbody_valign_top-expected
    tbody_valign_baseline-expected
    th_valign_top-expected
    thead_valign_top-expected
    th_valign_baseline-expected
    td_valign_top-expected
    td_valign_middle-expected
    tfoot_valign_bottom-expected
    th_valign_bottom-expected
    td_valign_bottom-expected
    tr_valign_bottom-expected
    tfoot_valign_middle-expected
    tr_valign_baseline-expected
    thead_valign_bottom-expected
    tfoot_valign_top-expected
In directory LayoutTests/platform/chromium-mac/tables/mozilla_expected_failures/bugs
  No checksum:
    bug6933-expected
    97619-expected
In directory LayoutTests/platform/chromium-mac/transitions
  No PNG:
    transition-end-event-rendering-expected
In directory LayoutTests/platform/chromium-mac/webarchive
  No PNG:
    test-link-rel-icon-expected

In directory LayoutTests/platform/chromium-win/compositing/repaint
  No checksum:
    become-overlay-composited-layer-expected
In directory LayoutTests/platform/chromium-win/fast/backgrounds/size
  No PNG:
    backgroundSize03-expected
In directory LayoutTests/platform/chromium-win/fast/block/float
  No PNG:
    float-on-zero-height-line-expected
    br-with-clear-expected
    negative-margin-clear-expected
    009-expected
    overhanging-after-height-decrease-offsets-expected
    nowrap-clear-min-width-expected
    034-expected
    narrow-after-wide-expected
    4145535Crash-expected
In directory LayoutTests/platform/chromium-win/fast/block/positioning
  No PNG:
    010-expected
    003-expected
    038-expected
    015-expected
    018-expected
    049-expected
    029-expected
    030-expected
    relative-overconstrained-expected
    028-expected
    021-expected
    017-expected
    013-expected
    006-expected
    035-expected
    005-expected
    042-expected
    039-expected
    041-expected
    024-expected
    move-with-auto-width-expected
    044-expected
    033-expected
    023-expected
    025-expected
    027-expected
    020-expected
    032-expected
    036-expected
    016-expected
    050-expected
    009-expected
    045-expected
    008-expected
    007-expected
    043-expected
    048-expected
    062-expected
    019-expected
    014-expected
    004-expected
    012-expected
    022-expected
    040-expected
    037-expected
    026-expected
    034-expected
    011-expected
    031-expected
    046-expected
In directory LayoutTests/platform/chromium-win/fast/borders
  No PNG:
    outline-offset-min-assert-expected
In directory LayoutTests/platform/chromium-win/fast/canvas
  No PNG:
    canvas-transform-identity-expected
    canvas-transform-infinity-expected
    canvas-transform-multiply-expected
    canvas-transform-nan-expected
    canvas-transform-non-invertible-expected
In directory LayoutTests/platform/chromium-win/fast/clip
  No PNG:
    017-expected
In directory LayoutTests/platform/chromium-win/fast/css
  No PNG:
    last-child-style-sharing-expected
    transition-color-unspecified-expected
    content-dynamic-expected
    affected-by-hover-after-style-change-expected
    006-expected
In directory LayoutTests/platform/chromium-win/fast/dom
  No PNG:
    css-cached-import-rule-expected
    delete-contents-expected
    css-insert-import-rule-expected
In directory LayoutTests/platform/chromium-win/fast/dom/HTMLStyleElement
  No PNG:
    insert-parser-generated-expected
In directory LayoutTests/platform/chromium-win/fast/dynamic
  No PNG:
    staticY-expected
    positioned-movement-with-positioned-children-expected
    subtree-boundary-percent-height-expected
    001-expected
    floating-to-positioned-expected
    subtree-table-cell-height-expected
    005-expected
    float-withdrawal-2-expected
    float-no-longer-overhanging-expected
    genContentDestroyChildren-expected
    floating-to-positioned-2-expected
    subtree-no-common-root-static-y-expected
In directory LayoutTests/platform/chromium-win/fast/events
  No PNG:
    onloadFrameCrash-expected
In directory LayoutTests/platform/chromium-win/fast/forms
  No checksum:
    searchfield-heights-expected
In directory LayoutTests/platform/chromium-win/fast/frames
  No PNG:
    calculate-relative-expected
    empty-cols-attribute-expected
    empty-frame-src-expected
    002-expected
    frame-length-fractional-expected
    calculate-fixed-expected
    calculate-percentage-expected
    calculate-order-expected
    no-frame-borders-expected
    calculate-round-expected
    frame-set-whitespace-attributes-expected
In directory LayoutTests/platform/chromium-win/fast/images
  No PNG:
    pdf-as-tiled-background-expected
    svg-width-100p-as-background-expected
    svg-as-image-expected
    svg-as-relative-image-expected
    svg-as-background-expected
    image-in-map-expected
In directory LayoutTests/platform/chromium-win/fast/inline-block
  No PNG:
    005-expected
    004-expected
In directory LayoutTests/platform/chromium-win/fast/invalid
  No PNG:
    table-residual-style-crash-expected
    015-expected
In directory LayoutTests/platform/chromium-win/fast/js
  No PNG:
    missing-title-end-tag-js-expected
    missing-style-end-tag-js-expected
    exception-linenums-in-html-3-expected
In directory LayoutTests/platform/chromium-win/fast/layers
  No PNG:
    positioned-inside-root-with-margins-expected
    zindex-ridonkulous-expected
In directory LayoutTests/platform/chromium-win/fast/lists
  No PNG:
    outSideListMarkers-expected
In directory LayoutTests/platform/chromium-win/fast/media
  No checksum:
    media-query-invalid-value-expected
In directory LayoutTests/platform/chromium-win/fast/overflow
  No PNG:
    position-relative-expected
    unreachable-content-test-expected
In directory LayoutTests/platform/chromium-win/fast/parser
  No PNG:
    style-script-head-test-expected
In directory LayoutTests/platform/chromium-win/fast/reflections
  No PNG:
    table-cell-expected
In directory LayoutTests/platform/chromium-win/fast/repaint
  No PNG:
    button-spurious-layout-hint-expected
    float-in-new-block-with-layout-delta-expected
In directory LayoutTests/platform/chromium-win/fast/replaced
  No PNG:
    008-expected
In directory LayoutTests/platform/chromium-win/fast/table
  No PNG:
    vertical-align-baseline-readjust-expected
    growCellForImageQuirk-expected
    024-expected
    max-width-integer-overflow-expected
    percent-widths-stretch-expected
    giantCellspacing-expected
    031-expected
    caption-relayout-expected
    div-as-col-span-expected
In directory LayoutTests/platform/chromium-win/fast/text
  No PNG:
    align-center-rtl-spill-expected
    apply-start-width-after-skipped-text-expected
    selection-painted-separately-expected
    soft-hyphen-3-expected
    stripNullFromText-expected
    fixed-pitch-control-characters-expected
In directory LayoutTests/platform/chromium-win/fast/tokenizer
  No PNG:
    missing-style-end-tag-2-expected
    missing-style-end-tag-1-expected
    script-after-frameset-expected
In directory LayoutTests/platform/chromium-win/fast/xsl
  No PNG:
    xslt-mismatched-tags-in-xslt-expected
In directory LayoutTests/platform/chromium-win/printing
  No checksum:
    return-from-printing-mode-expected
In directory LayoutTests/platform/chromium-win/svg/custom
  No PNG:
    transform-ignore-after-invalid-expected
    use-recalcStyle-crash-expected
    inner-svg-hit-test-expected
    fill-update-expected
    inner-percent-expected
    transformedMaskFails-expected
    use-empty-reference-expected
    invalid-transforms-expected
    gradient-add-stops-expected
    clip-path-with-transform-expected
    create-metadata-element-expected
    percentage-rect2-expected
    visibility-override-expected
    polyline-invalid-points-expected
    resource-client-removal-expected
    visibility-override-mask-expected
    evt-onload-expected
    transform-scale-parse-expected
    image-with-aspect-ratio-stretch-expected
    viewport-update2-expected
    dynamic-viewBox-expected
    transform-with-ending-space-expected
    repaint-on-image-bounds-change-expected
    SVGRect-interface-expected
    mask-excessive-malloc-expected
    pattern-in-defs-expected
    viewport-no-width-height-expected
    viewport-clip-expected
    svg-absolute-children-expected
    scroll-hit-test-expected
    xhtml-no-svg-renderer-expected
    SVGNumber-interface-expected
    clip-path-href-changes-expected
    polyline-setattribute-points-null-expected
    SVGPoint-interface-expected
    xml-stylesheet-expected
    css-pixels-dpi-expected
    polyline-hittest-expected
    rgbcolor-syntax-expected
    gradient-attr-update-expected
    visibility-override-clip-expected
    viewport-update-expected
    hover-default-fill-expected
    clip-path-child-changes-expected
    rootelement-expected
    class-selector-expected
    transform-removeAttributeNS-expected
    gradient-stop-style-change-expected
    text-gradient-no-content-expected
    transform-invalid-expected
    display-none-expected
    prevent-default-expected
    sheet-title-expected
    filter-source-alpha-expected
    mask-inside-defs-expected
    createelement-expected
    conditional-processing-outside-switch-expected
    clip-path-units-changes-expected
    poly-identify-expected
    attribute-namespace-check-expected
    use-symbol-overflow-expected
In directory LayoutTests/platform/chromium-win/svg/hixie/dynamic
  No PNG:
    003-expected
    004-expected
    006-expected
In directory LayoutTests/platform/chromium-win/svg/hixie/error
  No PNG:
    007-expected
    001-expected
    015-expected
    014-test-expected
    016-expected
    006-expected
    009-expected
    008-expected
    014-expected
In directory LayoutTests/platform/chromium-win/svg/hixie/painting
  No PNG:
    001-expected
In directory LayoutTests/platform/chromium-win/svg/hixie/processing-model
  No PNG:
    005-expected
In directory LayoutTests/platform/chromium-win/svg/hixie/rendering-model
  No PNG:
    003a-expected
In directory LayoutTests/platform/chromium-win/svg/hixie/use
  No PNG:
    002-expected
    002-test-expected
    001-expected
In directory LayoutTests/platform/chromium-win/svg/hixie/viewbox
  No PNG:
    003-expected
    002-expected
    004-expected
    001-expected
In directory LayoutTests/platform/chromium-win/svg/W3C-SVG-1.1
  No PNG:
    struct-frag-01-t-expected
In directory LayoutTests/platform/chromium-win/svg/webarchive
  No PNG:
    svg-script-subresouces-expected
In directory LayoutTests/platform/chromium-win/tables/mozilla/bugs
  No PNG:
    bug222336-expected
    bug222467-expected
    bug8032-2-expected
    bug9024-expected
    bug269566-expected
    adforce_imgis_com-expected
    bug1818-6-expected
    bug277062-expected
    bug103533-expected
    bug2954-expected
    bug28933-expected
In directory LayoutTests/platform/chromium-win/tables/mozilla/core
  No PNG:
    col_widths_fix_autoFixPer-expected
In directory LayoutTests/platform/chromium-win/tables/mozilla_expected_failures/bugs
  No PNG:
    bug3517-expected
    bug128876-expected
In directory LayoutTests/platform/chromium-win/transitions
  No PNG:
    transition-end-event-rendering-expected
In directory LayoutTests/platform/chromium-win/webarchive
  No PNG:
    test-td-background-expected
    test-img-src-expected
    test-link-rel-icon-expected
    test-body-background-expected
    test-input-src-expected
    test-script-src-expected
    test-object-data-expected
    archive-with-unencoded-url-expected
    test-xml-stylesheet-expected
    test-table-background-expected


In directory LayoutTests/platform/chromium-linux/animations
  No PNG:
    suspend-transform-animation-expected
In directory LayoutTests/platform/chromium-linux/fast/block/float
  No checksum:
    clear-element-too-wide-for-containing-block-expected
In directory LayoutTests/platform/chromium-linux/fast/blockflow
  No checksum:
    english-bt-text-expected
    english-rl-text-expected
    english-lr-text-expected
In directory LayoutTests/platform/chromium-linux/fast/lists
  No PNG:
    outSideListMarkers-expected
In directory LayoutTests/platform/chromium-linux/fast/multicol
  No checksum:
    margin-collapse-expected
    border-padding-pagination-expected
In directory LayoutTests/platform/chromium-linux/svg/text
  No PNG:
    text-hkern-expected
    text-vkern-expected
In directory LayoutTests/platform/chromium-linux/tables/mozilla/bugs
  No checksum:
    bug16012-expected
In directory LayoutTests/platform/chromium-linux/transitions
  No PNG:
    suspend-transform-transition-expected
Comment 1 Mihai Parparita 2011-02-25 11:06:08 PST
Created attachment 83842 [details]
find-mismatched-results.py

Script that I used to find these
Comment 2 Tony Chang 2011-02-25 11:17:47 PST
I have a script that generates .checksum from .png files, but it's probably not useful.  Odds are if there's a png and no checksum, we should delete the png.

If there's a checksum and no png, we can probably run the rebaseline script to grab the pngs.

If there's a mismatch, we probably want to manually verify the differences.  The rebaseline script can still help with this.

A pre-submit hook would be useful for catching the missing file cases.
Comment 3 Tony Chang 2011-02-25 11:19:32 PST
Created attachment 83846 [details]
generate checksums from pngs

For reference, here's how to calculate the checksum in python.  I think it only works for win/linux since mac does something weird with opacity.
Comment 4 Tony Chang 2011-03-04 15:09:22 PST
Created attachment 84809 [details]
Patch
Comment 5 Tony Chang 2011-03-04 15:10:17 PST
(In reply to comment #4)
> Created an attachment (id=84809) [details]
> Patch

May as well get find-mismatched-results.py checked in.  This is based on Mihai's code, but I wanted something I could pipe to xargs rm.
Comment 6 Dirk Pranke 2011-03-07 14:43:26 PST
Comment on attachment 84809 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=84809&action=review

Looks fine otherwise.

> Tools/Scripts/find-mismatched-results:51
> +        option_parser.error("Please --missing-pngs or --missing-checksums.")

Nit. "Please specify either ..."

> Tools/Scripts/find-mismatched-results:55
> +            '..', '..', 'LayoutTests')]

Nit. Should replace this with a webkitpy routine to find the top of the tree. Unfortunately, common.checkout.scm.find_checkout_root() doesn't really do the right thing. layout_tests.port.get().path_from_webkit_base('LayoutTests') should, I think. Maybe add a FIXME?
Comment 7 Tony Chang 2011-03-07 15:02:37 PST
Created attachment 84985 [details]
Patch
Comment 8 Mihai Parparita 2011-03-07 15:09:11 PST
Comment on attachment 84985 [details]
Patch

Nit: The name find-mismatched-results is rather generic, maybe find-mismatched-layout-test-results?
Comment 9 Dirk Pranke 2011-03-07 15:11:56 PST
Comment on attachment 84985 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=84985&action=review

> Tools/ChangeLog:5
> +        [Chromium] .checksum files without .pngs and vice-versa

Nit (if it's not too late): "Find .checksum files ..." ?
Comment 10 Tony Chang 2011-03-07 15:41:03 PST
Committed r80504: <http://trac.webkit.org/changeset/80504>
Comment 11 Tony Chang 2011-03-07 15:43:56 PST
I'm going to go ahead and delete pngs without checksums (all ports) and try grabbing pngs from the bots for checksums without pngs (chromium ports only).

We should probably also have a presubmit check for this, but that can be tracked in a different bug.
Comment 12 Tony Chang 2011-03-08 11:43:32 PST
(In reply to comment #11)
> I'm going to go ahead and delete pngs without checksums (all ports) and try grabbing pngs from the bots for checksums without pngs (chromium ports only).

I deleted pngs without checksums, so `find-mismatched-layout-test-results --missing-checksums` should return nothing now.

I was unable to grab pngs from the bots because passing tests don't have archived results.  I went ahead and landed pngs for chromium-linux, but most of the missing pngs are in chromium-mac.