Bug 86956 – [EFL][DRT] Rebaseline test results for BUG 86355

• Component: WebKit EFL
• Status: RESOLVED FIXED

[Kangil Han]

I couldn't upload patch with all rebaselined results due to file size limitation in BUG 86355.
Therefore, rebaselining first here.

[Kangil Han]

Created attachment 142893 [details]
Patch

[Gyuyoung Kim]

I wanna verify this rebaseline is no problem on EFL release | debug buildbot before landing trunk. Kangil, I think you need to show test results to us.

[Raphael Kubo da Costa (:rakuco)]

Comment on attachment 142893 [details]
Patch

Looking at the PNGs, only compositing/iframes/iframe-copy-on-scroll.html seems to have regressed, but bug 79853 affects quite a few tests here and should be sorted out first.

[Kangil Han]

Here is result with BUG (86355 + 86956 + 86957)

=> Results: 22451/29690 tests passed (75.6%)

=> Tests to be fixed (7238):
      2 crashes                  ( 0.0%)
      2 tests timed out          ( 0.0%)
     46 no expected results found ( 0.6%)
    325 text diff mismatch       ( 4.5%)
     39 image mismatch           ( 0.5%)
   6824 skipped                  (94.3%)

=> Tests that will only be fixed if they crash (WONTFIX) (1):
      1 skipped                  (100.0%)

10:56:22.520 27332 29539 tests ran as expected, 151 didn't:
10:56:22.520 27332 

Expected to crash, but passed: (1)
  fast/loader/opaque-base-url.html

Expected to fail, but passed: (1)
  media/event-attributes.html


Unexpected flakiness: text diff mismatch (37)
  compositing/repaint/opacity-between-absolute.html = TEXT PASS
  fast/workers/storage/interrupt-database.html = TEXT PASS
  http/tests/appcache/access-via-redirect.php = TEXT PASS
  http/tests/appcache/auth.html = TEXT PASS
  http/tests/appcache/crash-when-navigating-away-then-back.html = TEXT PASS
  http/tests/appcache/credential-url.html = TEXT PASS
  http/tests/appcache/cyrillic-uri.html = TEXT PASS
  http/tests/appcache/different-https-origin-resource-main.html = TEXT PASS
  http/tests/appcache/different-scheme.html = TEXT PASS
  http/tests/appcache/empty-manifest.html = TEXT PASS
  http/tests/appcache/foreign-fallback.html = TEXT PASS
  http/tests/appcache/foreign-iframe-main.html = TEXT PASS
  http/tests/appcache/idempotent-update.html = TEXT PASS
  http/tests/appcache/interrupted-update.html = TEXT PASS
  http/tests/appcache/local-content.html = TEXT PASS
  http/tests/appcache/main-resource-hash.html = TEXT PASS
  http/tests/appcache/main-resource-redirect.html = TEXT PASS
  http/tests/appcache/manifest-parsing.html = TEXT PASS
  http/tests/appcache/manifest-with-empty-file.html = TEXT PASS
  http/tests/appcache/offline-access.html = TEXT PASS
  http/tests/appcache/online-whitelist.html = TEXT PASS
  http/tests/appcache/origin-quota-continued-download-multiple-manifests.html = TEXT PASS
  http/tests/appcache/origin-quota-continued-download.html = TEXT PASS
  http/tests/appcache/origin-quota.html = TEXT PASS
  http/tests/appcache/progress-counter.html = TEXT PASS
  http/tests/appcache/reload.html = TEXT PASS
  http/tests/appcache/remove-cache.html = TEXT PASS
  http/tests/appcache/simple.html = TEXT PASS
  http/tests/appcache/top-frame-1.html = TEXT PASS
  http/tests/appcache/top-frame-2.html = TEXT PASS
  http/tests/appcache/top-frame-3.html = TEXT PASS
  http/tests/appcache/top-frame-4.html = TEXT PASS
  http/tests/appcache/update-cache.html = TEXT PASS
  http/tests/appcache/video.html = TEXT PASS
  http/tests/appcache/whitelist-wildcard.html = TEXT PASS
  http/tests/appcache/wrong-content-type.html = TEXT PASS
  http/tests/appcache/xhr-foreign-resource.html = TEXT PASS

Unexpected flakiness: tests timed out (1)
  fast/workers/storage/use-same-database-in-page-and-workers.html = FAIL TIMEOUT PASS


Regressions: Unexpected text diff mismatch : (103)
  compositing/bounds-in-flipped-writing-mode.html = TEXT
  css3/selectors3/html/css3-modsel-23.html = TEXT
  css3/selectors3/html/css3-modsel-24.html = TEXT
  css3/selectors3/html/css3-modsel-68.html = TEXT
  css3/selectors3/html/css3-modsel-69.html = TEXT
  css3/selectors3/xhtml/css3-modsel-23.xml = TEXT
  css3/selectors3/xhtml/css3-modsel-24.xml = TEXT
  css3/selectors3/xhtml/css3-modsel-68.xml = TEXT
  css3/selectors3/xhtml/css3-modsel-69.xml = TEXT
  css3/selectors3/xml/css3-modsel-23.xml = TEXT
  css3/selectors3/xml/css3-modsel-24.xml = TEXT
  css3/selectors3/xml/css3-modsel-68.xml = TEXT
  css3/selectors3/xml/css3-modsel-69.xml = TEXT
  fast/backgrounds/background-shorthand-with-backgroundSize-style.html = TEXT
  fast/canvas/canvas-imageSmoothingEnabled.html = TEXT
  fast/css-grid-layout/display-grid-set-get.html = TEXT
  fast/css-grid-layout/grid-columns-rows-get-set-multiple.html = TEXT
  fast/css-grid-layout/grid-columns-rows-get-set.html = TEXT
  fast/css-grid-layout/grid-item-column-row-get-set.html = TEXT
  fast/css/getComputedStyle/computed-style-without-renderer.html = TEXT
  fast/css/input-search-padding.html = TEXT
  fast/css/line-height.html = TEXT
  fast/css/text-input-with-webkit-border-radius.html = TEXT
  fast/css/text-overflow-input.html = TEXT
  fast/dom/background-shorthand-csstext.html = TEXT
  fast/dom/isindex-001.html = TEXT
  fast/dom/isindex-002.html = TEXT
  fast/events/autoscroll.html = TEXT
  fast/forms/basic-inputs.html = TEXT
  fast/forms/box-shadow-override.html = TEXT
  fast/forms/control-restrict-line-height.html = TEXT
  fast/forms/encoding-test.html = TEXT
  fast/forms/fieldset-align.html = TEXT
  fast/forms/floating-textfield-relayout.html = TEXT
  fast/forms/form-hides-table.html = TEXT
  fast/forms/input-align.html = TEXT
  fast/forms/input-appearance-bkcolor.html = TEXT
  fast/forms/input-appearance-default-bkcolor.html = TEXT
  fast/forms/input-appearance-focus.html = TEXT
  fast/forms/input-appearance-height.html = TEXT
  fast/forms/input-appearance-preventDefault.html = TEXT
  fast/forms/input-appearance-selection.html = TEXT
  fast/forms/input-appearance-visibility.html = TEXT
  fast/forms/input-appearance-width.html = TEXT
  fast/forms/input-disabled-color.html = TEXT
  fast/forms/input-double-click-selection-gap-bug.html = TEXT
  fast/forms/input-field-text-truncated.html = TEXT
  fast/forms/input-readonly-dimmed.html = TEXT
  fast/forms/input-readonly-empty.html = TEXT
  fast/forms/input-spaces.html = TEXT
  fast/forms/input-table.html = TEXT
  fast/forms/input-text-click-inside.html = TEXT
  fast/forms/input-text-double-click.html = TEXT
  fast/forms/input-text-option-delete.html = TEXT
  fast/forms/input-text-self-emptying-click.html = TEXT
  fast/forms/input-text-word-wrap.html = TEXT
  fast/forms/input-type-text-min-width.html = TEXT
  fast/forms/input-value.html = TEXT
  fast/forms/input-width.html = TEXT
  fast/forms/minWidthPercent.html = TEXT
  fast/forms/number/input-appearance-number-rtl.html = TEXT
  fast/forms/number/input-appearance-spinbutton-disabled-readonly.html = TEXT
  fast/forms/number/input-appearance-spinbutton-layer.html = TEXT
  fast/forms/search-cancel-button-style-sharing.html = TEXT
  fast/forms/search-display-none-cancel-button.html = TEXT
  fast/forms/search-rtl.html = TEXT
  fast/forms/search-styled.html = TEXT
  fast/forms/search-vertical-alignment.html = TEXT
  fast/forms/searchfield-heights.html = TEXT
  fast/forms/tabbing-input-iframe.html = TEXT
  fast/forms/text-control-intrinsic-widths.html = TEXT
  fast/forms/text-style-color.html = TEXT
  fast/forms/textfield-focus-ring.html = TEXT
  fast/forms/textfield-outline.html = TEXT
  fast/forms/visual-hebrew-text-field.html = TEXT
  fast/frames/take-focus-from-iframe.html = TEXT
  fast/html/details-no-summary4.html = TEXT
  fast/html/details-open-javascript.html = TEXT
  fast/html/details-open2.html = TEXT
  fast/html/details-open4.html = TEXT
  fast/lists/dynamic-marker-crash.html = TEXT
  fast/loader/file-protocol-fragment.html = TEXT
  fast/parser/script-already-started-flag-in-fragment-parsing-mode.html = TEXT
  fast/repaint/renderer-destruction-by-invalidateSelection-crash.html = TEXT
  fast/repaint/search-field-cancel.html = TEXT
  fast/repaint/subtree-root-skipped.html = TEXT
  fast/replaced/replaced-breaking-mixture.html = TEXT
  fast/replaced/replaced-breaking.html = TEXT
  fast/replaced/table-replaced-element.html = TEXT
  fast/replaced/width100percent-textfield.html = TEXT
  fast/table/003.html = TEXT
  fast/table/colspanMinWidth.html = TEXT
  fast/table/form-with-table-style.html = TEXT
  fast/table/spanOverlapRepaint.html = TEXT
  fast/table/text-field-baseline.html = TEXT
  fast/text/textIteratorNilRenderer.html = TEXT
  http/tests/local/link-stylesheet-load-order-preload.html = TEXT
  http/tests/navigation/javascriptlink-frames.html = TEXT
  mathml/presentation/tables.xhtml = TEXT
  sputnik/Conformance/15_Native_Objects/15.8_Math/15.8.2/15.8.2.16_sin/S15.8.2.16_A7.html = TEXT
  sputnik/Conformance/15_Native_Objects/15.8_Math/15.8.2/15.8.2.18_tan/S15.8.2.18_A7.html = TEXT
  svg/css/stars-with-shadow.html = TEXT
  svg/hittest/svg-ellipse-non-scale-stroke.xhtml = TEXT

Regressions: Unexpected image mismatch : (4)
  fast/forms/input-textarea-padding-match.html = IMAGE
  fast/table/multiple-captions-crash3.html = IMAGE
  fast/table/table-column-generates-anonymous-table.html = IMAGE
  ietestcenter/css3/valuesandunits/units-000.htm = IMAGE

Regressions: Unexpected crashes : (2)
  fast/overflow/overflow-height-float-not-removed-crash2.html = CRASH
  svg/animations/svg-two-animate-elements-crash.svg = CRASH

Regressions: Unexpected no expected results found : (2)
  compositing/iframes/resizer.html = MISSING
  compositing/layer-creation/fixed-position-scroll.html = MISSING


------------------------------------------------------------------------------

[Kangil Han]

I will be back soon with regression check. :)

[Kangil Han]

gyuyoung, rakuco: I've also found some image result regression. :(
Possible solutions at this stage are:
  1. Do nothing till BUG 79853 being solved.
  2. Update rebaselined results that look fine and add remain cases to skipped expectation list.
Which one would you prefer?
nit, please ignore my #4, #5 comments that aren't informative.

[Raphael Kubo da Costa (:rakuco)]

CC'ing some of our Intel friends. I've reverted Ryuan's commit and the scrollbars should be OK now.

My idea is for you to send a new patch and for them to help test if they spot any regressions.

[Kangil Han]

(In reply to comment #7)
> CC'ing some of our Intel friends. I've reverted Ryuan's commit and the scrollbars should be OK now.
> 
> My idea is for you to send a new patch and for them to help test if they spot any regressions.

Okay, I will pull latest source code, check regression and upload results. :)

[Kangil Han]

I've found 114 regression cases with BUG 86355.
I will split expected results into 3 due to file size limitation.
:)

[Kangil Han]

Created attachment 143222 [details]
Patch

[Gyuyoung Kim]

I also think you need to verify whether this patch doesn't influence on layout test by intel guys.

[Kangil Han]

(In reply to comment #11)
> I also think you need to verify whether this patch doesn't influence on layout test by intel guys.

Sure, I will ask for help for it.

[Dominik Röttsches (drott)]

Kangil, 
a) these were generated on a 64bit system, right?
b) when you run $ run-webkit-tests --efl --debug 
   on your system (64bit, I hope) with your new baselines applied, 
   do you get to 0 failures?

[Kangil Han]

(In reply to comment #13)
> Kangil, 

FOA, thanks for asking me!

> a) these were generated on a 64bit system, right?

Sorry, nope.
But, I think it wouldn't be a problem since we use same expected result set for both 32bit and 64bit work environment. :)
Please let me know if mine wasn't correct.

> b) when you run $ run-webkit-tests --efl --debug 
>    on your system (64bit, I hope) with your new baselines applied, 
>    do you get to 0 failures?

Actually, it was not 0 failures w/o my patches.
Therefore, I did rebaselining only for the diffs happened from my patch.

Could you please help me to run layout test if regression happens?
You can simply get latest attachments from (BUG 86355 + BUG 86956 + BUG 86957).

[Kangil Han]

Created attachment 143283 [details]
patch

[Kangil Han]

Created attachment 143285 [details]
patch

[Kangil Han]

Created attachment 143286 [details]
patch

[Gyuyoung Kim]

If one of intel guys run layout test and there is no problem, I will set cq+

[Kangil Han]

gyuyoung: tmpsantos said, he could help me.
tmpsantos: Could you please let me know the result?

[Kangil Han]

*** Bug 86957 has been marked as a duplicate of this bug. ***

[Kangil Han]

Created attachment 143522 [details]
patch

[Gyuyoung Kim]

Comment on attachment 143522 [details]
patch

Intel guy checked this patch on buildbot.

[WebKit Review Bot]

Comment on attachment 143522 [details]
patch

Rejecting attachment 143522 [details] from commit-queue.

Failed to run "['/mnt/git/webkit-commit-queue/Tools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '-..." exit_code: 1

ERROR: /mnt/git/webkit-commit-queue/LayoutTests/ChangeLog neither lists a valid reviewer nor contains the string "Unreviewed" or "Rubber stamp" (case insensitive).

Full output: http://queues.webkit.org/results/12771217

[Gyuyoung Kim]

Comment on attachment 143522 [details]
patch

View in context: https://bugs.webkit.org/attachment.cgi?id=143522&action=review

> LayoutTests/ChangeLog:6
> +        Reviewed by NOBODY (OOPS!).

Please change this with Unreviewed.

[Kangil Han]

Created attachment 143539 [details]
patch

[WebKit Review Bot]

Comment on attachment 143539 [details]
patch

Clearing flags on attachment: 143539

Committed r118171: <http://trac.webkit.org/changeset/118171>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.