Bug 63186 - svg/zoom/page/zoom-svg-through-object-with-*.xhtml are flaky
Summary: svg/zoom/page/zoom-svg-through-object-with-*.xhtml are flaky
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: All All
: P2 Normal
Assignee: Adam Klein
URL:
Keywords: LayoutTestFailure
: 61664 61757 61918 62023 62143 62555 62560 62563 (view as bug list)
Depends on: 71673
Blocks:
  Show dependency treegraph
 
Reported: 2011-06-22 14:45 PDT by Adam Roben (:aroben)
Modified: 2011-11-22 15:09 PST (History)
12 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Adam Roben (:aroben) 2011-06-22 14:45:32 PDT
The svg/zoom/page/zoom-svg-through-object-with-*.xhtml tests are flaky. See dupes for examples.
Comment 1 Adam Roben (:aroben) 2011-06-22 14:46:17 PDT
*** Bug 61664 has been marked as a duplicate of this bug. ***
Comment 2 Adam Roben (:aroben) 2011-06-22 14:46:20 PDT
*** Bug 61757 has been marked as a duplicate of this bug. ***
Comment 3 Adam Roben (:aroben) 2011-06-22 14:46:22 PDT
*** Bug 61918 has been marked as a duplicate of this bug. ***
Comment 4 Adam Roben (:aroben) 2011-06-22 14:46:24 PDT
*** Bug 62023 has been marked as a duplicate of this bug. ***
Comment 5 Adam Roben (:aroben) 2011-06-22 14:46:27 PDT
*** Bug 62555 has been marked as a duplicate of this bug. ***
Comment 6 Adam Roben (:aroben) 2011-06-22 14:47:00 PDT
*** Bug 62558 has been marked as a duplicate of this bug. ***
Comment 7 Adam Roben (:aroben) 2011-06-22 14:47:16 PDT
*** Bug 62560 has been marked as a duplicate of this bug. ***
Comment 8 Adam Roben (:aroben) 2011-06-22 14:47:20 PDT
*** Bug 62563 has been marked as a duplicate of this bug. ***
Comment 9 Adam Roben (:aroben) 2011-06-22 15:02:47 PDT
Disabled the tests in r89484.
Comment 10 Nikolas Zimmermann 2011-06-23 03:01:31 PDT
What a pity, these tests are pretty important :(
I haven't been able to reproduce this on my SL machines so far, I'll try --random -p runs soon, maybe I can finally see the problem on my own...
Comment 11 Nikolas Zimmermann 2011-06-25 02:54:22 PDT
(In reply to comment #10)
> What a pity, these tests are pretty important :(
> I haven't been able to reproduce this on my SL machines so far, I'll try --random -p runs soon, maybe I can finally see the problem on my own...

Some results:

run-webkit-tests --tolerance 0 -p --random svg/zoom/page svg/zoom/page....
No errors.

run-webkit-tests --tolerance 0 -p --iterations 20 svg/zoom/page svg/...
No errors.

Also running without pixel tests makes no difference for me :(

Note: I moved the foo-disabled files to foo before running the tests, so I am sure they are enabled.
I've also tried -1 (singly) runs, w/o luck in reproducing.
Comment 12 Adam Roben (:aroben) 2011-06-26 11:12:02 PDT
Perhaps some earlier test affects these ones? Then the tests would pass or fail depending on exactly when DRT is relaunched (which is dependent on which tests crash, how many tests are run in total, etc.).
Comment 13 Nikolas Zimmermann 2011-06-27 08:41:40 PDT
(In reply to comment #12)
> Perhaps some earlier test affects these ones? Then the tests would pass or fail depending on exactly when DRT is relaunched (which is dependent on which tests crash, how many tests are run in total, etc.).

I'll try to debug more tomorrow.
Comment 14 Nikolas Zimmermann 2011-07-26 05:49:10 PDT
The root of all evil is probably the bug described in 64974. I'm working on a proper fix.
Comment 15 Nikolas Zimmermann 2011-08-06 08:23:44 PDT
(In reply to comment #14)
> The root of all evil is probably the bug described in 64974. I'm working on a proper fix.

The fix landed in r92545. It's better now, but there's still a problem with the page zoom tests in general, seems easy to trigger on non-mac eg gtk/win.

I'm testing following, as it seems to be a problem that we call notifyDone() right after the zooming.
-            layoutTestController.notifyDone();
+            setTimeout(function() { layoutTestController.notifyDone(); }, 0);

Let's see.
Comment 16 Andrei Popescu 2011-08-08 07:49:11 PDT
(In reply to comment #15)
> (In reply to comment #14)
> > The root of all evil is probably the bug described in 64974. I'm working on a proper fix.
> 
> The fix landed in r92545. It's better now, but there's still a problem with the page zoom tests in general, seems easy to trigger on non-mac eg gtk/win.
> 
> I'm testing following, as it seems to be a problem that we call notifyDone() right after the zooming.
> -            layoutTestController.notifyDone();
> +            setTimeout(function() { layoutTestController.notifyDone(); }, 0);
> 
> Let's see.

FYI, these tests currently fail on Chromium too.
Comment 17 Nikolas Zimmermann 2011-08-08 08:31:50 PDT
(In reply to comment #16)
> (In reply to comment #15)
> > (In reply to comment #14)
> > > The root of all evil is probably the bug described in 64974. I'm working on a proper fix.
> > 
> > The fix landed in r92545. It's better now, but there's still a problem with the page zoom tests in general, seems easy to trigger on non-mac eg gtk/win.
> > 
> > I'm testing following, as it seems to be a problem that we call notifyDone() right after the zooming.
> > -            layoutTestController.notifyDone();
> > +            setTimeout(function() { layoutTestController.notifyDone(); }, 0);
> > 
> > Let's see.
> 
> FYI, these tests currently fail on Chromium too.
You mean they're flakey or fail every time? Or do they just need a re-baseline?
Comment 18 Andrei Popescu 2011-08-08 08:39:10 PDT
(In reply to comment #17)
> (In reply to comment #16)
> > (In reply to comment #15)
> > > (In reply to comment #14)
> > > > The root of all evil is probably the bug described in 64974. I'm working on a proper fix.
> > > 
> > > The fix landed in r92545. It's better now, but there's still a problem with the page zoom tests in general, seems easy to trigger on non-mac eg gtk/win.
> > > 
> > > I'm testing following, as it seems to be a problem that we call notifyDone() right after the zooming.
> > > -            layoutTestController.notifyDone();
> > > +            setTimeout(function() { layoutTestController.notifyDone(); }, 0);
> > > 
> > > Let's see.
> > 
> > FYI, these tests currently fail on Chromium too.
> You mean they're flakey or fail every time? Or do they just need a re-baseline?

They fail every time. I am not sure whether they need a re-baseline or not, the actual results look slightly suspicious:

http://test-results.appspot.com/dashboards/flakiness_dashboard.html#showExpectations=true&tests=svg%2Fzoom%2Fpage%2Fzoom-svg-through-object-with-absolute-size-2.xhtml
Comment 19 Nikolas Zimmermann 2011-08-09 01:22:36 PDT
(In reply to comment #18)
> They fail every time. I am not sure whether they need a re-baseline or not, the actual results look slightly suspicious:
> 
> http://test-results.appspot.com/dashboards/flakiness_dashboard.html#showExpectations=true&tests=svg%2Fzoom%2Fpage%2Fzoom-svg-through-object-with-absolute-size-2.xhtml

This is a known problem on Chromium (the extra scrollbar).
When these svg/zoom/page/zoom-svg-through-object* tests were landed the first time, they were flakey on at least Win/Chromium/Gtk, producing extra-scrollbars on Chromium all the time, but never on other platforms.

The tests should now no longer be flakey - the extra scrollbar problem on chromium remains though.
We need someone with Chromium to debug this, I fear.

Could you rebaseline them for now, to see whether they are stable at least now on Chromium?
Comment 20 Andrei Popescu 2011-08-09 05:39:13 PDT
(In reply to comment #19)
> (In reply to comment #18)
> > They fail every time. I am not sure whether they need a re-baseline or not, the actual results look slightly suspicious:
> > 
> > http://test-results.appspot.com/dashboards/flakiness_dashboard.html#showExpectations=true&tests=svg%2Fzoom%2Fpage%2Fzoom-svg-through-object-with-absolute-size-2.xhtml
> 
> This is a known problem on Chromium (the extra scrollbar).
> When these svg/zoom/page/zoom-svg-through-object* tests were landed the first time, they were flakey on at least Win/Chromium/Gtk, producing extra-scrollbars on Chromium all the time, but never on other platforms.
> 
> The tests should now no longer be flakey - the extra scrollbar problem on chromium remains though.
> We need someone with Chromium to debug this, I fear.
> 
> Could you rebaseline them for now, to see whether they are stable at least now on Chromium?

Thanks for the background! I have filed:

http://code.google.com/p/chromium/issues/detail?id=92037

and Mike Reed was kind enough to volunteer looking into the issue. Mike, would you be ok with me rebaselining the tests for now?
Comment 21 Adam Roben (:aroben) 2011-08-17 07:54:34 PDT
*** Bug 62143 has been marked as a duplicate of this bug. ***
Comment 22 James Robinson 2011-09-02 11:04:46 PDT
They are not stable, the tests are still flaky on all chromium platforms.  The text output differs from run to run.
Comment 23 James Robinson 2011-09-02 11:04:58 PDT
Example test failure (from a mac):

--- /b/build/slave/Webkit_Mac10_6__CG__dbg_/build/layout-test-results/svg/zoom/page/zoom-svg-through-object-with-percentage-size-expected.txt 
+++ /b/build/slave/Webkit_Mac10_6__CG__dbg_/build/layout-test-results/svg/zoom/page/zoom-svg-through-object-with-percentage-size-actual.txt 
@@ -25,7 +25,7 @@
                     RenderTableCell {td} at (1,90) size 343x263 [r=2 c=0 rs=1 cs=1]
                       RenderEmbeddedObject {object} at (5,5) size 333x250
                         layer at (0,0) size 333x250
-                          RenderView at (0,0) size 333x250
+                          RenderView at (0,0) size 318x235
                         layer at (0,0) size 333x250
                           RenderSVGRoot {svg} at (0,0) size 333x250
                             RenderSVGContainer {g} at (18,31) size 260x164
Comment 24 Nikolas Zimmermann 2011-11-10 12:51:49 PST
(In reply to comment #22)
> They are not stable, the tests are still flaky on all chromium platforms.  The text output differs from run to run.

That's correct, I found the root of the problem. Bug 71673 has a patch, I expect all problems to be gone then.
Comment 25 Nikolas Zimmermann 2011-11-11 03:50:48 PST
(In reply to comment #24)
> That's correct, I found the root of the problem. Bug 71673 has a patch, I expect all problems to be gone then.

r99937 has the fix. We probably need some rebaselines and expectations update for chromium. Crossing fingers and watching the flakiness dashboard, I hope extra scrollbars are gone everywhere.
Comment 26 Adam Klein 2011-11-22 15:09:44 PST
Committed r101028: <http://trac.webkit.org/changeset/101028>