Bug 80703 – [Qt] svg/animations tests are very flaky

• Status: RESOLVED
• Resolution: INVALID
• Component: Tools / Tests

[Csaba Osztrogonác]

Qt bots are regularly red because of flaky svg/animations tests

[Csaba Osztrogonác]

--- /ramdisk/qt-linux-release/build/layout-test-results/svg/animations/svgstring-animation-fallback-to-discrete-expected.txt 
+++ /ramdisk/qt-linux-release/build/layout-test-results/svg/animations/svgstring-animation-fallback-to-discrete-actual.txt 
@@ -7,7 +7,7 @@
 
 PASS getComputedStyle(rect).visibility is 'visible'
 PASS rect.style.visibility is ""
-PASS getComputedStyle(rect).visibility is 'hidden'
+FAIL getComputedStyle(rect).visibility should be hidden. Was visible.
 PASS rect.style.visibility is ""
 PASS getComputedStyle(rect).visibility is 'visible'
 PASS rect.style.visibility is ""

[Csaba Osztrogonác]

--- /home/webkitbuildbot/slaves/release32bit-qt5/buildslave/qt-linux-32-release-qt5/build/layout-test-results/svg/animations/svgstring-animation-fallback-to-discrete-expected.txt 
+++ /home/webkitbuildbot/slaves/release32bit-qt5/buildslave/qt-linux-32-release-qt5/build/layout-test-results/svg/animations/svgstring-animation-fallback-to-discrete-actual.txt 
@@ -7,7 +7,7 @@
 
 PASS getComputedStyle(rect).visibility is 'visible'
 PASS rect.style.visibility is ""
-PASS getComputedStyle(rect).visibility is 'hidden'
+FAIL getComputedStyle(rect).visibility should be hidden. Was visible.
 PASS rect.style.visibility is ""
 PASS getComputedStyle(rect).visibility is 'visible'
 PASS rect.style.visibility is ""

[Csaba Osztrogonác]

--- /home/webkitbuildbot/slaves/release32bit-qt5/buildslave/qt-linux-32-release-qt5/build/layout-test-results/svg/animations/svglengthlist-animation-3-expected.txt 
+++ /home/webkitbuildbot/slaves/release32bit-qt5/buildslave/qt-linux-32-release-qt5/build/layout-test-results/svg/animations/svglengthlist-animation-3-actual.txt 
@@ -8,11 +8,11 @@
 
 PASS text.x.animVal.numberOfItems is 1
 PASS text.x.animVal.getItem(0).value is 50
-PASS text.x.animVal.numberOfItems is 4
-PASS text.x.animVal.getItem(0).value is 70
-PASS text.x.animVal.getItem(1).value is 80
-PASS text.x.animVal.getItem(2).value is 90
-PASS text.x.animVal.getItem(3).value is 110
+FAIL text.x.animVal.numberOfItems should be 4. Was 1.
+FAIL text.x.animVal.getItem(0).value should be 70. Was 50.
+FAIL text.x.animVal.getItem(1).value should be 80. Threw exception Error: INDEX_SIZE_ERR: DOM Exception 1
+FAIL text.x.animVal.getItem(2).value should be 90. Threw exception Error: INDEX_SIZE_ERR: DOM Exception 1
+FAIL text.x.animVal.getItem(3).value should be 110. Threw exception Error: INDEX_SIZE_ERR: DOM Exception 1
 PASS text.x.animVal.numberOfItems is 4
 PASS text.x.animVal.getItem(0).value is 70
 PASS text.x.animVal.getItem(1).value is 80

[Csaba Osztrogonác]

--- /home/webkitbuildbot/slaves/release32bit-qt5/buildslave/qt-linux-32-release-qt5/build/layout-test-results/svg/animations/animate-gradient-transform-expected.txt 
+++ /home/webkitbuildbot/slaves/release32bit-qt5/buildslave/qt-linux-32-release-qt5/build/layout-test-results/svg/animations/animate-gradient-transform-actual.txt 
@@ -7,7 +7,7 @@
 
 PASS gradient.gradientTransform.baseVal.consolidate().matrix.e is 0
 PASS gradient.gradientTransform.animVal.consolidate().matrix.e threw exception Error: NO_MODIFICATION_ALLOWED_ERR: DOM Exception 7.
-PASS gradient.gradientTransform.baseVal.consolidate().matrix.e is 100
+FAIL gradient.gradientTransform.baseVal.consolidate().matrix.e should be 100. Was 99.99999237060547.
 PASS gradient.gradientTransform.animVal.consolidate().matrix.e threw exception Error: NO_MODIFICATION_ALLOWED_ERR: DOM Exception 7.
 PASS gradient.gradientTransform.baseVal.consolidate().matrix.e is 200
 PASS gradient.gradientTransform.animVal.consolidate().matrix.e threw exception Error: NO_MODIFICATION_ALLOWED_ERR: DOM Exception 7.

[Csaba Osztrogonác]

--- /home/webkitbuildbot/slaves/release32bit-qt5/buildslave/qt-linux-32-release-qt5/build/layout-test-results/svg/animations/animate-inherit-css-property-expected.txt 
+++ /home/webkitbuildbot/slaves/release32bit-qt5/buildslave/qt-linux-32-release-qt5/build/layout-test-results/svg/animations/animate-inherit-css-property-actual.txt 
@@ -7,7 +7,7 @@
 
 PASS getComputedStyle(rect).fill is '#ff0000'
 PASS rect.style.fill is ""
-PASS getComputedStyle(rect).fill is '#804000'
+FAIL getComputedStyle(rect).fill should be #804000. Was #803f00.
 PASS rect.style.fill is ""
 PASS getComputedStyle(rect).fill is '#008000'
 PASS rect.style.fill is ""

[Csaba Osztrogonác]

--- /home/webkitbuildbot/slaves/release32bit-qt5/buildslave/qt-linux-32-release-qt5/build/layout-test-results/svg/animations/svglengthlist-animation-3-expected.txt 
+++ /home/webkitbuildbot/slaves/release32bit-qt5/buildslave/qt-linux-32-release-qt5/build/layout-test-results/svg/animations/svglengthlist-animation-3-actual.txt 
@@ -8,11 +8,11 @@
 
 PASS text.x.animVal.numberOfItems is 1
 PASS text.x.animVal.getItem(0).value is 50
-PASS text.x.animVal.numberOfItems is 4
-PASS text.x.animVal.getItem(0).value is 70
-PASS text.x.animVal.getItem(1).value is 80
-PASS text.x.animVal.getItem(2).value is 90
-PASS text.x.animVal.getItem(3).value is 110
+FAIL text.x.animVal.numberOfItems should be 4. Was 1.
+FAIL text.x.animVal.getItem(0).value should be 70. Was 50.
+FAIL text.x.animVal.getItem(1).value should be 80. Threw exception Error: INDEX_SIZE_ERR: DOM Exception 1
+FAIL text.x.animVal.getItem(2).value should be 90. Threw exception Error: INDEX_SIZE_ERR: DOM Exception 1
+FAIL text.x.animVal.getItem(3).value should be 110. Threw exception Error: INDEX_SIZE_ERR: DOM Exception 1
 PASS text.x.animVal.numberOfItems is 4
 PASS text.x.animVal.getItem(0).value is 70
 PASS text.x.animVal.getItem(1).value is 80

[Csaba Osztrogonác]

I'm going to skip these craziest tests:

svg/animations/animate-gradient-transform.html
svg/animations/animate-inherit-css-property.html
svg/animations/svgstring-animation-fallback-to-discrete.html
svg/animations/svglengthlist-animation-3.html

[Csaba Osztrogonác]

cc SVG experts, you might be interested in fixing this bug.

But I'm afraid my mails automatically land in your SPAM box, because I 
cc-ed you guys to dozens of SVG regressions/bugs and you never answered.

[Csaba Osztrogonác]

*** Bug 80291 has been marked as a duplicate of this bug. ***

[Csaba Osztrogonác]

I skipped them until (unlikely) proper fix - http://trac.webkit.org/changeset/110297

[Nikolas Zimmermann]

(In reply to comment #8)
> cc SVG experts, you might be interested in fixing this bug.
> 
> But I'm afraid my mails automatically land in your SPAM box, because I 
> cc-ed you guys to dozens of SVG regressions/bugs and you never answered.

Jeez, I wasn't ware that this is in such a bad shape for Qt.
I see most problems are related to discrete animations.
Say rect.x gets animated from x=10 to x=100 exactly at 2s (discrete calcMode).
Then I sample at 1.999s and 2.001s (once before it changes, once afterwards).

Is it possible that SharedTimerQt works differently then the Mac/Chromium/.. timers leading to precision problems? I'm not sure at all. :(

Ossy said these problems aren't reproducable. I can't debug this w/o a Qt environment - can anyone take this over?

[Csaba Osztrogonác]

--- /ramdisk/qt-linux-release/build/layout-test-results/svg/animations/animate-color-fill-currentColor-expected.txt 
+++ /ramdisk/qt-linux-release/build/layout-test-results/svg/animations/animate-color-fill-currentColor-actual.txt 
@@ -18,14 +18,14 @@
 PASS fillColor.blue.getFloatValue(CSSPrimitiveValue.CSS_NUMBER) is 0
 
 Half-time condition:
-PASS document.defaultView.getComputedStyle(rect).getPropertyValue('fill') is "#804000"
+FAIL document.defaultView.getComputedStyle(rect).getPropertyValue('fill') should be #804000. Was #803f00.
 PASS (fillPaint = document.defaultView.getComputedStyle(rect).getPropertyCSSValue('fill')).toString() is "[object SVGPaint]"
 PASS fillPaint.paintType is SVGPaint.SVG_PAINTTYPE_CURRENTCOLOR
 PASS fillPaint.uri is ""
 PASS fillPaint.colorType is SVGColor.SVG_COLORTYPE_CURRENTCOLOR
 PASS (fillColor = fillPaint.rgbColor).toString() is "[object RGBColor]"
 PASS fillColor.red.getFloatValue(CSSPrimitiveValue.CSS_NUMBER) is 128
-PASS fillColor.green.getFloatValue(CSSPrimitiveValue.CSS_NUMBER) is 64
+FAIL fillColor.green.getFloatValue(CSSPrimitiveValue.CSS_NUMBER) should be 64. Was 63.
 PASS fillColor.blue.getFloatValue(CSSPrimitiveValue.CSS_NUMBER) is 0
 
 End condition:

[Csaba Osztrogonác]

One more fail:

--- /home/webkitbuildbot/slaves/release32bit-NRWT/buildslave/qt-linux-32-release-NRWT/build/layout-test-results/svg/animations/animate-end-attribute-expected.txt 
+++ /home/webkitbuildbot/slaves/release32bit-NRWT/buildslave/qt-linux-32-release-NRWT/build/layout-test-results/svg/animations/animate-end-attribute-actual.txt 
@@ -2,7 +2,7 @@
 
 PASS rect.x.animVal.value is 100
 PASS rect.x.baseVal.value is 100
-PASS rect.x.animVal.value is 50
+FAIL rect.x.animVal.value should be 50. Was 49.999996185302734.
 PASS rect.x.baseVal.value is 100
 PASS rect.x.animVal.value is 200
 PASS rect.x.baseVal.value is 100

[Nikolas Zimmermann]

Created attachment 131611 [details]
Patch

[Rob Buis]

Comment on attachment 131611 [details]
Patch

LGTM.

[Nikolas Zimmermann]

(In reply to comment #15)
> (From update of attachment 131611 [details])
> LGTM.

Ossy I'm going to give this a try, and also unskip the previously skipped Qt test. Let's hope it works.

[Nikolas Zimmermann]

Committed r110574: <http://trac.webkit.org/changeset/110574>

[Nikolas Zimmermann]

Reopening until we can verify the problem is gone for Qt.

[Csaba Osztrogonác]

svg/animations/animate-elem-08-t-drt.html started to fail:
http://build.webkit.sed.hu/results/x86-64%20Linux%20Qt%20Release/r110680%20%2834927%29/svg/animations/animate-elem-08-t-drt-pretty-diff.html

Unfortunately the build was broken for long time. Now it works again, let's check the bot after several build.

[Kristóf Kosztyó]

(In reply to comment #19)
> svg/animations/animate-elem-08-t-drt.html started to fail:
 I add to skipped at r110686 to make the bot green.

[Eric Seidel (no email)]

Comment on attachment 131611 [details]
Patch

Cleared Rob Buis's review+ from obsolete attachment 131611 [details] so that this bug does not appear in http://webkit.org/pending-commit.

[Ádám Kallai]

Is Anybody working this bug?

[Jocelyn Turcotte]

=== Bulk closing of Qt bugs ===

If you believe that this bug report is still relevant for a non-Qt port of webkit.org, please re-open it and remove [Qt] from the summary.

If you believe that this is still an important QtWebKit bug, please fill a new report at https://bugreports.qt-project.org and add a link to this issue. See http://qt-project.org/wiki/ReportingBugsInQt for additional guidelines.