Regression: fast/hidpi/image-srcset-svg-canvas-2x.html is failing consistently on iOS EWS

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Aakash Jain]

Regression: fast/hidpi/image-srcset-svg-canvas-2x.html is failing consistently on iOS EWS.
This test was passing earlier on iOS, since yesterday (Jan 29) it started failing consistently on iOS EWS.

First failure in:
- Build: https://ews-build.webkit.org/#/builders/24/builds/9662
- Result: https://ews-build.webkit.org/results/iOS-13-Simulator-WK2-Tests-EWS/r389296-9662/results.html

Recent examples:
https://ews-build.webkit.org/#/builders/24/builds/9680
https://ews-build.webkit.org/#/builders/24/builds/9678
https://ews-build.webkit.org/#/builders/24/builds/9675

This issue doesn't seems to be happening on build.webkit.org bots: https://results.webkit.org/?suite=layout-tests&test=fast%2Fhidpi%2Fimage-srcset-svg-canvas-2x.html

[Radar WebKit Bug Importer]

<rdar://problem/59058757>

[Aakash Jain]

This test seems extremely flaky.

This test is slowing down EWS ios-wk2 queue and sometimes causes EWS to blame random patches for this test failure.

https://ews-build.webkit.org/#/builders/24/builds/9793 - "Found 1 pre-existing test failure: fast/hidpi/image-srcset-svg-canvas-2x.html"

https://ews-build.webkit.org/#/builders/24/builds/9786 - "Found flaky test: fast/hidpi/image-srcset-svg-canvas-2x.html"

https://ews-build.webkit.org/#/builders/24/builds/9779 - "Found 1 new test failure: fast/hidpi/image-srcset-svg-canvas-2x.html"

We should consider updating test expectations meanwhile while it is being investigated.

[Truitt Savell]

I am working on reproducing this failure with test lists.

[Truitt Savell]

This is not easily reproducing with a test list

[Truitt Savell]

Marked as failing in https://trac.webkit.org/changeset/255985/webkit

Sense this will end up skipping the test on EWS I expect another test to begin failing in its place.

[Truitt Savell]

The next test in line did end up failing. Now fast/hidpi/image-srcset-svg-canvas.html is failing on iOS-ews

[Truitt Savell]

Created attachment 390086 [details]
Test list from bot

This is the list of tests that run before the failing test on the test runner. most likely one of these tests is setting a bad state

[Truitt Savell]

I am able to reproduce the failure by using this command on the news bot:

run-webkit-tests --no-build --no-show-results --no-new-test-results --clobber-old-results --release --ios-simulator --child-processes 1 compositing/webgl/webgl-reflection.html fast/hidpi/image-srcset-svg2.html fast/hidpi/image-srcset-svg-canvas.html

remove either of the first two tests and it will no longer fail.


So something in the interaction of these three tests is causing this cascading failure:

compositing/webgl/webgl-reflection.html 
fast/hidpi/image-srcset-svg2.html 
fast/hidpi/image-srcset-svg-canvas.html

[Truitt Savell]

This is the first news run that shows the original test failure from fast/hidpi/image-srcset-relative-svg-canvas-2x.html

https://ews-build.webkit.org/#/builders/24/builds/9513

[Aakash Jain]

(In reply to Truitt Savell from comment #6)
> The next test in line did end up failing. Now fast/hidpi/image-srcset-svg-canvas.html is failing on iOS-ews
Filed https://bugs.webkit.org/show_bug.cgi?id=207851 to track that.

[Truitt Savell]

Changed expectations to force a skip of compositing/webgl/webgl-reflection.html on EWS instead of the two test fast/hidpi/image-srcset-relative-svg-canvas-2x.html and fast/hidpi/image-srcset-svg-canvas-2x.html.

https://trac.webkit.org/changeset/257091/webkit

[Alexey Proskuryakov]

*** Bug 207851 has been marked as a duplicate of this bug. ***

[Said Abou-Hallawa]

Created attachment 394772 [details]
Patch

[Said Abou-Hallawa]

With disabling the canvas accelerated drawing, the iOS EWS passed the all the tests from the first time. I will resubmit the patch one more time and see how the bots behave.

[Said Abou-Hallawa]

Created attachment 394779 [details]
Patch

[Aakash Jain]

(In reply to Said Abou-Hallawa from comment #14)
> I will resubmit the patch one more time and see how the bots behave.
Thanks for the patch. I have pressed the 'Rebuild' button 5 times on one of the successful ios-wk2 build (https://ews-build.webkit.org/#/builders/24/builds/14055). 

Let's see the results in following ones:
https://ews-build.webkit.org/#/buildrequests/411396
https://ews-build.webkit.org/#/buildrequests/411397
https://ews-build.webkit.org/#/buildrequests/411398
https://ews-build.webkit.org/#/buildrequests/411399
https://ews-build.webkit.org/#/buildrequests/411400

[Said Abou-Hallawa]

Created attachment 394795 [details]
Patch

[Said Abou-Hallawa]

Created attachment 394804 [details]
Patch

[Said Abou-Hallawa]

Created attachment 394805 [details]
Patch

[Said Abou-Hallawa]

Created attachment 394806 [details]
Patch

[Said Abou-Hallawa]

Created attachment 394807 [details]
Patch

[Said Abou-Hallawa]

Created attachment 394808 [details]
Patch

[Aakash Jain]

I have again pressed 'Rebuild' button 6 times on one of the recent successful ios-wk2 build (https://ews-build.webkit.org/#/builders/24/builds/14091) for patch 394808.

Let's see result in following builds:
https://ews-build.webkit.org/#/builders/24/builds/14094
https://ews-build.webkit.org/#/builders/24/builds/14095
https://ews-build.webkit.org/#/builders/24/builds/14096
https://ews-build.webkit.org/#/builders/24/builds/14097
https://ews-build.webkit.org/#/builders/24/builds/14098
https://ews-build.webkit.org/#/builders/24/builds/14099

[Said Abou-Hallawa]

iOS EWS bots succeeded in processing the latest patch from the first time with the 6 uploads I did and with the 6 retries that Aakash did.

[Said Abou-Hallawa]

Created attachment 394820 [details]
Patch

[Alexey Proskuryakov]

Comment on attachment 394820 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=394820&action=review

> LayoutTests/ChangeLog:9
> +        Disable the canvas scaling anti-aliasing by applying the CSS property 
> +        "image-rendering: pixelated;" to the <img> and the <canvas> elements.

Did you figure out what the state leaking form compositing/webgl/webgl-reflection.html is? With the root cause not fixed, it seems very likely that other tests will be affected.

[Said Abou-Hallawa]

Comment on attachment 394820 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=394820&action=review

>> LayoutTests/ChangeLog:9
>> +        "image-rendering: pixelated;" to the <img> and the <canvas> elements.
> 
> Did you figure out what the state leaking form compositing/webgl/webgl-reflection.html is? With the root cause not fixed, it seems very likely that other tests will be affected.

No I have not. But I looked at the canvas drawing code carefully and I did not see anything that can be leaked from a previous page. Most likely the leaking in underlaying simulator frameworks.

I think we should avoid testing the antialiasing in all the hidpi tests. So I think this patch is worthy landing. We can keep the bug open and continue investigating but let's land this patch and see the results. And hopefully this will shorten iOS EWS bots processing time.

[Darin Adler]

Comment on attachment 394820 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=394820&action=review

>>> LayoutTests/ChangeLog:9
>>> +        "image-rendering: pixelated;" to the <img> and the <canvas> elements.
>> 
>> Did you figure out what the state leaking form compositing/webgl/webgl-reflection.html is? With the root cause not fixed, it seems very likely that other tests will be affected.
> 
> No I have not. But I looked at the canvas drawing code carefully and I did not see anything that can be leaked from a previous page. Most likely the leaking in underlaying simulator frameworks.
> 
> I think we should avoid testing the antialiasing in all the hidpi tests. So I think this patch is worthy landing. We can keep the bug open and continue investigating but let's land this patch and see the results. And hopefully this will shorten iOS EWS bots processing time.

I am convinced.

[EWS]

Committed r259174: <https://trac.webkit.org/changeset/259174>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 394820 [details].