NEW 170772
Getting snapshot of WKWebView fails ImageDiff tests 
https://bugs.webkit.org/show_bug.cgi?id=170772
Summary Getting snapshot of WKWebView fails ImageDiff tests
Jonathan Bedard
Reported 2017-04-12 09:05:01 PDT
When testing, WebKit will frequently take a snapshot of the current WKWebView and compare it to some expected value. While developing this functionality for on-device testing (https://bugs.webkit.org/show_bug.cgi?id=169421), it was suggested that the snapshotting API should be used. I have attached test results from this run. For 6 of the tests, the snapshotting API does not give the expected result. Looking at the results, expected results and the screen of the device while running the tests, this seems to be a bug in the snapshotting API. I think compositing/regions/position-layers-inside-region-overflow-hidden.html demonstrates this most clearly.
Attachments
Test results when using snapshot API for ImageDiff (14.52 MB, application/zip)
2017-04-12 09:06 PDT, Jonathan Bedard 		no flags
Details
Patch (3.42 KB, patch)
2017-06-09 09:57 PDT, Jonathan Bedard 		buildbot: commit-queue-
DetailsFormatted DiffDiff
Archive of layout-test-results from ews125 for ios-simulator-wk2 (1.57 MB, application/zip)
2017-06-09 11:23 PDT, Build Bot 		no flags
Details
View All Add attachment proposed patch, testcase, etc.
Jonathan Bedard
Comment 1 2017-04-12 09:06:09 PDT
Created attachment 306912 [details] Test results when using snapshot API for ImageDiff
Jonathan Bedard
Comment 2 2017-06-03 08:16:04 PDT
Originally there were 5 of these failures. Now, I only see compositing/geometry/clipped-out-perspective.html and compositing/hidpi-subpixel-transform-origin.html.
Jonathan Bedard
Comment 3 2017-06-09 09:57:03 PDT
Created attachment 312447 [details] Patch
Jonathan Bedard
Comment 4 2017-06-09 09:58:11 PDT
Local on-device testing has indicated that these tests no longer fail. I'd like to see if EWS agrees with this for iOS-Simulators.
Build Bot
Comment 5 2017-06-09 11:23:55 PDT
Comment on attachment 312447 [details] Patch Attachment 312447 [details] did not pass ios-sim-ews (ios-simulator-wk2): Output: http://webkit-queues.webkit.org/results/3901268 New failing tests: compositing/regions/position-layers-inside-region-overflow-hidden.html compositing/masks/compositing-clip-path-and-mask.html compositing/regions/position-layer-inside-region-overflow-hidden.html fast/harness/snapshot-captures-compositing.html compositing/hidpi-subpixel-transform-origin.html compositing/regions/position-layers-inside-regions-overflow-hidden.html compositing/geometry/clipped-out-perspective.html
Build Bot
Comment 6 2017-06-09 11:23:56 PDT
Created attachment 312462 [details] Archive of layout-test-results from ews125 for ios-simulator-wk2 The attached test failures were seen while running run-webkit-tests on the ios-sim-ews. Bot: ews125 Port: ios-simulator-wk2 Platform: Mac OS X 10.12.5
Jonathan Bedard
Comment 7 2017-06-12 10:28:59 PDT
It appears that these tests only fail on Simulator. They succeed on device.
Note You need to log in before you can comment on or make changes to this bug.