Bug 207858

Summary: Changed results due to ANGLE use
Product: WebKit Reporter: Dean Jackson <dino>
Component: WebGLAssignee: Dean Jackson <dino>
Status: NEW ---    
Severity: Normal CC: ap, dino, jacob_uphoff, jdarpinian, justin_fan, kbr, Lawrence.j, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Bug Depends on: 212286, 212287, 209139, 210524, 211055, 211758, 212056, 212249, 212271, 212277    
Bug Blocks: 126404    

Description Dean Jackson 2020-02-17 13:46:44 PST
While switching over to ANGLE, the following tests required something more than a rebaseline.

macOS:

fast/canvas/webgl/readPixels-float.html
fast/canvas/webgl/draw-elements-out-of-bounds-uint-index.html
fast/canvas/webgl/simulated-vertexAttrib0-invalid-indicies.html
fast/canvas/webgl/webgl2-texture-upload-enums.html
fast/canvas/webgl/texImage2D-video-flipY-false.html (times out)

iOS Simulator:

fast/canvas/webgl/context-attributes-alpha.html (ref test)
fast/canvas/webgl/match-page-color-space.html (ref test)
webgl/smell-test.html (ref test)
webgl/webgl-backing-store-size-update.html (ref test)
webgl/webgl-border.html (ref test)
webgl/webgl-box-shadow.html (ref test)
webgl/misc/switch-to-discrete-gpu.html (why is this even running on iOS?)
fast/canvas/webgl/oes-texture-half-float-uint16.html
fast/canvas/webgl/uninitialized-test.html
fast/canvas/webgl/webgl-depth-texture.html (marked as not supported! why?)
fast/canvas/webgl/webgl-texture-binding-preserved.html
webgl/1.0.3/conformance/canvas/texture-bindings-unaffected-on-resize.html
webgl/1.0.3/conformance/context/context-lost-restored.html
webgl/1.0.3/conformance/extensions/ext-sRGB.html
webgl/1.0.3/conformance/extensions/oes-element-index-uint.html
webgl/1.0.3/conformance/glsl/misc/shader-struct-scope.html
webgl/1.0.3/conformance/misc/delayed-drawing.html
webgl/1.0.3/conformance/misc/uninitialized-test.html
webgl/1.0.3/conformance/renderbuffers/framebuffer-object-attachment.html
webgl/1.0.3/conformance/state/state-uneffected-after-compositing.html
webgl/1.0.3/conformance/textures/default-texture.html
webgl/1.0.3/conformance/textures/tex-image-and-sub-image-2d-with-canvas-rgb565.html
webgl/1.0.3/conformance/textures/tex-image-and-sub-image-2d-with-canvas-rgba4444.html
webgl/1.0.3/conformance/textures/tex-image-and-sub-image-2d-with-canvas-rgba5551.html
webgl/1.0.3/conformance/textures/tex-image-and-sub-image-2d-with-canvas.html
webgl/1.0.3/conformance/textures/tex-image-and-sub-image-2d-with-webgl-canvas-rgb565.html
webgl/1.0.3/conformance/textures/tex-image-and-sub-image-2d-with-webgl-canvas-rgba4444.html
webgl/1.0.3/conformance/textures/tex-image-and-sub-image-2d-with-webgl-canvas-rgba5551.html
webgl/1.0.3/conformance/textures/tex-image-and-sub-image-2d-with-webgl-canvas.html
Comment 1 Radar WebKit Bug Importer 2020-02-17 13:46:54 PST
<rdar://problem/59524399>
Comment 2 Dean Jackson 2020-02-17 14:53:27 PST
Also fast/canvas/webgl/oes-texture-float-linear.html is saying OES_texture_float is not supported on iOS simulator.
Comment 3 Radar WebKit Bug Importer 2020-02-17 14:53:36 PST
<rdar://problem/59527402>
Comment 4 Jacob Uphoff 2020-02-17 16:43:08 PST
Please add expectations for the following:

fast/canvas/webgl/program-test.html
webgl/1.0.3/conformance/programs/program-test.html
Comment 5 Dean Jackson 2020-02-17 17:43:49 PST
WebGL 2 macOS:

webgl/2.0.0/conformance/extensions/oes-vertex-array-object.html
webgl/2.0.0/conformance/programs/program-test.html
webgl/2.0.0/conformance2/buffers/buffer-type-restrictions.html (improved overall)
webgl/2.0.0/conformance2/buffers/get-buffer-sub-data.html
webgl/2.0.0/conformance2/buffers/one-large-uniform-buffer.html
webgl/2.0.0/conformance2/extensions/promoted-extensions.html
webgl/2.0.0/conformance2/misc/expando-loss-2.html
webgl/2.0.0/conformance2/reading/read-pixels-from-fbo-test.html
webgl/2.0.0/conformance2/renderbuffers/invalidate-framebuffer.html
webgl/2.0.0/conformance2/renderbuffers/multisample-with-full-sample-counts.html
webgl/2.0.0/conformance2/renderbuffers/multisampled-renderbuffer-initialization.html
webgl/2.0.0/conformance2/rendering/canvas-resizing-with-pbo-bound.html
webgl/2.0.0/conformance2/rendering/instanced-arrays.html
webgl/2.0.0/conformance2/rendering/instanced-rendering-bug.html
webgl/2.0.0/conformance2/rendering/rgb-format-support.html
webgl/2.0.0/conformance2/textures/image_bitmap_from_image/* (all timed out)
webgl/2.0.0/conformance2/textures/image_bitmap_from_video/* (all timed out)
webgl/2.0.0/conformance2/textures/video (timed out)
webgl/2.0.0/conformance2/vertex_arrays/vertex-array-object.html

(These are just a list of things that started failing more. The results might have been misleading to begin with.)
Comment 6 Ryan Haddad 2020-02-17 17:52:55 PST
(In reply to Jacob Uphoff from comment #4)
> Please add expectations for the following:
> 
> fast/canvas/webgl/program-test.html
> webgl/1.0.3/conformance/programs/program-test.html
https://trac.webkit.org/changeset/256802/webkit
Comment 7 Dean Jackson 2020-02-18 09:48:33 PST
*** Bug 207896 has been marked as a duplicate of this bug. ***
Comment 8 Dean Jackson 2020-02-18 09:48:45 PST
iOS:

fast/canvas/webgl/oes-vertex-array-object.html
Comment 9 Ryan Haddad 2020-02-18 09:52:44 PST
rdar://59524399
Comment 10 Jacob Uphoff 2020-02-20 10:33:37 PST
fast/images/exif-orientation-webgl-texture.html

This test is also failing with the changes made here
Comment 11 Dean Jackson 2020-02-21 14:38:44 PST
iOS:
fast/images/exif-orientation-webgl-texture.html
Comment 12 Jason Lawrence 2020-02-24 14:27:01 PST
This test is flaky failing on Mojave:
webgl/1.0.3/conformance/textures/tex-image-and-sub-image-2d-with-webgl-canvas-rgba5551.html
Comment 13 Kenneth Russell 2020-02-24 14:46:48 PST
(In reply to Jason Lawrence from comment #12)
> This test is flaky failing on Mojave:
> webgl/1.0.3/conformance/textures/tex-image-and-sub-image-2d-with-webgl-
> canvas-rgba5551.html

Link to the above failures:
https://results.webkit.org/?suite=layout-tests&test=webgl%2F1.0.3%2Fconformance%2Ftextures%2Ftex-image-and-sub-image-2d-with-webgl-canvas-rgba5551.html
Comment 15 Kenneth Russell 2020-02-24 14:57:30 PST
The diffs are:

 31[ 28: PASS ] shouldBe 0,255,0
 32[ 29: PASS ] shouldBe 255,0,0
 33[ 30: PASS ] shouldBe 0,255,0
 34[ 31: PASS ] shouldBe 255,0,0
 35[ 32: PASS ] shouldBe 0,255,0
 36[ 33: FAIL ] at (0, 0) expected: 255,0,0 was 0,0,0
 37[ 34: FAIL ] at (0, 16) expected: 0,255,0 was 0,0,0
 38[ 35: FAIL ] at (0, 16) expected: 255,0,0 was 0,0,0
 39[ 36: FAIL ] at (0, 0) expected: 0,255,0 was 0,0,0
 40[ 37: FAIL ] at (0, 0) expected: 255,0,0 was 0,0,0
 41[ 38: FAIL ] at (0, 16) expected: 0,255,0 was 0,0,0
 42[ 39: FAIL ] at (0, 16) expected: 255,0,0 was 0,0,0
 43[ 40: FAIL ] at (0, 0) expected: 0,255,0 was 0,0,0
...

The test case that failed in this case is the second whole iteration through the test cases in LayoutTests/webgl/1.0.3/resources/webgl_test_files/conformance/resources/tex-image-and-sub-image-2d-with-webgl-canvas.js , the first test case in the list:

Testing texImage2D with flipY=true canvas size: 1x2 with red-green
Checking top
PASS shouldBe 255,0,0
Checking bottom
PASS shouldBe 0,255,0
Comment 16 Kenneth Russell 2020-02-24 15:01:20 PST
The failure of webgl/1.0.3/conformance/textures/tex-image-and-sub-image-2d-with-webgl-canvas-rgba5551.html in 
https://build.webkit.org/results/Apple%20Mojave%20Debug%20WK2%20(Tests)/r257234%20(7731)/results.html failed a different sub-test, however.

We should probably mark this test flaky. Not 100% sure what could be causing this; Chrome doesn't seem to have a suppression for this test, but it is running against a newer conformance suite snapshot.
Comment 17 Dean Jackson 2020-02-29 13:40:44 PST
webgl/1.0.3/conformance/renderbuffers/framebuffer-object-attachment.html fixed by 208331
Comment 18 Dean Jackson 2020-03-30 10:06:32 PDT
webgl/2.0.0/conformance2/rendering/framebuffer-completeness-unaffected.html
Comment 19 Justin Fan 2020-05-22 16:38:14 PDT
These are the remaining WebGL1-based regressions:

iOS/Simulator:
https://bugs.webkit.org/show_bug.cgi?id=212277 webgl/1.0.3/conformance/extensions/ext-sRGB.html
https://bugs.webkit.org/show_bug.cgi?id=212271 fast/canvas/webgl/webgl-depth-texture.html
https://bugs.webkit.org/show_bug.cgi?id=209139 [ iOS wk2 ] webgl/1.0.3/conformance/textures/copy-tex-image-2d-formats.html is failing. (209139)
https://bugs.webkit.org/show_bug.cgi?id=212249 [ANGLE - iOS] fast/canvas/webgl/uninitialized-test.html is still failing
——
macOS and iOS: https://bugs.webkit.org/show_bug.cgi?id=212287 
fast/canvas/webgl/draw-elements-out-of-bounds-uint-index.html (Not a breaking failure)

macOS only: https://bugs.webkit.org/show_bug.cgi?id=212286 
fast/canvas/webgl/simulated-vertexAttrib0-invalid-indicies.html (Not a breaking failure, tested code path will be obsoleted)
Comment 20 Justin Fan 2020-05-22 16:39:16 PDT
final comment about "code path will be obsoleted" was in error, please ignore