Bug 169917 – [GTK] Many WebGL tests fail, are slow or timeout.

• Status: RESOLVED
• Resolution: WONTFIX
• Component: WebKitGTK

[Carlos Alberto Lopez Perez]

After Enabling WebGL, 3D and compositing layout tests in bug 71849 I have observed this WebGL related problems:


# WebGL related failures
webgl/1.0.2/conformance/canvas/drawingbuffer-test.html [ Failure Pass ]
webgl/1.0.2/conformance/extensions/oes-vertex-array-object.html [ Failure Pass ]
webgl/1.0.2/conformance/textures/texture-formats-test.html [ Failure Pass ]
webgl/1.0.3/conformance/context/context-hidden-alpha.html [ Failure Pass ]
compositing/overlap-blending/children-opacity-huge.html [ ImageOnlyFailure Pass ]
compositing/video/video-border-radius.html [ ImageOnlyFailure Pass ]
# WebGL timeouts or slow tests.
fast/canvas/webgl/oes-texture-half-float.html [ Slow ]
webgl/1.0.2/conformance/context/context-creation-and-destruction.html [ Timeout ]
webgl/1.0.2/conformance/extensions/oes-texture-float-with-video.html [ Timeout ]
webgl/1.0.2/conformance/glsl/misc/shader-uniform-packing-restrictions.html [ Slow ]
webgl/1.0.2/conformance/glsl/misc/shader-with-non-reserved-words.html [ Slow ]
webgl/1.0.2/conformance/rendering/multisample-corruption.html [ Timeout ]
webgl/1.0.2/conformance/textures/tex-image-and-sub-image-2d-with-video-rgb565.html [ Timeout ]
webgl/1.0.2/conformance/textures/tex-image-and-sub-image-2d-with-video-rgba4444.html [ Timeout ]
webgl/1.0.2/conformance/textures/tex-image-and-sub-image-2d-with-video-rgba5551.html [ Timeout ]
webgl/1.0.2/conformance/textures/tex-image-and-sub-image-2d-with-video.html [ Timeout ]
webgl/1.0.2/conformance/uniforms/gl-uniform-arrays.html [ Slow ]
webgl/1.0.3/conformance/extensions/oes-texture-half-float-with-video.html [ Timeout ]
webgl/1.0.3/conformance/glsl/bugs/complex-glsl-does-not-crash.html [ Slow ]
webgl/1.0.3/conformance/glsl/bugs/temp-expressions-should-not-crash.html [ Slow ]
webgl/1.0.3/conformance/textures/texture-upload-size.html [ Timeout ]
webgl/1.0.2/conformance/uniforms/out-of-bounds-uniform-array-access.html [ Slow ]
webgl/1.0.2/conformance/ogles/GL/atan/atan_001_to_008.html [ Slow ]
webgl/1.0.2/conformance/uniforms/uniform-default-values.html [ Slow ]
webgl/1.0.2/conformance/ogles/GL/pow/pow_009_to_016.html [ Slow ]
webgl/1.0.2/conformance/ogles/GL/pow/pow_017_to_024.html [ Slow ]
webgl/1.0.2/conformance/ogles/GL/reflect/reflect_001_to_006.html [ Slow ]
webgl/1.0.2/conformance/ogles/GL/refract/refract_001_to_006.html [ Slow ]
webgl/1.0.2/conformance/ogles/GL/swizzlers/swizzlers_097_to_104.html [ Slow ]
webgl/1.0.2/conformance/ogles/GL/log/log_001_to_008.html [ Slow ]
webgl/1.0.3/conformance/uniforms/uniform-values-per-program.html [ Slow ]
webgl/1.0.3/conformance/context/context-eviction-with-garbage-collection.html [ Slow ]
webgl/1.0.3/conformance/glsl/misc/struct-nesting-of-variable-names.html [ Slow ]
webgl/max-active-contexts-webglcontextlost-prevent-default.html [ Timeout ]
# WebGL crashes
fast/canvas/webgl/webgl-draw-buffers.html [ Failure Crash ]
webgl/1.0.3/conformance/extensions/webgl-draw-buffers.html [ Crash ]

[Carlos Alberto Lopez Perez]

Also this ones:

fast/canvas/webgl/antialiasing-enabled.html [ Failure Pass ]
fast/canvas/webgl/context-attributes-alpha-depth-stencil-antialias.html [ Failure ]
fast/canvas/webgl/copyBufferSubData.html [ Failure ]
fast/canvas/webgl/framebuffer-object-attachment.html [ Failure ]
fast/canvas/webgl/getBufferSubData-webgl1.html [ Failure ]
fast/canvas/webgl/glsl-conformance.html [ Failure ]
fast/canvas/webgl/premultiplyalpha-test.html [ Failure ]
fast/canvas/webgl/read-pixels-test.html [ Failure Pass ]
fast/canvas/webgl/oes-vertex-array-object.html [ Failure Pass ]
fast/canvas/webgl/unprefixed-anisotropic-extension.html [ Failure Pass ]
webgl/1.0.2/conformance/canvas/buffer-offscreen-test.html [ Failure ]
webgl/1.0.2/conformance/context/context-attribute-preserve-drawing-buffer.html [ Failure ]
webgl/1.0.2/conformance/context/context-attributes-alpha-depth-stencil-antialias.html [ Failure ]
webgl/1.0.2/conformance/context/premultiplyalpha-test.html [ Failure ]
webgl/1.0.2/conformance/extensions/oes-texture-float.html [ Failure ]
webgl/1.0.2/conformance/reading/read-pixels-test.html [ Failure Pass ]
webgl/1.0.2/conformance/textures/copy-tex-image-2d-formats.html [ Failure ]
webgl/1.0.2/conformance/textures/texture-attachment-formats.html [ Failure ]
webgl/1.0.2/conformance/textures/texture-mips.html [ Failure ]
webgl/1.0.2/conformance/textures/texture-npot.html [ Failure ]
webgl/1.0.2/conformance/textures/texture-size-limit.html [ Failure ]
webgl/1.0.3/conformance/attribs/gl-bindAttribLocation-aliasing.html [ Failure ]
webgl/1.0.3/conformance/attribs/gl-bindAttribLocation-matrix.html [ Failure ]
webgl/1.0.3/conformance/extensions/oes-texture-half-float.html [ Failure ]
webgl/1.0.3/conformance/extensions/webgl-compressed-texture-size-limit.html [ Failure ]
webgl/1.0.3/conformance/glsl/misc/shaders-with-invariance.html [ Failure ]
webgl/1.0.3/conformance/glsl/misc/shaders-with-name-conflicts.html [ Failure ]
webgl/1.0.3/conformance/renderbuffers/feedback-loop.html [ Failure ]
webgl/1.0.3/conformance/rendering/point-no-attributes.html [ Failure ]
webgl/1.0.3/conformance/textures/texture-copying-feedback-loops.html [ Failure ]
webgl/many-contexts-access-after-loss.html [ Failure ]
webgl/many-contexts.html [ Failure ]
webgl/max-active-contexts-console-warning.html [ Failure ]
webgl/max-active-contexts-gc.html [ Failure ]
webgl/max-active-contexts-oldest-context-lost.html [ Failure ]
webgl/webgl-vertex-array-object-defined.html [ Failure ]
webgl/webgl-border.html [ ImageOnlyFailure ]
webgl/webgl-box-shadow.html [ ImageOnlyFailure ]

[Carlos Alberto Lopez Perez]

There are some differences on the results of those test if you run then inside xvfb or on the native display (Xorg/X11 in my case with Intel GPU. glxinfo: http://sprunge.us/WBjL ). You can run them on the native display by passing --display-server=xorg to the script run-webkit-tests.


Differences between --display-server=xorg and --display-server=xvfb :

* webgl/1.0.2/conformance/textures/texture-mips.html and  webgl/1.0.2/conformance/textures/texture-npot.html pass on native Xorg but fail on Xvfb.

* webgl/1.0.3/conformance/extensions/webgl-draw-buffers.html crashes on Xvfb (crash log: http://sprunge.us/jTBV ) but fails (without crash) on native Xorg.

* fast/canvas/webgl/drawingbuffer-test.html fast/canvas/webgl/webgl-depth-texture.html pass on Xvfb but fail on native Xorg.

* More tests timeout or are slow on native Xorg.

[Carlos Alberto Lopez Perez]

Marked the test failing (and rebaselined others were the diff looked right) in https://trac.webkit.org/r214223

[Carlos Alberto Lopez Perez]

Marked the diff of failures (and passes) between Xvfb and native Wayland at: https://trac.webkit.org/changeset/214229/trunk/LayoutTests/platform/gtk-wayland/TestExpectations

[Michael Catanzaro]

I'm adding a Failure expectation for webgl/1.0.3/conformance/extensions/webgl-draw-buffers.html because it has now (for the first time) run without crashing. (But it prints FAIL instead of PASS.)

I'm concerned by the very high number of failures in this bug. Oh well.

[Michael Catanzaro]

*** Bug 188102 has been marked as a duplicate of this bug. ***

[Nikolas Zimmermann]

According to https://build.webkit.org/builders/GTK%20Linux%2064-bit%20Release%20%28Tests%29/builds/11947/steps/layout-test/logs/stdio these tests actually pass just fine:

  webgl/1.0.3/conformance/extensions/oes-texture-half-float.html
  webgl/1.0.3/conformance/glsl/misc/shaders-with-name-conflicts.html
  webgl/1.0.3/conformance/rendering/point-no-attributes.html
  webgl/1.0.3/conformance/textures/texture-copying-feedback-loops.html
  webgl/1.0.3/conformance/textures/texture-size-limit.html


since at least a few builds that I checked.
Updating expectations.

[Miguel Gomez]

8 new webgl tests are slow:
webkit.org/b/169917 webgl/1.0.3/conformance/extensions/webgl-draw-buffers.html [ Slow ]
webkit.org/b/169917 webgl/1.0.3/conformance/glsl/bugs/long-expressions-should-not-crash.html [ Slow ]
webkit.org/b/169917 webgl/1.0.3/conformance/glsl/constructors/glsl-construct-bvec2.html [ Slow ]
webkit.org/b/169917 webgl/1.0.3/conformance/glsl/constructors/glsl-construct-ivec2.html [ Slow ]
webkit.org/b/169917 webgl/1.0.3/conformance/glsl/constructors/glsl-construct-vec2.html [ Slow ]
webkit.org/b/169917 webgl/1.0.3/conformance/glsl/misc/shader-uniform-packing-restrictions.html [ Slow ]
webkit.org/b/169917 webgl/1.0.3/conformance/glsl/misc/shader-varying-packing-restrictions.html [ Slow ]
webkit.org/b/169917 webgl/1.0.3/conformance/glsl/misc/struct-nesting-of-variable-names.html [ Slow ]

[Diego Pino]

*** Bug 206145 has been marked as a duplicate of this bug. ***

[Diego Pino]

Several tests are failing after test suite update in r252121.

3 new tests added in r252121 are failing:

	* webgl/1.0.3/conformance/more/functions/readPixelsBadArgs.html [ Failure ]
	* webgl/1.0.3/conformance/more/functions/texImage2DHTML.html [ Failure ]
	* webgl/1.0.3/conformance/more/functions/texSubImage2DHTML.html [ Failure ]

1 existing test started to fail:

	* webgl/1.0.3/conformance/glsl/misc/shader-struct-scope.html [ Failure ]

2 existing tests started to time out:

	* webgl/1.0.3/conformance/glsl/misc/shader-uniform-packing-restrictions.html [ Timeout ]
	* webgl/1.0.3/conformance/glsl/misc/struct-nesting-of-variable-names.html [ Timeout ]

1 existing test is now flaky:

	* webgl/1.0.3/conformance/textures/texture-size.html [ Timeout Pass ]

[Diego Pino]

The only test filed under this bug was:

  - fast/canvas/webgl/oes-vertex-array-object.html

GTK post-commit test bot reports the test has been failing consistently for the last 4000 revisions.

Diff:

--- /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/fast/canvas/webgl/oes-vertex-array-object-expected.txt
+++ /home/buildbot/worker/gtk-linux-64-release-tests/build/layout-test-results/fast/canvas/webgl/oes-vertex-array-object-actual.txt
@@ -1,3 +1,7 @@
+CONSOLE MESSAGE: *** Error loading shader '[object WebGLShader]':INVALID_OPERATION
+CONSOLE MESSAGE: Error in compiling shader
+CONSOLE MESSAGE: *** Error loading shader '[object WebGLShader]':INVALID_OPERATION
+CONSOLE MESSAGE: Error in compiling shader
 This test verifies the functionality of the OES_vertex_array_object extension, if it is available.
 
 On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
@@ -32,20 +36,30 @@
 Testing that attribute values are not attached to bindings
 PASS Vertex attribute values are not attached to bindings
 Testing draws with various VAO bindings
-PASS Draw 0 passed pixel test
+FAIL Draw 0 failed pixel test
 PASS Draw 1 passed pixel test
 PASS Draw 2 passed pixel test
 Testing using deleted buffers referenced by VAOs
-PASS should be 255,0,0,255
-PASS should be 0,255,0,255
-PASS should be 0,0,255,255
-PASS should be 0,255,255,255
+FAIL buffer removed too early
+FAIL buffer removed too early
+FAIL buffer removed too early
+FAIL should be 255,0,0,255
+at (0, 0) expected: 255,0,0,255 was 255,255,255,255
+FAIL should be 0,255,0,255
+at (0, 0) expected: 0,255,0,255 was 255,255,255,255
+FAIL should be 0,0,255,255
+at (0, 0) expected: 0,0,255,255 was 255,255,255,255
+FAIL should be 0,255,255,255
+at (0, 0) expected: 0,255,255,255 was 255,255,255,255
 Testing that VAOs don't effect ARRAY_BUFFER binding.
-PASS should be red
-PASS should be green
-PASS getError was expected value: NO_ERROR : there should be no errors
+FAIL should be red
+at (0, 0) expected: 255,0,0,255 was 255,255,255,255
+FAIL should be green
+at (0, 0) expected: 0,255,0,255 was 255,255,255,255
+FAIL getError expected: NO_ERROR. Was INVALID_OPERATION : there should be no errors
 
 PASS successfullyParsed is true
+Some tests failed.
 
 TEST COMPLETE

[Diego Pino]

Actually there are still many tests filed under this bug.

fast/canvas/webgl/oes-vertex-array-object.html was marked as flaky, but I have changed its status to failure only.

[Alejandro G. Castro]

After replacing the WebGL backend with ANGLE we have a new bug to check the new failures, a lot of failing tests in this bug pass now. The new bug is:

https://bugs.webkit.org/show_bug.cgi?id=245840

The gardening commit is:

https://commits.webkit.org/255008@main