Bug 189641 - WebGL conformance: Failures and Timeouts in suite 2.0.0/conformance
Summary: WebGL conformance: Failures and Timeouts in suite 2.0.0/conformance
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebGL (show other bugs)
Version: Other
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on: 183438 211484 214765 214792 214977 215346 215433 241027
Blocks: 126404
  Show dependency treegraph
 
Reported: 2018-09-14 18:23 PDT by Justin Fan
Modified: 2022-08-11 03:31 PDT (History)
9 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Justin Fan 2018-09-14 18:23:46 PDT
Separating these from the deqp tests for now, as those are much more long-running. 

In 2.0.0/conformance, we are timing out 53 tests:

  webgl/2.0.0/conformance/canvas/rapid-resizing.html [ Timeout ]
  webgl/2.0.0/conformance/context/context-creation-and-destruction.html [ Timeout ]
  webgl/2.0.0/conformance/context/context-creation.html [ Timeout ]
  webgl/2.0.0/conformance/extensions/oes-texture-float-with-canvas.html [ Timeout ]
  webgl/2.0.0/conformance/extensions/oes-texture-float-with-image.html [ Timeout ]
  webgl/2.0.0/conformance/extensions/oes-texture-float-with-video.html [ Timeout ]
  webgl/2.0.0/conformance/extensions/oes-texture-half-float-with-canvas.html [ Timeout ]
  webgl/2.0.0/conformance/extensions/oes-texture-half-float-with-image.html [ Timeout ]
  webgl/2.0.0/conformance/extensions/oes-texture-half-float-with-video.html [ Timeout ]
  webgl/2.0.0/conformance/extensions/webgl-compressed-texture-size-limit.html [ Timeout ]
  webgl/2.0.0/conformance/glsl/misc/shader-with-non-reserved-words.html [ Timeout ]
  webgl/2.0.0/conformance/glsl/misc/shader-with-reserved-words.html [ Timeout ]
  webgl/2.0.0/conformance/textures/canvas/tex-2d-rgb-rgb-unsigned_byte.html [ Timeout ]
  webgl/2.0.0/conformance/textures/canvas/tex-2d-rgb-rgb-unsigned_short_5_6_5.html [ Timeout ]
  webgl/2.0.0/conformance/textures/canvas/tex-2d-rgba-rgba-unsigned_byte.html [ Timeout ]
  webgl/2.0.0/conformance/textures/canvas/tex-2d-rgba-rgba-unsigned_short_4_4_4_4.html [ Timeout ]
  webgl/2.0.0/conformance/textures/canvas/tex-2d-rgba-rgba-unsigned_short_5_5_5_1.html [ Timeout ]
  webgl/2.0.0/conformance/textures/image/tex-2d-rgb-rgb-unsigned_byte.html [ Timeout ]
  webgl/2.0.0/conformance/textures/image/tex-2d-rgb-rgb-unsigned_short_5_6_5.html [ Timeout ]
  webgl/2.0.0/conformance/textures/image/tex-2d-rgba-rgba-unsigned_byte.html [ Timeout ]
  webgl/2.0.0/conformance/textures/image/tex-2d-rgba-rgba-unsigned_short_4_4_4_4.html [ Timeout ]
  webgl/2.0.0/conformance/textures/image/tex-2d-rgba-rgba-unsigned_short_5_5_5_1.html [ Timeout ]
  webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgb-rgb-unsigned_byte.html [ Timeout ]
  webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgb-rgb-unsigned_short_5_6_5.html [ Timeout ]
  webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgba-rgba-unsigned_byte.html [ Timeout ]
  webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgba-rgba-unsigned_short_4_4_4_4.html [ Timeout ]
  webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgba-rgba-unsigned_short_5_5_5_1.html [ Timeout ]
  webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgb-rgb-unsigned_byte.html [ Timeout ]
  webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgb-rgb-unsigned_short_5_6_5.html [ Timeout ]
  webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgba-rgba-unsigned_byte.html [ Timeout ]
  webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgba-rgba-unsigned_short_4_4_4_4.html [ Timeout ]
  webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgba-rgba-unsigned_short_5_5_5_1.html [ Timeout ]
  webgl/2.0.0/conformance/textures/image_bitmap_from_video/tex-2d-rgb-rgb-unsigned_byte.html [ Timeout ]
  webgl/2.0.0/conformance/textures/image_bitmap_from_video/tex-2d-rgb-rgb-unsigned_short_5_6_5.html [ Timeout ]
  webgl/2.0.0/conformance/textures/image_bitmap_from_video/tex-2d-rgba-rgba-unsigned_byte.html [ Timeout ]
  webgl/2.0.0/conformance/textures/image_bitmap_from_video/tex-2d-rgba-rgba-unsigned_short_4_4_4_4.html [ Timeout ]
  webgl/2.0.0/conformance/textures/image_bitmap_from_video/tex-2d-rgba-rgba-unsigned_short_5_5_5_1.html [ Timeout ]
  webgl/2.0.0/conformance/textures/misc/gl-teximage.html [ Timeout ]
  webgl/2.0.0/conformance/textures/misc/tex-image-canvas-corruption.html [ Timeout ]
  webgl/2.0.0/conformance/textures/misc/texture-npot-video.html [ Timeout ]
  webgl/2.0.0/conformance/textures/misc/texture-size-limit.html [ Timeout ]
  webgl/2.0.0/conformance/textures/misc/texture-transparent-pixels-initialized.html [ Timeout ]
  webgl/2.0.0/conformance/textures/misc/texture-upload-size.html [ Timeout ]
  webgl/2.0.0/conformance/textures/video/tex-2d-rgb-rgb-unsigned_byte.html [ Timeout ]
  webgl/2.0.0/conformance/textures/video/tex-2d-rgb-rgb-unsigned_short_5_6_5.html [ Timeout ]
  webgl/2.0.0/conformance/textures/video/tex-2d-rgba-rgba-unsigned_byte.html [ Timeout ]
  webgl/2.0.0/conformance/textures/video/tex-2d-rgba-rgba-unsigned_short_4_4_4_4.html [ Timeout ]
  webgl/2.0.0/conformance/textures/video/tex-2d-rgba-rgba-unsigned_short_5_5_5_1.html [ Timeout ]
  webgl/2.0.0/conformance/textures/webgl_canvas/tex-2d-rgb-rgb-unsigned_byte.html [ Timeout ]
  webgl/2.0.0/conformance/textures/webgl_canvas/tex-2d-rgb-rgb-unsigned_short_5_6_5.html [ Timeout ]
  webgl/2.0.0/conformance/textures/webgl_canvas/tex-2d-rgba-rgba-unsigned_byte.html [ Timeout ]
  webgl/2.0.0/conformance/textures/webgl_canvas/tex-2d-rgba-rgba-unsigned_short_4_4_4_4.html [ Timeout ]
  webgl/2.0.0/conformance/textures/webgl_canvas/tex-2d-rgba-rgba-unsigned_short_5_5_5_1.html [ Timeout ]

And failing 59: 

webgl/2.0.0/conformance/attribs/gl-vertexattribpointer.html
webgl/2.0.0/conformance/buffers/buffer-data-and-buffer-sub-data.html
webgl/2.0.0/conformance/buffers/buffer-uninitialized.html
webgl/2.0.0/conformance/context/constants-and-properties.html
webgl/2.0.0/conformance/context/context-lost-restored.html
webgl/2.0.0/conformance/context/methods.html
webgl/2.0.0/conformance/context/user-defined-properties-on-context.html
webgl/2.0.0/conformance/extensions/ext-sRGB.html
webgl/2.0.0/conformance/extensions/ext-texture-filter-anisotropic.html
webgl/2.0.0/conformance/extensions/get-extension.html
webgl/2.0.0/conformance/extensions/oes-texture-float.html
webgl/2.0.0/conformance/extensions/oes-texture-half-float.html
webgl/2.0.0/conformance/extensions/webgl-depth-texture.html
webgl/2.0.0/conformance/extensions/webgl-draw-buffers-framebuffer-unsupported.html
webgl/2.0.0/conformance/extensions/webgl-draw-buffers.html
webgl/2.0.0/conformance/glsl/misc/shader-struct-scope.html
webgl/2.0.0/conformance/glsl/misc/shaders-with-invariance.html
webgl/2.0.0/conformance/glsl/misc/shaders-with-name-conflicts.html
webgl/2.0.0/conformance/limits/gl-line-width.html
webgl/2.0.0/conformance/misc/bad-arguments-test.html
webgl/2.0.0/conformance/misc/error-reporting.html
webgl/2.0.0/conformance/misc/expando-loss.html
webgl/2.0.0/conformance/misc/null-object-behaviour.html
webgl/2.0.0/conformance/misc/object-deletion-behaviour.html
webgl/2.0.0/conformance/misc/webgl-specific.html
webgl/2.0.0/conformance/more/functions/readPixelsBadArgs.html
webgl/2.0.0/conformance/more/functions/texImage2DHTML.html
webgl/2.0.0/conformance/more/functions/texSubImage2DHTML.html
webgl/2.0.0/conformance/more/functions/vertexAttribPointerBadArgs.html
webgl/2.0.0/conformance/ogles/GL/biuDepthRange/biuDepthRange_001_to_002.html
webgl/2.0.0/conformance/ogles/GL/gl_FragCoord/gl_FragCoord_001_to_003.html
webgl/2.0.0/conformance/programs/get-active-test.html
webgl/2.0.0/conformance/reading/read-pixels-test.html
webgl/2.0.0/conformance/renderbuffers/feedback-loop.html
webgl/2.0.0/conformance/renderbuffers/framebuffer-object-attachment.html
webgl/2.0.0/conformance/rendering/clipping-wide-points.html
webgl/2.0.0/conformance/rendering/point-no-attributes.html
webgl/2.0.0/conformance/state/gl-enable-enum-test.html
webgl/2.0.0/conformance/state/gl-object-get-calls.html
webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgb-rgb-unsigned_byte.html
webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgb-rgb-unsigned_short_5_6_5.html
webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgba-rgba-unsigned_byte.html
webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgba-rgba-unsigned_short_4_4_4_4.html
webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgba-rgba-unsigned_short_5_5_5_1.html
webgl/2.0.0/conformance/textures/image_bitmap_from_image_bitmap/tex-2d-rgb-rgb-unsigned_byte.html
webgl/2.0.0/conformance/textures/image_bitmap_from_image_bitmap/tex-2d-rgb-rgb-unsigned_short_5_6_5.html
webgl/2.0.0/conformance/textures/image_bitmap_from_image_bitmap/tex-2d-rgba-rgba-unsigned_byte.html
webgl/2.0.0/conformance/textures/image_bitmap_from_image_bitmap/tex-2d-rgba-rgba-unsigned_short_4_4_4_4.html
webgl/2.0.0/conformance/textures/image_bitmap_from_image_bitmap/tex-2d-rgba-rgba-unsigned_short_5_5_5_1.html
webgl/2.0.0/conformance/textures/image_bitmap_from_image_data/tex-2d-rgb-rgb-unsigned_byte.html
webgl/2.0.0/conformance/textures/image_bitmap_from_image_data/tex-2d-rgb-rgb-unsigned_short_5_6_5.html
webgl/2.0.0/conformance/textures/image_bitmap_from_image_data/tex-2d-rgba-rgba-unsigned_byte.html
webgl/2.0.0/conformance/textures/image_bitmap_from_image_data/tex-2d-rgba-rgba-unsigned_short_4_4_4_4.html
webgl/2.0.0/conformance/textures/image_bitmap_from_image_data/tex-2d-rgba-rgba-unsigned_short_5_5_5_1.html
webgl/2.0.0/conformance/textures/misc/copy-tex-image-and-sub-image-2d.html
webgl/2.0.0/conformance/textures/misc/cube-incomplete-fbo.html
webgl/2.0.0/conformance/textures/misc/origin-clean-conformance.html
webgl/2.0.0/conformance/textures/misc/tex-sub-image-2d-bad-args.html
webgl/2.0.0/conformance/textures/misc/texture-copying-feedback-loops.html
Comment 1 Radar WebKit Bug Importer 2018-09-14 18:24:35 PDT
<rdar://problem/44476592>
Comment 2 Justin Fan 2018-09-19 15:44:32 PDT
# This test spits out numbers in its failure messages that appear to be different between processes
webkit.org/b/189686 webgl/2.0.0/conformance2/reading/read-pixels-from-fbo-test.html [ Skip ]
Comment 3 Dawei Fenton (:realdawei) 2018-09-20 11:59:55 PDT
7 tests are consistently failing:
https://build.webkit.org/results/Apple%20High%20Sierra%20Release%20WK2%20(Tests)/r236260%20(6843)/results.html

webgl/2.0.0/conformance/buffers/buffer-uninitialized.html
webgl/2.0.0/conformance/extensions/webgl-draw-buffers.html
webgl/2.0.0/conformance/renderbuffers/feedback-loop.html
webgl/2.0.0/conformance/rendering/clipping-wide-points.html
webgl/2.0.0/conformance/textures/misc/copy-tex-image-2d-formats.html
webgl/2.0.0/conformance2/state/gl-object-get-calls.html
webgl/2.0.0/conformance2/textures/misc/tex-3d-size-limit.html

I marked these tests as [ Failure ] in https://trac.webkit.org/changeset/236272/webkit
Comment 4 Dawei Fenton (:realdawei) 2018-09-28 15:03:28 PDT
These tests are consistently timing out:
https://build.webkit.org/builders/Apple%20iOS%2012%20Simulator%20Release%20WK2%20%28Tests%29/builds/129/steps/layout-test/logs/stdio


  webgl/2.0.0/conformance/ogles/GL/acos/acos_001_to_006.html
  webgl/2.0.0/conformance/ogles/GL/asin/asin_001_to_006.html
  webgl/2.0.0/conformance/ogles/GL/atan/atan_001_to_008.html
  webgl/2.0.0/conformance/ogles/GL/faceforward/faceforward_001_to_006.html
  webgl/2.0.0/conformance/ogles/GL/floor/floor_001_to_006.html
  webgl/2.0.0/conformance/ogles/GL/functions/functions_001_to_008.html
  webgl/2.0.0/conformance/ogles/GL/log/log_001_to_008.html
  webgl/2.0.0/conformance/ogles/GL/log2/log2_001_to_008.html
  webgl/2.0.0/conformance/ogles/GL/mat/mat_001_to_008.html

Skipped in: https://trac.webkit.org/changeset/236614/webkit
Comment 5 Dawei Fenton (:realdawei) 2018-10-30 16:46:40 PDT
webgl/2.0.0/conformance/attribs/gl-vertexattribpointer.html is timing on on iOS

marked as [ Slow ] in: https://trac.webkit.org/changeset/237609/webkit
Comment 6 Kenneth Russell 2020-07-24 21:39:01 PDT
Re-testing after many fixes to WebKit's WebGL 2.0 implementation.

The following currently-skipped tests are now observed to pass:

webgl/2.0.0/conformance/context/context-creation-and-destruction.html
webgl/2.0.0/conformance/context/context-creation.html
webgl/2.0.0/conformance/extensions/oes-texture-float-with-canvas.html
webgl/2.0.0/conformance/extensions/oes-texture-half-float-with-canvas.html
webgl/2.0.0/conformance/extensions/webgl-compressed-texture-size-limit.html
webgl/2.0.0/conformance/textures/misc/texture-size-limit.html
webgl/2.0.0/conformance2/sync/sync-webgl-specific.html



The following tests pass in the MiniBrowser but will likely time out in the harness:
webgl/2.0.0/conformance/glsl/misc/shader-with-non-reserved-words.html
webgl/2.0.0/conformance/glsl/misc/shader-with-reserved-words.html



The following tests still time out:
webgl/2.0.0/conformance2/misc/uninitialized-test-2.html



The following tests need bug fixes from upstream at:
https://www.khronos.org/registry/webgl/conformance-suites/2.0.0/
to address timeouts or other failures:

webgl/2.0.0/conformance/extensions/oes-texture-float-with-image.html
webgl/2.0.0/conformance/extensions/oes-texture-float-with-video.html
webgl/2.0.0/conformance/textures/image/*
webgl/2.0.0/conformance/textures/misc/gl-teximage.html
webgl/2.0.0/conformance/textures/misc/tex-image-canvas-corruption.html
webgl/2.0.0/conformance/textures/misc/texture-transparent-pixels-initialized.html
webgl/2.0.0/conformance/textures/misc/texture-upload-size.html
webgl/2.0.0/conformance/textures/video/*
webgl/2.0.0/conformance2/textures/image/*
webgl/2.0.0/conformance2/textures/video/*
webgl/2.0.0/conformance2/rendering/draw-buffers.html

The following tests need updates, but still fail due to bugs in WebKit:
webgl/2.0.0/conformance/textures/image_bitmap_from_blob/*
webgl/2.0.0/conformance/textures/image_bitmap_from_image/*
webgl/2.0.0/conformance/textures/image_bitmap_from_video/*
webgl/2.0.0/conformance/textures/misc/texture-npot-video.html
webgl/2.0.0/conformance2/textures/image_bitmap_from_blob/*
webgl/2.0.0/conformance2/textures/image_bitmap_from_image/*
webgl/2.0.0/conformance2/textures/image_bitmap_from_video/*
webgl/2.0.0/conformance2/textures/misc/tex-image-with-bad-args-from-dom-elements.html
Comment 7 Kenneth Russell 2020-08-07 16:28:47 PDT
With the fixes from Bug 211484 and Bug 183438, all of the image_bitmap* conformance tests are now passing.
Comment 8 Kimmo Kinnunen 2021-03-18 02:23:56 PDT
This skips now
webgl/2.0.0/conformance/extensions/oes-texture-float-with-canvas.html [ Slow ]
webgl/2.0.0/conformance/extensions/oes-texture-half-float-with-canvas.html [ Slow ]
webgl/2.0.0/conformance/glsl/misc/shader-with-non-reserved-words.html [ Skip ]
webgl/2.0.0/conformance/glsl/misc/shader-with-reserved-words.html [ Skip ]
webgl/2.0.0/conformance/textures/misc/texture-size-limit.html [ Slow ]
webgl/2.0.0/conformance/textures/misc/texture-npot-video.html [ Skip ]
webgl/2.0.0/conformance2/misc/uninitialized-test-2.html [ Skip ]