Bug 189686 – WebGL 2 updated test expectations for and enable conformance/conformance2 layout tests

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Justin Fan]

WebGL 2 partial passes and failing test expectations for conformance/conformance2 layout tests

[Justin Fan]

Created attachment 349992 [details]
Patch

[EWS Watchlist]

Comment on attachment 349992 [details]
Patch

Attachment 349992 [details] did not pass mac-ews (mac):
Output: https://webkit-queues.webkit.org/results/9251666

New failing tests:
webgl/2.0.0/conformance2/reading/read-pixels-from-fbo-test.html
webgl/2.0.0/conformance/extensions/webgl-draw-buffers.html

[EWS Watchlist]

Created attachment 349995 [details]
Archive of layout-test-results from ews103 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103  Port: mac-sierra  Platform: Mac OS X 10.12.6

[EWS Watchlist]

Comment on attachment 349992 [details]
Patch

Attachment 349992 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/9251702

New failing tests:
webgl/2.0.0/conformance2/reading/read-pixels-from-fbo-test.html
webgl/2.0.0/conformance/extensions/webgl-draw-buffers.html

[EWS Watchlist]

Created attachment 349996 [details]
Archive of layout-test-results from ews106 for mac-sierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews106  Port: mac-sierra-wk2  Platform: Mac OS X 10.12.6

[EWS Watchlist]

Comment on attachment 349992 [details]
Patch

Attachment 349992 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: https://webkit-queues.webkit.org/results/9251716

Number of test failures exceeded the failure limit.

[EWS Watchlist]

Created attachment 349998 [details]
Archive of layout-test-results from ews126 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews126  Port: ios-simulator-wk2  Platform: Mac OS X 10.13.6

[EWS Watchlist]

Comment on attachment 349992 [details]
Patch

Attachment 349992 [details] did not pass mac-debug-ews (mac):
Output: https://webkit-queues.webkit.org/results/9251740

New failing tests:
webgl/2.0.0/conformance2/reading/read-pixels-from-fbo-test.html
webgl/2.0.0/conformance/textures/misc/tex-image-and-sub-image-2d-with-array-buffer-view.html

[EWS Watchlist]

Created attachment 349999 [details]
Archive of layout-test-results from ews117 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews117  Port: mac-sierra  Platform: Mac OS X 10.12.6

[EWS Watchlist]

Comment on attachment 349992 [details]
Patch

Attachment 349992 [details] did not pass mac-debug-ews (mac):
Output: https://webkit-queues.webkit.org/results/9252531

New failing tests:
webgl/2.0.0/conformance2/reading/read-pixels-from-fbo-test.html
webgl/2.0.0/conformance/textures/misc/tex-image-and-sub-image-2d-with-array-buffer-view.html

[EWS Watchlist]

Created attachment 350004 [details]
Archive of layout-test-results from ews116 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews116  Port: mac-sierra  Platform: Mac OS X 10.12.6

[Justin Fan]

These tests may have to be marked [ Slow ] on mac-debug

  webgl/2.0.0/conformance/attribs/gl-vertexattribpointer.html [ Timeout Pass ]
  webgl/2.0.0/conformance/glsl/bugs/complex-glsl-does-not-crash.html [ Timeout Pass ]
  webgl/2.0.0/conformance/glsl/misc/shader-uniform-packing-restrictions.html [ Timeout Pass ]
  webgl/2.0.0/conformance/rendering/many-draw-calls.html [ Timeout Pass ]
  webgl/2.0.0/conformance/rendering/multisample-corruption.html [ Timeout Pass ]
  webgl/2.0.0/conformance2/attribs/gl-vertexattribipointer.html [ Timeout Pass ]
  webgl/2.0.0/conformance2/state/gl-object-get-calls.html [ Timeout Pass ]
  webgl/2.0.0/conformance2/textures/misc/copy-texture-image-luma-format.html [ Timeout Pass ]

These tests are failing on iOS-sim; will check in platform-specific results
  webgl/2.0.0/conformance/buffers/buffer-uninitialized.html [ Failure ]
  webgl/2.0.0/conformance/context/context-attribute-preserve-drawing-buffer.html [ Failure ]
  webgl/2.0.0/conformance/context/context-attributes-alpha-depth-stencil-antialias.html [ Failure ]
  webgl/2.0.0/conformance/context/context-hidden-alpha.html [ Failure ]
  webgl/2.0.0/conformance/context/context-no-alpha-fbo-with-alpha.html [ Failure ]
  webgl/2.0.0/conformance/extensions/ext-sRGB.html [ Failure ]
  webgl/2.0.0/conformance/extensions/get-extension.html [ Failure ]
  webgl/2.0.0/conformance/extensions/oes-texture-float-linear.html [ Failure ]
  webgl/2.0.0/conformance/extensions/oes-texture-float.html [ Failure ]
  webgl/2.0.0/conformance/extensions/oes-texture-half-float-linear.html [ Failure ]
  webgl/2.0.0/conformance/extensions/webgl-compressed-texture-pvrtc.html [ Failure ]
  webgl/2.0.0/conformance/extensions/webgl-draw-buffers-framebuffer-unsupported.html [ Failure ]
  webgl/2.0.0/conformance/extensions/webgl-draw-buffers.html [ Failure ]
  webgl/2.0.0/conformance/glsl/misc/shaders-with-invariance.html [ Failure ]
  webgl/2.0.0/conformance/renderbuffers/framebuffer-object-attachment.html [ Failure ]
  webgl/2.0.0/conformance/textures/misc/cube-incomplete-fbo.html [ Failure ]
  webgl/2.0.0/conformance2/extensions/ext-color-buffer-float.html [ Failure ]
  webgl/2.0.0/conformance2/extensions/promoted-extensions-in-shaders.html [ Failure ]
  webgl/2.0.0/conformance2/misc/views-with-offsets.html [ Failure ]
  webgl/2.0.0/conformance2/reading/read-pixels-from-fbo-test.html [ Failure ]
  webgl/2.0.0/conformance2/renderbuffers/multisample-with-full-sample-counts.html [ Failure ]
  webgl/2.0.0/conformance2/renderbuffers/multisampled-renderbuffer-initialization.html [ Failure ]
  webgl/2.0.0/conformance2/renderbuffers/readbuffer.html [ Failure ]
  webgl/2.0.0/conformance2/rendering/clear-srgb-color-buffer.html [ Failure ]
  webgl/2.0.0/conformance2/rendering/clipping-wide-points.html [ Failure ]
  webgl/2.0.0/conformance2/state/gl-get-calls.html [ Failure ]
  webgl/2.0.0/conformance2/state/gl-object-get-calls.html [ Failure ]
  webgl/2.0.0/conformance2/textures/misc/copy-texture-image.html [ Failure ]

[Jon Lee]

> [ FAIL ] 1 failures reported

:(((

[Justin Fan]

(In reply to Jon Lee from comment #13)
> > [ FAIL ] 1 failures reported
> 
> :(((

:(

Are you sad because it's incorrectly pluralized, or because there's only one sub-test failure?

[Justin Fan]

Additional failing tests on ios-sim
webgl/2.0.0/conformance/ogles/GL/build/build_009_to_016.html
webgl/2.0.0/conformance/reading/read-pixels-test.html
webgl/2.0.0/conformance/uniforms/uniform-default-values.html
webgl/2.0.0/conformance2/textures/misc/tex-3d-size-limit.html
webgl/2.0.0/conformance2/textures/misc/tex-image-with-bad-args.html
webgl/2.0.0/conformance2/textures/misc/tex-input-validation.html
webgl/2.0.0/conformance2/textures/misc/tex-mipmap-levels.html
webgl/2.0.0/conformance2/textures/misc/tex-new-formats.html
webgl/2.0.0/conformance2/textures/misc/tex-srgb-mipmap.html
webgl/2.0.0/conformance2/textures/misc/tex-storage-2d.html
webgl/2.0.0/conformance2/textures/misc/texture-npot.html

[Justin Fan]

Created attachment 350078 [details]
pitch

[Justin Fan]

<rdar://problem/44468728>

[Dean Jackson]

Comment on attachment 350078 [details]
pitch

Jon be happier with correct pluralisation :)

It would be nice if we could avoid putting things in the top-level TestExpectations, but I don't have a good alternative.

[Justin Fan]

Okay! I'll wait for this patch to land with the new expectations then update the test harness for pluralization updates lol

[WebKit Commit Bot]

Comment on attachment 350078 [details]
pitch

Clearing flags on attachment: 350078

Committed r236236: <https://trac.webkit.org/changeset/236236>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Dawei Fenton (:realdawei)]

(In reply to WebKit Commit Bot from comment #20)
> Comment on attachment 350078 [details]
> pitch
> 
> Clearing flags on attachment: 350078
> 
> Committed r236236: <https://trac.webkit.org/changeset/236236>

7 tests are failing:
https://build.webkit.org/results/Apple%20High%20Sierra%20Release%20WK2%20(Tests)/r236260%20(6843)/results.html

Marked these tests as [ Failure ] in https://trac.webkit.org/changeset/236272/webkit

[Dawei Fenton (:realdawei)]

(In reply to Dawei Fenton (:realdawei) from comment #22)
> (In reply to WebKit Commit Bot from comment #20)
> > Comment on attachment 350078 [details]
> > pitch
> > 
> > Clearing flags on attachment: 350078
> > 
> > Committed r236236: <https://trac.webkit.org/changeset/236236>
> 
> 7 tests are failing:
> https://build.webkit.org/results/
> Apple%20High%20Sierra%20Release%20WK2%20(Tests)/r236260%20(6843)/results.html
> 
> Marked these tests as [ Failure ] in
> https://trac.webkit.org/changeset/236272/webkit

Tests that were failing were:

webgl/2.0.0/conformance/buffers/buffer-uninitialized.html
webgl/2.0.0/conformance/extensions/webgl-draw-buffers.html
webgl/2.0.0/conformance/renderbuffers/feedback-loop.html
webgl/2.0.0/conformance/rendering/clipping-wide-points.html
webgl/2.0.0/conformance/textures/misc/copy-tex-image-2d-formats.html
webgl/2.0.0/conformance2/state/gl-object-get-calls.html
webgl/2.0.0/conformance2/textures/misc/tex-3d-size-limit.html

[Dawei Fenton (:realdawei)]

The following two tests were timing on out on iOS 11 Simulator WK2-debug:

https://build.webkit.org/results/Apple%20iOS%2011%20Simulator%20Debug%20WK2%20(Tests)/r236269%20(6532)/results.html

webgl/2.0.0/conformance/textures/misc/tex-image-and-sub-image-2d-with-array-buffer-view.html
webgl/2.0.0/conformance2/textures/misc/copy-texture-image-luma-format.html

skipped in: https://trac.webkit.org/changeset/236280/webkit

[Jon Lee]

(In reply to Justin Fan from comment #14)
> (In reply to Jon Lee from comment #13)
> > > [ FAIL ] 1 failures reported
> > 
> > :(((
> 
> :(
> 
> Are you sad because it's incorrectly pluralized

this

[Justin Fan]

Closing this bug for now. Further test expectation modifications should be tracked in new bugs.