Bug 189686 - WebGL 2 updated test expectations for and enable conformance/conformance2 layout tests
Summary: WebGL 2 updated test expectations for and enable conformance/conformance2 lay...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Justin Fan
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2018-09-17 19:32 PDT by Justin Fan
Modified: 2018-09-21 16:14 PDT (History)
7 users (show)

See Also:


Attachments
Patch (10.89 MB, patch)
2018-09-17 19:38 PDT, Justin Fan
ews-watchlist: commit-queue-
Details | Formatted Diff | Diff
Archive of layout-test-results from ews103 for mac-sierra (2.35 MB, application/zip)
2018-09-17 20:48 PDT, EWS Watchlist
no flags Details
Archive of layout-test-results from ews106 for mac-sierra-wk2 (3.20 MB, application/zip)
2018-09-17 21:01 PDT, EWS Watchlist
no flags Details
Archive of layout-test-results from ews126 for ios-simulator-wk2 (2.49 MB, application/zip)
2018-09-17 21:28 PDT, EWS Watchlist
no flags Details
Archive of layout-test-results from ews117 for mac-sierra (3.33 MB, application/zip)
2018-09-17 21:45 PDT, EWS Watchlist
no flags Details
Archive of layout-test-results from ews116 for mac-sierra (3.51 MB, application/zip)
2018-09-17 23:21 PDT, EWS Watchlist
no flags Details
pitch (11.26 MB, patch)
2018-09-18 17:03 PDT, Justin Fan
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Justin Fan 2018-09-17 19:32:11 PDT
WebGL 2 partial passes and failing test expectations for conformance/conformance2 layout tests
Comment 1 Justin Fan 2018-09-17 19:38:42 PDT
Created attachment 349992 [details]
Patch
Comment 2 EWS Watchlist 2018-09-17 20:48:07 PDT
Comment on attachment 349992 [details]
Patch

Attachment 349992 [details] did not pass mac-ews (mac):
Output: https://webkit-queues.webkit.org/results/9251666

New failing tests:
webgl/2.0.0/conformance2/reading/read-pixels-from-fbo-test.html
webgl/2.0.0/conformance/extensions/webgl-draw-buffers.html
Comment 3 EWS Watchlist 2018-09-17 20:48:08 PDT
Created attachment 349995 [details]
Archive of layout-test-results from ews103 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103  Port: mac-sierra  Platform: Mac OS X 10.12.6
Comment 4 EWS Watchlist 2018-09-17 21:01:57 PDT
Comment on attachment 349992 [details]
Patch

Attachment 349992 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/9251702

New failing tests:
webgl/2.0.0/conformance2/reading/read-pixels-from-fbo-test.html
webgl/2.0.0/conformance/extensions/webgl-draw-buffers.html
Comment 5 EWS Watchlist 2018-09-17 21:01:58 PDT
Created attachment 349996 [details]
Archive of layout-test-results from ews106 for mac-sierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews106  Port: mac-sierra-wk2  Platform: Mac OS X 10.12.6
Comment 6 EWS Watchlist 2018-09-17 21:28:51 PDT
Comment on attachment 349992 [details]
Patch

Attachment 349992 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: https://webkit-queues.webkit.org/results/9251716

Number of test failures exceeded the failure limit.
Comment 7 EWS Watchlist 2018-09-17 21:28:52 PDT
Created attachment 349998 [details]
Archive of layout-test-results from ews126 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews126  Port: ios-simulator-wk2  Platform: Mac OS X 10.13.6
Comment 8 EWS Watchlist 2018-09-17 21:45:26 PDT
Comment on attachment 349992 [details]
Patch

Attachment 349992 [details] did not pass mac-debug-ews (mac):
Output: https://webkit-queues.webkit.org/results/9251740

New failing tests:
webgl/2.0.0/conformance2/reading/read-pixels-from-fbo-test.html
webgl/2.0.0/conformance/textures/misc/tex-image-and-sub-image-2d-with-array-buffer-view.html
Comment 9 EWS Watchlist 2018-09-17 21:45:27 PDT
Created attachment 349999 [details]
Archive of layout-test-results from ews117 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews117  Port: mac-sierra  Platform: Mac OS X 10.12.6
Comment 10 EWS Watchlist 2018-09-17 23:21:01 PDT
Comment on attachment 349992 [details]
Patch

Attachment 349992 [details] did not pass mac-debug-ews (mac):
Output: https://webkit-queues.webkit.org/results/9252531

New failing tests:
webgl/2.0.0/conformance2/reading/read-pixels-from-fbo-test.html
webgl/2.0.0/conformance/textures/misc/tex-image-and-sub-image-2d-with-array-buffer-view.html
Comment 11 EWS Watchlist 2018-09-17 23:21:03 PDT
Created attachment 350004 [details]
Archive of layout-test-results from ews116 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews116  Port: mac-sierra  Platform: Mac OS X 10.12.6
Comment 12 Justin Fan 2018-09-18 13:01:19 PDT
These tests may have to be marked [ Slow ] on mac-debug

  webgl/2.0.0/conformance/attribs/gl-vertexattribpointer.html [ Timeout Pass ]
  webgl/2.0.0/conformance/glsl/bugs/complex-glsl-does-not-crash.html [ Timeout Pass ]
  webgl/2.0.0/conformance/glsl/misc/shader-uniform-packing-restrictions.html [ Timeout Pass ]
  webgl/2.0.0/conformance/rendering/many-draw-calls.html [ Timeout Pass ]
  webgl/2.0.0/conformance/rendering/multisample-corruption.html [ Timeout Pass ]
  webgl/2.0.0/conformance2/attribs/gl-vertexattribipointer.html [ Timeout Pass ]
  webgl/2.0.0/conformance2/state/gl-object-get-calls.html [ Timeout Pass ]
  webgl/2.0.0/conformance2/textures/misc/copy-texture-image-luma-format.html [ Timeout Pass ]

These tests are failing on iOS-sim; will check in platform-specific results
  webgl/2.0.0/conformance/buffers/buffer-uninitialized.html [ Failure ]
  webgl/2.0.0/conformance/context/context-attribute-preserve-drawing-buffer.html [ Failure ]
  webgl/2.0.0/conformance/context/context-attributes-alpha-depth-stencil-antialias.html [ Failure ]
  webgl/2.0.0/conformance/context/context-hidden-alpha.html [ Failure ]
  webgl/2.0.0/conformance/context/context-no-alpha-fbo-with-alpha.html [ Failure ]
  webgl/2.0.0/conformance/extensions/ext-sRGB.html [ Failure ]
  webgl/2.0.0/conformance/extensions/get-extension.html [ Failure ]
  webgl/2.0.0/conformance/extensions/oes-texture-float-linear.html [ Failure ]
  webgl/2.0.0/conformance/extensions/oes-texture-float.html [ Failure ]
  webgl/2.0.0/conformance/extensions/oes-texture-half-float-linear.html [ Failure ]
  webgl/2.0.0/conformance/extensions/webgl-compressed-texture-pvrtc.html [ Failure ]
  webgl/2.0.0/conformance/extensions/webgl-draw-buffers-framebuffer-unsupported.html [ Failure ]
  webgl/2.0.0/conformance/extensions/webgl-draw-buffers.html [ Failure ]
  webgl/2.0.0/conformance/glsl/misc/shaders-with-invariance.html [ Failure ]
  webgl/2.0.0/conformance/renderbuffers/framebuffer-object-attachment.html [ Failure ]
  webgl/2.0.0/conformance/textures/misc/cube-incomplete-fbo.html [ Failure ]
  webgl/2.0.0/conformance2/extensions/ext-color-buffer-float.html [ Failure ]
  webgl/2.0.0/conformance2/extensions/promoted-extensions-in-shaders.html [ Failure ]
  webgl/2.0.0/conformance2/misc/views-with-offsets.html [ Failure ]
  webgl/2.0.0/conformance2/reading/read-pixels-from-fbo-test.html [ Failure ]
  webgl/2.0.0/conformance2/renderbuffers/multisample-with-full-sample-counts.html [ Failure ]
  webgl/2.0.0/conformance2/renderbuffers/multisampled-renderbuffer-initialization.html [ Failure ]
  webgl/2.0.0/conformance2/renderbuffers/readbuffer.html [ Failure ]
  webgl/2.0.0/conformance2/rendering/clear-srgb-color-buffer.html [ Failure ]
  webgl/2.0.0/conformance2/rendering/clipping-wide-points.html [ Failure ]
  webgl/2.0.0/conformance2/state/gl-get-calls.html [ Failure ]
  webgl/2.0.0/conformance2/state/gl-object-get-calls.html [ Failure ]
  webgl/2.0.0/conformance2/textures/misc/copy-texture-image.html [ Failure ]
Comment 13 Jon Lee 2018-09-18 13:34:45 PDT
> [ FAIL ] 1 failures reported

:(((
Comment 14 Justin Fan 2018-09-18 14:49:19 PDT
(In reply to Jon Lee from comment #13)
> > [ FAIL ] 1 failures reported
> 
> :(((

:(

Are you sad because it's incorrectly pluralized, or because there's only one sub-test failure?
Comment 15 Justin Fan 2018-09-18 15:15:44 PDT
Additional failing tests on ios-sim
webgl/2.0.0/conformance/ogles/GL/build/build_009_to_016.html
webgl/2.0.0/conformance/reading/read-pixels-test.html
webgl/2.0.0/conformance/uniforms/uniform-default-values.html
webgl/2.0.0/conformance2/textures/misc/tex-3d-size-limit.html
webgl/2.0.0/conformance2/textures/misc/tex-image-with-bad-args.html
webgl/2.0.0/conformance2/textures/misc/tex-input-validation.html
webgl/2.0.0/conformance2/textures/misc/tex-mipmap-levels.html
webgl/2.0.0/conformance2/textures/misc/tex-new-formats.html
webgl/2.0.0/conformance2/textures/misc/tex-srgb-mipmap.html
webgl/2.0.0/conformance2/textures/misc/tex-storage-2d.html
webgl/2.0.0/conformance2/textures/misc/texture-npot.html
Comment 16 Justin Fan 2018-09-18 17:03:51 PDT
Created attachment 350078 [details]
pitch
Comment 17 Justin Fan 2018-09-19 15:32:44 PDT
<rdar://problem/44468728>
Comment 18 Dean Jackson 2018-09-19 17:19:16 PDT
Comment on attachment 350078 [details]
pitch

Jon be happier with correct pluralisation :)

It would be nice if we could avoid putting things in the top-level TestExpectations, but I don't have a good alternative.
Comment 19 Justin Fan 2018-09-19 17:27:53 PDT
Okay! I'll wait for this patch to land with the new expectations then update the test harness for pluralization updates lol
Comment 20 WebKit Commit Bot 2018-09-19 18:05:11 PDT
Comment on attachment 350078 [details]
pitch

Clearing flags on attachment: 350078

Committed r236236: <https://trac.webkit.org/changeset/236236>
Comment 21 WebKit Commit Bot 2018-09-19 18:05:13 PDT
All reviewed patches have been landed.  Closing bug.
Comment 22 Dawei Fenton (:realdawei) 2018-09-20 11:28:01 PDT
(In reply to WebKit Commit Bot from comment #20)
> Comment on attachment 350078 [details]
> pitch
> 
> Clearing flags on attachment: 350078
> 
> Committed r236236: <https://trac.webkit.org/changeset/236236>

7 tests are failing:
https://build.webkit.org/results/Apple%20High%20Sierra%20Release%20WK2%20(Tests)/r236260%20(6843)/results.html

Marked these tests as [ Failure ] in https://trac.webkit.org/changeset/236272/webkit
Comment 23 Dawei Fenton (:realdawei) 2018-09-20 11:33:11 PDT
(In reply to Dawei Fenton (:realdawei) from comment #22)
> (In reply to WebKit Commit Bot from comment #20)
> > Comment on attachment 350078 [details]
> > pitch
> > 
> > Clearing flags on attachment: 350078
> > 
> > Committed r236236: <https://trac.webkit.org/changeset/236236>
> 
> 7 tests are failing:
> https://build.webkit.org/results/
> Apple%20High%20Sierra%20Release%20WK2%20(Tests)/r236260%20(6843)/results.html
> 
> Marked these tests as [ Failure ] in
> https://trac.webkit.org/changeset/236272/webkit

Tests that were failing were:

webgl/2.0.0/conformance/buffers/buffer-uninitialized.html
webgl/2.0.0/conformance/extensions/webgl-draw-buffers.html
webgl/2.0.0/conformance/renderbuffers/feedback-loop.html
webgl/2.0.0/conformance/rendering/clipping-wide-points.html
webgl/2.0.0/conformance/textures/misc/copy-tex-image-2d-formats.html
webgl/2.0.0/conformance2/state/gl-object-get-calls.html
webgl/2.0.0/conformance2/textures/misc/tex-3d-size-limit.html
Comment 24 Dawei Fenton (:realdawei) 2018-09-20 14:14:18 PDT
The following two tests were timing on out on iOS 11 Simulator WK2-debug:

https://build.webkit.org/results/Apple%20iOS%2011%20Simulator%20Debug%20WK2%20(Tests)/r236269%20(6532)/results.html

webgl/2.0.0/conformance/textures/misc/tex-image-and-sub-image-2d-with-array-buffer-view.html
webgl/2.0.0/conformance2/textures/misc/copy-texture-image-luma-format.html

skipped in: https://trac.webkit.org/changeset/236280/webkit
Comment 25 Jon Lee 2018-09-20 17:19:28 PDT
(In reply to Justin Fan from comment #14)
> (In reply to Jon Lee from comment #13)
> > > [ FAIL ] 1 failures reported
> > 
> > :(((
> 
> :(
> 
> Are you sad because it's incorrectly pluralized

this
Comment 26 Justin Fan 2018-09-21 16:13:56 PDT
Closing this bug for now. Further test expectation modifications should be tracked in new bugs.