Bug 181293

Summary: Import the 2.0.0 version of Khronos WebGL conformance suite
Product: WebKit Reporter: Zan Dobersek <zan>
Component: WebGLAssignee: Justin Fan <justin_fan>
Status: RESOLVED FIXED    
Severity: Normal CC: achristensen, dino, ews-watchlist, justin_fan, rniwa, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Bug Depends on:    
Bug Blocks: 214792    
Attachments:
Description Flags
Patch
none
Importing and skipping the WebGL Conformance Suite 2.0.0
none
Patch
none
Full Patch try #2 dino: review+

Description Zan Dobersek 2018-01-04 08:56:07 PST
Should be useful to test both existing WebGL1 support and the WebGL2 development.
Comment 1 Zan Dobersek 2018-01-04 09:39:50 PST
Created attachment 330470 [details]
Patch
Comment 2 EWS Watchlist 2018-01-04 13:15:29 PST
Attachment 330470 [details] did not pass style-queue:

Traceback (most recent call last):
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkit-patch", line 84, in <module>
    main()
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkit-patch", line 79, in main
    WebKitPatch(os.path.abspath(__file__)).main()
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/multicommandtool.py", line 305, in main
    result = command.check_arguments_and_execute(options, args, self)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/multicommandtool.py", line 123, in check_arguments_and_execute
    return self.execute(options, args, tool) or 0
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/commands/abstractsequencedcommand.py", line 55, in execute
    self._sequence.run_and_handle_errors(tool, options, state)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 73, in run_and_handle_errors
    self._run(tool, options, state)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 67, in _run
    step(tool, options).run(state)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/steps/checkstyle.py", line 66, in run
    self._tool.executive.run_and_throw_if_fail(self._tool.deprecated_port().check_webkit_style_command() + args, cwd=self._tool.scm().checkout_root)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/common/system/executive.py", line 135, in run_and_throw_if_fail
    exit_code = self._run_command_with_teed_output(args, child_stdout, **kwargs)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/common/system/executive.py", line 107, in _run_command_with_teed_output
    **kwargs)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/common/system/executive.py", line 473, in popen
    return subprocess.Popen(string_args, **kwargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1335, in _execute_child
    raise child_exception
OSError: [Errno 7] Argument list too long


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 3 Justin Fan 2018-05-29 17:33:13 PDT
*** Bug 186074 has been marked as a duplicate of this bug. ***
Comment 4 Radar WebKit Bug Importer 2018-05-30 10:25:58 PDT
<rdar://problem/40654103>
Comment 5 Justin Fan 2018-06-01 18:13:15 PDT
Other than expected failures due to new webgl 2 tests and updated webgl 1 tests, I’m seeing some timeouts on webkit-test-runner.

 webgl/2.0.0/conformance/glsl/bugs/loop-if-loop-gradient.html [ Timeout ]
 webgl/2.0.0/conformance/glsl/bugs/qualcomm-crash.html [ Timeout ]
 webgl/2.0.0/conformance/glsl/bugs/qualcomm-loop-with-continue-crash.html [ Timeout ]

are timing out due to a call to window.requestAnimationFrame that gets finishTest() passed as the callback. Since webkit-test-runner does not support RAF, finishTest is never called. Changing these calls to wtu.requestAnimFrame(), which will call setTimeout on platforms that do not support RAF, solved the issue.

 webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgb-rgb-unsigned_byte.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgb-rgb-unsigned_short_5_6_5.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgba-rgba-unsigned_byte.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgba-rgba-unsigned_short_4_4_4_4.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgba-rgba-unsigned_short_5_5_5_1.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgb-rgb-unsigned_byte.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgb-rgb-unsigned_short_5_6_5.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgba-rgba-unsigned_byte.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgba-rgba-unsigned_short_4_4_4_4.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgba-rgba-unsigned_short_5_5_5_1.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgb-rgb-unsigned_byte.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgb-rgb-unsigned_short_5_6_5.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgba-rgba-unsigned_byte.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgba-rgba-unsigned_short_4_4_4_4.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgba-rgba-unsigned_short_5_5_5_1.html [ Timeout ]

Are also timing out, as are their conformance2 versions. Still investigating these.
Comment 6 Zan Dobersek 2018-06-05 07:05:00 PDT
(In reply to Justin Fan from comment #5)
> Other than expected failures due to new webgl 2 tests and updated webgl 1
> tests, I’m seeing some timeouts on webkit-test-runner.
> 
>  webgl/2.0.0/conformance/glsl/bugs/loop-if-loop-gradient.html [ Timeout ]
>  webgl/2.0.0/conformance/glsl/bugs/qualcomm-crash.html [ Timeout ]
>  webgl/2.0.0/conformance/glsl/bugs/qualcomm-loop-with-continue-crash.html [
> Timeout ]
> 
> are timing out due to a call to window.requestAnimationFrame that gets
> finishTest() passed as the callback. Since webkit-test-runner does not
> support RAF, finishTest is never called. Changing these calls to
> wtu.requestAnimFrame(), which will call setTimeout on platforms that do not
> support RAF, solved the issue.
> 

Can you clarify why webkit-test-runner (I guess this is WebKitTestRunner binary) doesn't support requestAnimationFrame() in a way that's usable by these tests?

>  webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgb-rgb-
> unsigned_byte.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgb-rgb-
> unsigned_short_5_6_5.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgba-rgba-
> unsigned_byte.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgba-rgba-
> unsigned_short_4_4_4_4.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgba-rgba-
> unsigned_short_5_5_5_1.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgb-rgb-
> unsigned_byte.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgb-rgb-
> unsigned_short_5_6_5.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgba-rgba-
> unsigned_byte.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgba-rgba-
> unsigned_short_4_4_4_4.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgba-rgba-
> unsigned_short_5_5_5_1.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgb-rgb-
> unsigned_byte.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgb-rgb-
> unsigned_short_5_6_5.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgba-rgba-
> unsigned_byte.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgba-rgba-
> unsigned_short_4_4_4_4.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgba-rgba-
> unsigned_short_5_5_5_1.html [ Timeout ]
> 
> Are also timing out, as are their conformance2 versions. Still investigating
> these.

I think these time out due to ImageBitmap not yet being supported as a possible variant of the TexImageSource typedef, and as such not handled in current texImage/texSubImage implementations.
Comment 7 Justin Fan 2018-06-05 14:15:03 PDT
Hi Zan, in js-test-pre.js, WebKit comments:       

// RAF doesn't work in LayoutTests. Disable it so the tests will
// use setTimeout instead.

and requestAnimationFrame is subsequently undefined.

In requestAnimFrame() defined in webgl-test-utils, the function first checks to see if window.requestAnimationFrame is defined. If it isn't, setTimeout() is call instead. 

I realize this explanation doesn't truly answer why "RAF doesn't work in LayoutTests" and I'm trying to find out more.
Comment 8 Zan Dobersek 2018-06-06 01:09:18 PDT
I think this disabling of rAF should probably be revisited. It's the first time I've noticed it, and it's been disabled since the first import of WebGL test suite back in 2013. Other tests are using rAF without any issue.

Re-enabling rAF doesn't have any effect on the tests here on the WPE port (i.e. results don't change).
Comment 9 Justin Fan 2018-06-06 14:41:43 PDT
Hi Zan, 

Having some technical difficulties with my main machine so I can't confirm just yet, but I think you're right about not disabling rAF; that code is pretty old. If it works here I'll put in a PR with Khronos so that it's no longer disabled for WebKit. 

For the ImageBitmap tests, I will mark those as skipped for now. Other than that, the plan is to enable the rest of the /conformance/ tests, and skip the /conformance2/ tests until we've implement support. 

Thanks for your help and input!
Comment 10 Justin Fan 2018-06-07 14:47:40 PDT
https://github.com/KhronosGroup/WebGL/pull/2656
Comment 11 Justin Fan 2018-06-07 18:57:28 PDT
Created attachment 342229 [details]
Importing and skipping the WebGL Conformance Suite 2.0.0
Comment 12 Justin Fan 2018-06-07 19:56:38 PDT
Created attachment 342232 [details]
Patch

Importing, generating, and skipping suite 2.0.0
Comment 13 Justin Fan 2018-06-08 14:02:08 PDT
Having serious fun trying to apply this patch. Will attempt dir-by-dir.
Comment 14 Justin Fan 2018-06-08 16:47:06 PDT
Created attachment 342345 [details]
Full Patch try #2
Comment 15 EWS Watchlist 2018-06-08 19:59:39 PDT
Attachment 342345 [details] did not pass style-queue:

Traceback (most recent call last):
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkit-patch", line 84, in <module>
    main()
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkit-patch", line 79, in main
    WebKitPatch(os.path.abspath(__file__)).main()
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/multicommandtool.py", line 305, in main
    result = command.check_arguments_and_execute(options, args, self)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/multicommandtool.py", line 123, in check_arguments_and_execute
    return self.execute(options, args, tool) or 0
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/commands/abstractsequencedcommand.py", line 55, in execute
    self._sequence.run_and_handle_errors(tool, options, state)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 73, in run_and_handle_errors
    self._run(tool, options, state)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 67, in _run
    step(tool, options).run(state)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/steps/checkstyle.py", line 66, in run
    self._tool.executive.run_and_throw_if_fail(self._tool.deprecated_port().check_webkit_style_command() + args, cwd=self._tool.scm().checkout_root)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/common/system/executive.py", line 135, in run_and_throw_if_fail
    exit_code = self._run_command_with_teed_output(args, child_stdout, **kwargs)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/common/system/executive.py", line 107, in _run_command_with_teed_output
    **kwargs)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/common/system/executive.py", line 481, in popen
    return subprocess.Popen(string_args, **kwargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1335, in _execute_child
    raise child_exception
OSError: [Errno 7] Argument list too long


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 16 Alex Christensen 2018-06-12 17:29:22 PDT
http://trac.webkit.org/r232783