Bug 181293 – Import the 2.0.0 version of Khronos WebGL conformance suite

• Component: WebGL
• Status: RESOLVED FIXED

[Zan Dobersek]

Should be useful to test both existing WebGL1 support and the WebGL2 development.

[Zan Dobersek]

Created attachment 330470 [details]
Patch

[EWS Watchlist]

Attachment 330470 [details] did not pass style-queue:

Traceback (most recent call last):
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkit-patch", line 84, in <module>
    main()
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkit-patch", line 79, in main
    WebKitPatch(os.path.abspath(__file__)).main()
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/multicommandtool.py", line 305, in main
    result = command.check_arguments_and_execute(options, args, self)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/multicommandtool.py", line 123, in check_arguments_and_execute
    return self.execute(options, args, tool) or 0
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/commands/abstractsequencedcommand.py", line 55, in execute
    self._sequence.run_and_handle_errors(tool, options, state)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 73, in run_and_handle_errors
    self._run(tool, options, state)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 67, in _run
    step(tool, options).run(state)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/steps/checkstyle.py", line 66, in run
    self._tool.executive.run_and_throw_if_fail(self._tool.deprecated_port().check_webkit_style_command() + args, cwd=self._tool.scm().checkout_root)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/common/system/executive.py", line 135, in run_and_throw_if_fail
    exit_code = self._run_command_with_teed_output(args, child_stdout, **kwargs)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/common/system/executive.py", line 107, in _run_command_with_teed_output
    **kwargs)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/common/system/executive.py", line 473, in popen
    return subprocess.Popen(string_args, **kwargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1335, in _execute_child
    raise child_exception
OSError: [Errno 7] Argument list too long


If any of these errors are false positives, please file a bug against check-webkit-style.

[Justin Fan]

*** Bug 186074 has been marked as a duplicate of this bug. ***

[Radar WebKit Bug Importer]

<rdar://problem/40654103>

[Justin Fan]

Other than expected failures due to new webgl 2 tests and updated webgl 1 tests, I’m seeing some timeouts on webkit-test-runner.

 webgl/2.0.0/conformance/glsl/bugs/loop-if-loop-gradient.html [ Timeout ]
 webgl/2.0.0/conformance/glsl/bugs/qualcomm-crash.html [ Timeout ]
 webgl/2.0.0/conformance/glsl/bugs/qualcomm-loop-with-continue-crash.html [ Timeout ]

are timing out due to a call to window.requestAnimationFrame that gets finishTest() passed as the callback. Since webkit-test-runner does not support RAF, finishTest is never called. Changing these calls to wtu.requestAnimFrame(), which will call setTimeout on platforms that do not support RAF, solved the issue.

 webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgb-rgb-unsigned_byte.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgb-rgb-unsigned_short_5_6_5.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgba-rgba-unsigned_byte.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgba-rgba-unsigned_short_4_4_4_4.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgba-rgba-unsigned_short_5_5_5_1.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgb-rgb-unsigned_byte.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgb-rgb-unsigned_short_5_6_5.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgba-rgba-unsigned_byte.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgba-rgba-unsigned_short_4_4_4_4.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgba-rgba-unsigned_short_5_5_5_1.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgb-rgb-unsigned_byte.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgb-rgb-unsigned_short_5_6_5.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgba-rgba-unsigned_byte.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgba-rgba-unsigned_short_4_4_4_4.html [ Timeout ]
 webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgba-rgba-unsigned_short_5_5_5_1.html [ Timeout ]

Are also timing out, as are their conformance2 versions. Still investigating these.

[Zan Dobersek]

(In reply to Justin Fan from comment #5)
> Other than expected failures due to new webgl 2 tests and updated webgl 1
> tests, I’m seeing some timeouts on webkit-test-runner.
> 
>  webgl/2.0.0/conformance/glsl/bugs/loop-if-loop-gradient.html [ Timeout ]
>  webgl/2.0.0/conformance/glsl/bugs/qualcomm-crash.html [ Timeout ]
>  webgl/2.0.0/conformance/glsl/bugs/qualcomm-loop-with-continue-crash.html [
> Timeout ]
> 
> are timing out due to a call to window.requestAnimationFrame that gets
> finishTest() passed as the callback. Since webkit-test-runner does not
> support RAF, finishTest is never called. Changing these calls to
> wtu.requestAnimFrame(), which will call setTimeout on platforms that do not
> support RAF, solved the issue.
> 

Can you clarify why webkit-test-runner (I guess this is WebKitTestRunner binary) doesn't support requestAnimationFrame() in a way that's usable by these tests?

>  webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgb-rgb-
> unsigned_byte.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgb-rgb-
> unsigned_short_5_6_5.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgba-rgba-
> unsigned_byte.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgba-rgba-
> unsigned_short_4_4_4_4.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_blob/tex-2d-rgba-rgba-
> unsigned_short_5_5_5_1.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgb-rgb-
> unsigned_byte.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgb-rgb-
> unsigned_short_5_6_5.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgba-rgba-
> unsigned_byte.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgba-rgba-
> unsigned_short_4_4_4_4.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_canvas/tex-2d-rgba-rgba-
> unsigned_short_5_5_5_1.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgb-rgb-
> unsigned_byte.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgb-rgb-
> unsigned_short_5_6_5.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgba-rgba-
> unsigned_byte.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgba-rgba-
> unsigned_short_4_4_4_4.html [ Timeout ]
>  webgl/2.0.0/conformance/textures/image_bitmap_from_image/tex-2d-rgba-rgba-
> unsigned_short_5_5_5_1.html [ Timeout ]
> 
> Are also timing out, as are their conformance2 versions. Still investigating
> these.

I think these time out due to ImageBitmap not yet being supported as a possible variant of the TexImageSource typedef, and as such not handled in current texImage/texSubImage implementations.

[Justin Fan]

Hi Zan, in js-test-pre.js, WebKit comments:       

// RAF doesn't work in LayoutTests. Disable it so the tests will
// use setTimeout instead.

and requestAnimationFrame is subsequently undefined.

In requestAnimFrame() defined in webgl-test-utils, the function first checks to see if window.requestAnimationFrame is defined. If it isn't, setTimeout() is call instead. 

I realize this explanation doesn't truly answer why "RAF doesn't work in LayoutTests" and I'm trying to find out more.

[Zan Dobersek]

I think this disabling of rAF should probably be revisited. It's the first time I've noticed it, and it's been disabled since the first import of WebGL test suite back in 2013. Other tests are using rAF without any issue.

Re-enabling rAF doesn't have any effect on the tests here on the WPE port (i.e. results don't change).

[Justin Fan]

Hi Zan, 

Having some technical difficulties with my main machine so I can't confirm just yet, but I think you're right about not disabling rAF; that code is pretty old. If it works here I'll put in a PR with Khronos so that it's no longer disabled for WebKit. 

For the ImageBitmap tests, I will mark those as skipped for now. Other than that, the plan is to enable the rest of the /conformance/ tests, and skip the /conformance2/ tests until we've implement support. 

Thanks for your help and input!

[Justin Fan]

https://github.com/KhronosGroup/WebGL/pull/2656

[Justin Fan]

Created attachment 342229 [details]
Importing and skipping the WebGL Conformance Suite 2.0.0

[Justin Fan]

Created attachment 342232 [details]
Patch

Importing, generating, and skipping suite 2.0.0

[Justin Fan]

Having serious fun trying to apply this patch. Will attempt dir-by-dir.

[Justin Fan]

Created attachment 342345 [details]
Full Patch try #2

[EWS Watchlist]

Attachment 342345 [details] did not pass style-queue:

Traceback (most recent call last):
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkit-patch", line 84, in <module>
    main()
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkit-patch", line 79, in main
    WebKitPatch(os.path.abspath(__file__)).main()
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/multicommandtool.py", line 305, in main
    result = command.check_arguments_and_execute(options, args, self)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/multicommandtool.py", line 123, in check_arguments_and_execute
    return self.execute(options, args, tool) or 0
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/commands/abstractsequencedcommand.py", line 55, in execute
    self._sequence.run_and_handle_errors(tool, options, state)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 73, in run_and_handle_errors
    self._run(tool, options, state)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 67, in _run
    step(tool, options).run(state)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/steps/checkstyle.py", line 66, in run
    self._tool.executive.run_and_throw_if_fail(self._tool.deprecated_port().check_webkit_style_command() + args, cwd=self._tool.scm().checkout_root)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/common/system/executive.py", line 135, in run_and_throw_if_fail
    exit_code = self._run_command_with_teed_output(args, child_stdout, **kwargs)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/common/system/executive.py", line 107, in _run_command_with_teed_output
    **kwargs)
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/common/system/executive.py", line 481, in popen
    return subprocess.Popen(string_args, **kwargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1335, in _execute_child
    raise child_exception
OSError: [Errno 7] Argument list too long


If any of these errors are false positives, please file a bug against check-webkit-style.

[Alex Christensen]

http://trac.webkit.org/r232783