Bug 188450 – test262-runner -s --test-only should replace test results

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Keith Miller]

Right now if you run the test262-runner and tell it to save results while only running a subset of the tests it will clear all the other results. It should probably regex replace the results for what actually changed.

[Keith Miller]

Created attachment 346931 [details]
Patch

[Michael Saboff]

Comment on attachment 346931 [details]
Patch

r=me

[WebKit Commit Bot]

Comment on attachment 346931 [details]
Patch

Clearing flags on attachment: 346931

Committed r235359: <https://trac.webkit.org/changeset/235359>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/43750234>

[Michael Catanzaro]

This introduced 10 webkitperl test failures for WPE/GTK:

Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl .................................. Experimental keys on scalar is now forbidden at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262/Runner.pm line 929.
Compilation failed in require at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45.
BEGIN failed--compilation aborted at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45.

Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl .................................. 1/13 
#   Failed test 'test262 test failed, ignore expectations (exit code: 1)'
#   at Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl line 136.
#          got: '255'
#     expected: '1'

#   Failed test 'test262 test failed, ignore expectations (new failures: 2)'
#   at Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl line 142.
#          got: '0'
#     expected: '2'
Experimental keys on scalar is now forbidden at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262/Runner.pm line 929.
Compilation failed in require at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45.
BEGIN failed--compilation aborted at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45.

#   Failed test 'test262 test passed, ignore expectations (exit code: 0)'
#   at Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl line 136.
#          got: '255'
#     expected: '0'
Experimental keys on scalar is now forbidden at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262/Runner.pm line 929.
Compilation failed in require at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45.
BEGIN failed--compilation aborted at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45.

#   Failed test 'test262 tests newly failed (exit code: 1)'
#   at Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl line 136.
#          got: '255'
#     expected: '1'

#   Failed test 'test262 tests newly failed (new failures: 2)'
#   at Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl line 142.
#          got: '0'
#     expected: '2'
Experimental keys on scalar is now forbidden at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262/Runner.pm line 929.
Compilation failed in require at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45.
BEGIN failed--compilation aborted at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45.

#   Failed test 'test262 tests newly passed (exit code: 0)'
#   at Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl line 136.
#          got: '255'
#     expected: '0'
Experimental keys on scalar is now forbidden at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262/Runner.pm line 929.
Compilation failed in require at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45.
BEGIN failed--compilation aborted at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45.

#   Failed test 'test262 tests fails, expected failure (exit code: 0)'
#   at Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl line 136.
#          got: '255'
#     expected: '0'
Experimental keys on scalar is now forbidden at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262/Runner.pm line 929.
Compilation failed in require at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45.
BEGIN failed--compilation aborted at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45.

#   Failed test 'test262 tests fails, with unexpected error string (exit code: 1)'
#   at Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl line 136.
#          got: '255'
#     expected: '1'

#   Failed test 'test262 tests fails, with unexpected error string (new failures: 2)'
#   at Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl line 142.
#          got: '0'
#     expected: '2'
Experimental keys on scalar is now forbidden at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262/Runner.pm line 929.
Compilation failed in require at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45.
BEGIN failed--compilation aborted at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45.

#   Failed test 'expectations yaml file format'
#   at Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl line 157.
# Looks like you failed 10 tests of 13.

                                                                                                            
Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl .................................. Dubious, test returned 10 (wstat 2560, 0xa00)
Failed 10/13 subtests

[Keith Miller]

It seems that perl has deprecated calling keys on a reference. Maybe this will fix that?: https://bugs.webkit.org/show_bug.cgi?id=189261. idk how to perl...