Bug 188450

Summary:

test262-runner -s --test-only should replace test results

Product:

WebKit

Reporter:

Keith Miller <keith_miller>

Component:

Tools / Tests

Assignee:

Keith Miller <keith_miller>

Status:

RESOLVED FIXED

Severity:

Normal

CC:

commit-queue, keith_miller, leo, lforschler, mcatanzaro, msaboff, webkit-bug-importer

Priority:

Keywords:

InRadar

Version:

WebKit Nightly Build

Hardware:

Unspecified

OS:

Unspecified

Attachments:

Description	Flags
Patch	none

Keith Miller

Reported 2018-08-09 14:08:20 PDT

Right now if you run the test262-runner and tell it to save results while only running a subset of the tests it will clear all the other results. It should probably regex replace the results for what actually changed.

Attachments
Patch (2.72 KB, patch) 2018-08-10 14:58 PDT, Keith Miller	no flags	Details Formatted Diff Diff
View All Add attachment proposed patch, testcase, etc.

Keith Miller

Comment 1 2018-08-10 14:58:32 PDT

Created attachment 346931 [details] Patch

Michael Saboff

Comment 2 2018-08-21 10:00:45 PDT

Comment on attachment 346931 [details] Patch r=me

WebKit Commit Bot

Comment 3 2018-08-27 04:49:16 PDT

Comment on attachment 346931 [details] Patch Clearing flags on attachment: 346931 Committed r235359: <https://trac.webkit.org/changeset/235359>

WebKit Commit Bot

Comment 4 2018-08-27 04:49:17 PDT

All reviewed patches have been landed. Closing bug.

Radar WebKit Bug Importer

Comment 5 2018-08-27 04:50:17 PDT

<rdar://problem/43750234>

Michael Catanzaro

Comment 6 2018-09-04 08:49:57 PDT

This introduced 10 webkitperl test failures for WPE/GTK: Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl .................................. Experimental keys on scalar is now forbidden at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262/Runner.pm line 929. Compilation failed in require at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45. BEGIN failed--compilation aborted at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45. Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl .................................. 1/13 # Failed test 'test262 test failed, ignore expectations (exit code: 1)' # at Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl line 136. # got: '255' # expected: '1' # Failed test 'test262 test failed, ignore expectations (new failures: 2)' # at Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl line 142. # got: '0' # expected: '2' Experimental keys on scalar is now forbidden at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262/Runner.pm line 929. Compilation failed in require at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45. BEGIN failed--compilation aborted at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45. # Failed test 'test262 test passed, ignore expectations (exit code: 0)' # at Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl line 136. # got: '255' # expected: '0' Experimental keys on scalar is now forbidden at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262/Runner.pm line 929. Compilation failed in require at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45. BEGIN failed--compilation aborted at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45. # Failed test 'test262 tests newly failed (exit code: 1)' # at Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl line 136. # got: '255' # expected: '1' # Failed test 'test262 tests newly failed (new failures: 2)' # at Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl line 142. # got: '0' # expected: '2' Experimental keys on scalar is now forbidden at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262/Runner.pm line 929. Compilation failed in require at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45. BEGIN failed--compilation aborted at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45. # Failed test 'test262 tests newly passed (exit code: 0)' # at Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl line 136. # got: '255' # expected: '0' Experimental keys on scalar is now forbidden at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262/Runner.pm line 929. Compilation failed in require at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45. BEGIN failed--compilation aborted at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45. # Failed test 'test262 tests fails, expected failure (exit code: 0)' # at Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl line 136. # got: '255' # expected: '0' Experimental keys on scalar is now forbidden at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262/Runner.pm line 929. Compilation failed in require at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45. BEGIN failed--compilation aborted at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45. # Failed test 'test262 tests fails, with unexpected error string (exit code: 1)' # at Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl line 136. # got: '255' # expected: '1' # Failed test 'test262 tests fails, with unexpected error string (new failures: 2)' # at Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl line 142. # got: '0' # expected: '2' Experimental keys on scalar is now forbidden at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262/Runner.pm line 929. Compilation failed in require at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45. BEGIN failed--compilation aborted at /home/slave/webkitgtk/gtk-linux-64-release-tests/build/Tools/Scripts/test262-runner line 45. # Failed test 'expectations yaml file format' # at Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl line 157. # Looks like you failed 10 tests of 13. Tools/Scripts/webkitperl/test262_unittest/test262-runner-tests.pl .................................. Dubious, test returned 10 (wstat 2560, 0xa00) Failed 10/13 subtests

Keith Miller

Comment 7 2018-09-04 10:17:21 PDT

It seems that perl has deprecated calling keys on a reference. Maybe this will fix that?: https://bugs.webkit.org/show_bug.cgi?id=189261. idk how to perl...

Note You need to log in before you can comment on or make changes to this bug.