Bug 42215 – r62810 added bad baselines?

• Status: RESOLVED
• Resolution: DUPLICATE
• Component: WebCore JavaScript

[Albert J. Wong]

I think r62810 might have added a number of bad baselines for ietestcenter tests.  Looking through the *-expected.txt files in LayoutTests/ietestcenter/Javascript, there's a number of them that say FAIL.  I did a "grep -5H FAIL *" in that directory and visually inspected the results, and I think most of these baselines actually baseline to a failure.  Here's a list of failing tests:


11.4.1-4.a-10-expected.txt
11.4.1-4.a-8-expected.txt
15.12.1.1-0-2-expected.txt
15.12.1.1-0-3-expected.txt
15.12.2-0-2-expected.txt
15.12.3-0-2-expected.txt
15.2.3.10-0-1-expected.txt
15.2.3.10-0-2-expected.txt
15.2.3.10-1-expected.txt
15.2.3.10-2-expected.txt
15.2.3.11-0-1-expected.txt
15.2.3.11-0-2-expected.txt
15.2.3.11-1-expected.txt
15.2.3.11-4-1-expected.txt
15.2.3.11-4-10-expected.txt
15.2.3.11-4-11-expected.txt
15.2.3.11-4-12-expected.txt
15.2.3.11-4-13-expected.txt
15.2.3.11-4-14-expected.txt
15.2.3.11-4-15-expected.txt
15.2.3.11-4-16-expected.txt
15.2.3.11-4-17-expected.txt
15.2.3.11-4-18-expected.txt
15.2.3.11-4-19-expected.txt
15.2.3.11-4-2-expected.txt
15.2.3.11-4-20-expected.txt
15.2.3.11-4-21-expected.txt
15.2.3.11-4-22-expected.txt
15.2.3.11-4-23-expected.txt
15.2.3.11-4-24-expected.txt
15.2.3.11-4-25-expected.txt
15.2.3.11-4-26-expected.txt
15.2.3.11-4-27-expected.txt
15.2.3.11-4-3-expected.txt
15.2.3.11-4-4-expected.txt
15.2.3.11-4-5-expected.txt
15.2.3.11-4-6-expected.txt
15.2.3.11-4-7-expected.txt
15.2.3.11-4-8-expected.txt
15.2.3.11-4-9-expected.txt
15.2.3.12-0-1-expected.txt
15.2.3.12-0-2-expected.txt
15.2.3.12-1-expected.txt
15.2.3.12-3-1-expected.txt
15.2.3.12-3-10-expected.txt
15.2.3.12-3-11-expected.txt
15.2.3.12-3-12-expected.txt
15.2.3.12-3-13-expected.txt
15.2.3.12-3-14-expected.txt
15.2.3.12-3-15-expected.txt
15.2.3.12-3-16-expected.txt
15.2.3.12-3-17-expected.txt
15.2.3.12-3-18-expected.txt
15.2.3.12-3-19-expected.txt
15.2.3.12-3-2-expected.txt
15.2.3.12-3-20-expected.txt
15.2.3.12-3-21-expected.txt
15.2.3.12-3-22-expected.txt
15.2.3.12-3-23-expected.txt
15.2.3.12-3-24-expected.txt
15.2.3.12-3-25-expected.txt
15.2.3.12-3-26-expected.txt
15.2.3.12-3-27-expected.txt
15.2.3.12-3-3-expected.txt
15.2.3.12-3-4-expected.txt
15.2.3.12-3-5-expected.txt
15.2.3.12-3-6-expected.txt
15.2.3.12-3-7-expected.txt
15.2.3.12-3-8-expected.txt
15.2.3.12-3-9-expected.txt
15.2.3.13-0-1-expected.txt
15.2.3.13-0-2-expected.txt
15.2.3.13-0-3-expected.txt
15.2.3.13-1-expected.txt

15.2.3.13-2-1-expected.txt
15.2.3.13-2-10-expected.txt
15.2.3.13-2-11-expected.txt
15.2.3.13-2-12-expected.txt
15.2.3.13-2-13-expected.txt
15.2.3.13-2-14-expected.txt
15.2.3.13-2-15-expected.txt
15.2.3.13-2-16-expected.txt
15.2.3.13-2-17-expected.txt
15.2.3.13-2-18-expected.txt
15.2.3.13-2-19-expected.txt
15.2.3.13-2-2-expected.txt
15.2.3.13-2-20-expected.txt
15.2.3.13-2-21-expected.txt
15.2.3.13-2-3-expected.txt
15.2.3.13-2-4-expected.txt
15.2.3.13-2-5-expected.txt
15.2.3.13-2-6-expected.txt
15.2.3.13-2-7-expected.txt
15.2.3.13-2-8-expected.txt
15.2.3.13-2-9-expected.txt
15.2.3.14-2-4-expected.txt
15.2.3.14-2-5-expected.txt
15.2.3.14-2-6-expected.txt
15.2.3.14-3-4-expected.txt
15.2.3.3-4-188-expected.txt
15.2.3.3-4-20-expected.txt
15.2.3.3-4-21-expected.txt
15.2.3.3-4-22-expected.txt
15.2.3.3-4-23-expected.txt
15.2.3.3-4-24-expected.txt
15.2.3.3-4-25-expected.txt
15.2.3.3-4-38-expected.txt
15.2.3.4-4-2-expected.txt
15.2.3.6-4-1-expected.txt
15.2.3.8-0-1-expected.txt
15.2.3.8-0-2-expected.txt
15.2.3.8-1-expected.txt
15.2.3.9-0-1-expected.txt
15.2.3.9-0-2-expected.txt
15.2.3.9-1-expected.txt
15.3.4.5-0-1-expected.txt
15.3.4.5-0-2-expected.txt
15.3.4.5-13.b-1-expected.txt
15.3.4.5-13.b-2-expected.txt
15.3.4.5-13.b-3-expected.txt
15.3.4.5-13.b-4-expected.txt
15.3.4.5-13.b-5-expected.txt
15.3.4.5-15-1-expected.txt
15.3.4.5-15-2-expected.txt
15.3.4.5-16-1-expected.txt
15.3.4.5-2-1-expected.txt
15.3.4.5-2-2-expected.txt
15.3.4.5-2-3-expected.txt
15.3.4.5-2-4-expected.txt
15.3.4.5-2-5-expected.txt
15.3.4.5-2-6-expected.txt
15.3.4.5-2-7-expected.txt
15.3.4.5-2-8-expected.txt
15.3.4.5-2-9-expected.txt
15.3.4.5-8-1-expected.txt
15.3.4.5-8-2-expected.txt
15.3.4.5-9-1-expected.txt
15.3.4.5-9-2-expected.txt
15.4.4.14-2-5-expected.txt
15.4.4.14-2-6-expected.txt
15.4.4.14-5-28-expected.txt
15.4.4.14-9-a-12-expected.txt
15.4.4.15-2-5-expected.txt
15.4.4.15-2-6-expected.txt
15.4.4.15-3-26-expected.txt
15.4.4.15-3-27-expected.txt
15.4.4.15-5-1-expected.txt
15.4.4.15-5-12-expected.txt
15.4.4.15-5-14-expected.txt
15.4.4.15-5-16-expected.txt
15.4.4.15-5-28-expected.txt
15.4.4.15-5-4-expected.txt
15.4.4.15-8-9-expected.txt
15.4.4.15-8-a-12-expected.txt

[Albert J. Wong]

Would it be better to just remove these baselines?

[Gavin Barraclough]

Not adding the results isn't a good solution, since this will result in everyone be prompted that they have new (failing) results.

Only partially importing the test suite would be somewhat confusing, and would not be our normal approach.

An alternative approach would be to add some/all of the failing tests to the skip-lists, and to remove their expected results.

In landing the tests with failing results I followed the precedent of the recently landed sputnik tests, plus a number or other older existing tests in fast/js.  An advantage to this approach is that it is trivially obvious to us which tests should reenable as we fix bugs; if we skip the tests then we will not be automatically warned as we fix problems and as such passing tests may be left skipped (and as would not be protected from being broken again).

cheers,
G.

*** This bug has been marked as a duplicate of bug 41876 ***