Bug 19445 - REGRESSION: jQuery test Suite fails on multiple tests
Summary: REGRESSION: jQuery test Suite fails on multiple tests
Status: CLOSED WORKSFORME
Alias: None
Product: WebKit
Classification: Unclassified
Component: JavaScriptCore (show other bugs)
Version: 528+ (Nightly build)
Hardware: Mac OS X 10.5
: P1 Normal
Assignee: Nobody
URL: http://jquery.com/test/
Keywords: InRadar, Regression
Depends on:
Blocks:
 
Reported: 2008-06-09 02:44 PDT by Alexander Willner
Modified: 2008-11-28 05:29 PST (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexander Willner 2008-06-09 02:44:28 PDT
First comment:
I'm sure this problem is known but I couldn't find a related bug searching for "jQuery".

Short description:
Using the latest Nightly Build (r34367) under OS X (10.5.3) the linked jQuery Test Suits fails at the following tests (test.subtest):
 * 101.4
 * 101.5
 * 106.10
 * 113.5
 * 113.6
 * 134.1

Got: "6 tests of 1159 failed."

Should be: "0 tests of 1159 failed."

Second comment:
With Safari 3.1 the same and some further tests fail too. Theses tests are fine with Mozilla Firefox 3.0 (Gecko/2008053008) - here the tests 139.2 and 139.3 is the only one that fail.
Comment 1 Alexey Proskuryakov 2008-06-09 20:27:53 PDT
In Safari 3.1.1, I'm seeing one test fail (64.1), and 3 failures in a local debug build of r34472 (101.4, 101.5, 16.10).
Comment 2 Alexey Proskuryakov 2008-06-09 20:28:20 PDT
<rdar://problem/5996602>
Comment 3 Alexander Willner 2008-06-10 06:33:05 PDT
(In reply to comment #1)
> In Safari 3.1.1, I'm seeing one test fail (64.1), and 3 failures in a local
> debug build of r34472 (101.4, 101.5, 16.10).

Well, I've was confused why you've other results then me (OS X 10.5.3, PowerPC, no plugins enabled) - so started some tests. It seems that the jQuery Test Suit isn't deterministic (at least it often shows different results). Here we come:

Safari (3.1.1/5525.20)
 * Run: 1 and 2
   * Tests completed in ~269571 milliseconds.
   * 9 tests of 1161 failed. (101.4|5, 106.10, 113.5|6|7, 114.5, )
 * Run: 3
   * Tests completed in 221206 milliseconds.
   * 5 tests of 1159 failed. (64.1, 101.4|5, 106.10, 134.1)

Safari (WebKit r34469)
 * Run: 0 (see initial posting)
   * 6 tests of 1159 failed. (101.4|5, 106.10, 113.5|6, 134.1)
 * Run: 1-3
   * Tests completed in ~180258 milliseconds.
   * 4 tests of 1159 failed. (101.4|5, 106.10, 134.1)
 * Run: 4
   * Tests completed in 361665 milliseconds.
   * 10 tests of 1161 failed. (101.4|5, 106.10, 113.5|6|7, 114.5, 134.1, 139.2|3)
 * Run: 5
   * Tests completed in 213043 milliseconds.
   * 8 tests of 1161 failed. (101.4|5, 106.10, 113.5|6|7, 114.5, 134.1)

Camino (Version 1.6.1 (1.8.1.14 2008051211))
 * Best run (till every test succeeded)
   * Tests completed in 448219 milliseconds.
   * 0 tests of 1157 failed.

Firefox (Gecko/2008053008 Firefox/3.0)
 * Best run (till every test succeeded)
   * Tests completed in 723878 milliseconds.
   * 0 tests of 1157 failed.

Comment 4 Alexander Willner 2008-06-11 02:46:08 PDT
And here with Safari 4 (Version/4.0dp1 Safari/526.11.2)
 * Run: 1
   * Tests completed in 109576 milliseconds.
   * 0 tests of 1157 failed.
 * Run: 2
   * Tests completed in 176862 milliseconds.
   * 4 tests of 1159 failed. (101.4|5, 106.10, 134.1)
Comment 5 Alexander Willner 2008-06-18 23:41:32 PDT
Since this problem occurs with Firefox, Safari and Opera I've opened a ticket at jQuery: http://dev.jquery.com/ticket/3062
Comment 6 Cameron Zwarich (cpst) 2008-06-29 02:14:18 PDT
All tests pass for me with the r34824 nightly and a local debug build of r34870. Can we close this?
Comment 7 Alexey Proskuryakov 2008-06-30 08:55:41 PDT
With a local debug build, I saw these tests fail: 101.4, 101.5, 106.10 (i.e. the same as in comment 1, modulo a typo). According to a comment in <http://dev.jquery.com/ticket/3062>, failures in "ajax module" test are sort of expected, and do not necessarily indicate any problems.
Comment 8 Cameron Zwarich (cpst) 2008-11-10 11:04:42 PST
All tests have been passing for a while. I will close this, and a new bug can be made for any particular test failures if they occur in the future.
Comment 9 Alexander Willner 2008-11-10 11:38:11 PST
(In reply to comment #8)
> All tests have been passing for a while. I will close this, and a new bug can
> be made for any particular test failures if they occur in the future.
> 

For me (latest build (r38240), PowerBook G4 (PowerPC), OS-X 10.5) test 146 still fails in 5 of 5 runs:

"146. fx module: animate option (queue === false) (1, 0, 1)"
Tests completed in 95714 milliseconds.
1 tests of 1270 failed.
Comment 10 Alexey Proskuryakov 2008-11-27 07:41:40 PST
(In reply to comment #9)
> For me (latest build (r38240), PowerBook G4 (PowerPC), OS-X 10.5) test 146
> still fails in 5 of 5 runs:

I can't reproduce this on Intel. Could you please file a new bug for this?
Comment 11 Alexander Willner 2008-11-28 05:29:31 PST
(In reply to comment #10)
> (In reply to comment #9)
> > For me (latest build (r38240), PowerBook G4 (PowerPC), OS-X 10.5) test 146
> > still fails in 5 of 5 runs:
> 
> I can't reproduce this on Intel. Could you please file a new bug for this?
> 

Finally with the latest nightly build I cannot reproduce that error anymore.