RESOLVED FIXED 56047
new-run-webkit-tests causes several tests to fail on mac port 
https://bugs.webkit.org/show_bug.cgi?id=56047
Summary new-run-webkit-tests causes several tests to fail on mac port
Tony Gentilcore
Reported 2011-03-09 13:28:32 PST
new-run-webkit-tests is waaay faster than run-webkit-tests. I'd love to be able to switch to it. However, there are several tests that fail or timeout on new-run-webkit-tests, but pass under run-webkit-tests. This is probably closely related to bug 35358, but that only refers to timeouts. Bug 35266 might also help the ones that timeout. Here's a list from r79638 on the mac port: fast/canvas/webgl/uninitialized-test.html Test timed out Text diff mismatch expected actual diff wdiff pretty diff fast/css/font-face-in-shadow-DOM.html Text diff mismatch expected actual diff wdiff pretty diff fast/forms/restore-selection-after-layout.html Text diff mismatch expected actual diff wdiff pretty diff fast/frames/lots-of-iframes.html Test timed out Text diff mismatch expected actual diff wdiff pretty diff fast/frames/lots-of-objects.html Test timed out Text diff mismatch expected actual diff wdiff pretty diff fast/overflow/lots-of-sibling-inline-boxes.html Test timed out Text diff mismatch expected actual diff wdiff pretty diff fast/parser/block-nesting-cap.html Test timed out Text diff mismatch expected actual diff wdiff pretty diff fast/wcss/wap-input-format.xhtml Text diff mismatch expected actual diff wdiff pretty diff fast/wcss/wap-input-required.xhtml Text diff mismatch expected actual diff wdiff pretty diff fast/workers/storage/use-same-database-in-page-and-workers.html DumpRenderTree crashed stack http/tests/navigation/image-load-in-unload-handler.html Text diff mismatch expected actual diff wdiff pretty diff http/tests/navigation/ping-cookie.html Text diff mismatch expected actual diff wdiff pretty diff http/tests/navigation/ping-same-origin.html Text diff mismatch expected actual diff wdiff pretty diff http/tests/security/isolatedWorld/all-window-properties.html Text diff mismatch expected actual diff wdiff pretty diff http/tests/security/isolatedWorld/all-window-prototypes.html Text diff mismatch expected actual diff wdiff pretty diff http/tests/xmlhttprequest/basic-auth-nopassword.html Text diff mismatch expected actual diff wdiff pretty diff java/lc3/JavaObject/JavaObjectToBoolean-001-n.html Text diff mismatch expected actual diff wdiff pretty diff java/lc3/JavaObject/JavaObjectToBoolean-002-n.html Text diff mismatch expected actual diff wdiff pretty diff java/lc3/JavaObject/JavaObjectToDouble-001.html Text diff mismatch expected actual diff wdiff pretty diff media/invalid-media-url-crash.html DumpRenderTree crashed stack platform/chromium-linux/fast/text/international/complex-joining-using-gpos.html Text diff mismatch expected actual diff wdiff pretty diff sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A1.1_T2.html Test timed out Text diff mismatch expected actual diff wdiff pretty diff sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A1.4_T2.html Test timed out Text diff mismatch expected actual diff wdiff pretty diff sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A2.1_T2.html Test timed out Text diff mismatch expected actual diff wdiff pretty diff sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A2.4_T2.html Test timed out Text diff mismatch expected actual diff wdiff pretty diff sputnik/Unicode/Unicode_218/S7.6_A5.3_T2.html Test timed out Text diff mismatch expected actual diff wdiff pretty diff transitions/cancel-transition.html Text diff mismatch expected actual diff wdiff pretty diff
Attachments
Add attachment proposed patch, testcase, etc.
Dirk Pranke
Comment 1 2011-03-09 13:33:56 PST
Yes, I've noticed this as well.
Dirk Pranke
Comment 2 2011-03-21 11:57:59 PDT
A number of tests appear to be sensitive to the order they are run in, and for some reason NRWT is running the tests in descending alphabetical order inside a directory. I've filed bug 56760 to fix that. There are still some tests that are failing for other reasons, so I'm leaving this open for now.
Dirk Pranke
Comment 3 2011-04-01 19:45:15 PDT
I believe all of the issues that were causing NRWT to not run cleanly (in a single thread, at least) on Mac SL have been resolved. I expect we will still need to shake out some flakiness as a result of contention between multiple processes, but I'll mark this as closed because I've fixed the main issues.
Note You need to log in before you can comment on or make changes to this bug.