Bug 56047
| Summary: | new-run-webkit-tests causes several tests to fail on mac port | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Tony Gentilcore <tonyg> |
| Component: | Tools / Tests | Assignee: | Dirk Pranke <dpranke> |
| Status: | RESOLVED FIXED | ||
| Severity: | Normal | CC: | dpranke, mihaip |
| Priority: | P2 | ||
| Version: | 528+ (Nightly build) | ||
| Hardware: | PC | ||
| OS: | OS X 10.5 | ||
| Bug Depends on: | 56760, 57662 | ||
| Bug Blocks: | 34984 | ||
Tony Gentilcore
new-run-webkit-tests is waaay faster than run-webkit-tests. I'd love to be able to switch to it. However, there are several tests that fail or timeout on new-run-webkit-tests, but pass under run-webkit-tests. This is probably closely related to bug 35358, but that only refers to timeouts. Bug 35266 might also help the ones that timeout.
Here's a list from r79638 on the mac port:
fast/canvas/webgl/uninitialized-test.html
Test timed out
Text diff mismatch expected actual diff wdiff pretty diff
fast/css/font-face-in-shadow-DOM.html
Text diff mismatch expected actual diff wdiff pretty diff
fast/forms/restore-selection-after-layout.html
Text diff mismatch expected actual diff wdiff pretty diff
fast/frames/lots-of-iframes.html
Test timed out
Text diff mismatch expected actual diff wdiff pretty diff
fast/frames/lots-of-objects.html
Test timed out
Text diff mismatch expected actual diff wdiff pretty diff
fast/overflow/lots-of-sibling-inline-boxes.html
Test timed out
Text diff mismatch expected actual diff wdiff pretty diff
fast/parser/block-nesting-cap.html
Test timed out
Text diff mismatch expected actual diff wdiff pretty diff
fast/wcss/wap-input-format.xhtml
Text diff mismatch expected actual diff wdiff pretty diff
fast/wcss/wap-input-required.xhtml
Text diff mismatch expected actual diff wdiff pretty diff
fast/workers/storage/use-same-database-in-page-and-workers.html
DumpRenderTree crashed stack
http/tests/navigation/image-load-in-unload-handler.html
Text diff mismatch expected actual diff wdiff pretty diff
http/tests/navigation/ping-cookie.html
Text diff mismatch expected actual diff wdiff pretty diff
http/tests/navigation/ping-same-origin.html
Text diff mismatch expected actual diff wdiff pretty diff
http/tests/security/isolatedWorld/all-window-properties.html
Text diff mismatch expected actual diff wdiff pretty diff
http/tests/security/isolatedWorld/all-window-prototypes.html
Text diff mismatch expected actual diff wdiff pretty diff
http/tests/xmlhttprequest/basic-auth-nopassword.html
Text diff mismatch expected actual diff wdiff pretty diff
java/lc3/JavaObject/JavaObjectToBoolean-001-n.html
Text diff mismatch expected actual diff wdiff pretty diff
java/lc3/JavaObject/JavaObjectToBoolean-002-n.html
Text diff mismatch expected actual diff wdiff pretty diff
java/lc3/JavaObject/JavaObjectToDouble-001.html
Text diff mismatch expected actual diff wdiff pretty diff
media/invalid-media-url-crash.html
DumpRenderTree crashed stack
platform/chromium-linux/fast/text/international/complex-joining-using-gpos.html
Text diff mismatch expected actual diff wdiff pretty diff
sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A1.1_T2.html
Test timed out
Text diff mismatch expected actual diff wdiff pretty diff
sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A1.4_T2.html
Test timed out
Text diff mismatch expected actual diff wdiff pretty diff
sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A2.1_T2.html
Test timed out
Text diff mismatch expected actual diff wdiff pretty diff
sputnik/Conformance/07_Lexical_Conventions/7.8_Literals/7.8.5_Regular_Expression_Literals/S7.8.5_A2.4_T2.html
Test timed out
Text diff mismatch expected actual diff wdiff pretty diff
sputnik/Unicode/Unicode_218/S7.6_A5.3_T2.html
Test timed out
Text diff mismatch expected actual diff wdiff pretty diff
transitions/cancel-transition.html
Text diff mismatch expected actual diff wdiff pretty diff
| Attachments | ||
|---|---|---|
| Add attachment proposed patch, testcase, etc. |
Dirk Pranke
Yes, I've noticed this as well.
Dirk Pranke
A number of tests appear to be sensitive to the order they are run in, and for some reason NRWT is running the tests in descending alphabetical order inside a directory. I've filed bug 56760 to fix that.
There are still some tests that are failing for other reasons, so I'm leaving this open for now.
Dirk Pranke
I believe all of the issues that were causing NRWT to not run cleanly (in a single thread, at least) on Mac SL have been resolved. I expect we will still need to shake out some flakiness as a result of contention between multiple processes, but I'll mark this as closed because I've fixed the main issues.