47293 2010-10-06 12:38:56 -0700 run-webkit-tests should identify if new failing/passing platform specific results match general expected results 2015-02-23 10:42:31 -0800 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) PC OS X 10.5 NEW P2 Normal --- 1 joepeck webkit-unassigned ap ddkilzer simon.fraser oldest_to_newest 290522 0 joepeck 2010-10-06 12:38:56 -0700 If a platform has expected results for a failing test, and eventually that test starts passing, it would be great if run-webkit-tests could point out that the new results match the general, non-platform-specific results. Example: LayoutTests/platform/fake-platform/fast/blah-expected.txt => "FAIL - not yet supported" LayoutTests/fast/blah-expected.txt => "PASS" After implementing "blah" and running tests you could see something like: Tests where results did not match expected results: fast/blah.html expected | actual | diff | pretty diff | * Tests now matching non-platform-specific results: fast/blah.html It would also be helpful if run-webkit-tests pointed out that the results are platform specific for a dumpAsText() test. 1071242 1 ddkilzer 2015-02-23 10:39:28 -0800 Other possibilities: - Have run-webkit-tests (with a special flag) not only mark the test as passing for this mode, but also delete the platform-specific result on disk. - Have run-webkit-tests add a --lint-results switch (or similar) that would (for a given platform) check that its most specific result matches the immediate upstream result and (optionally delete it when that happens). d 1071245 2 ddkilzer 2015-02-23 10:42:31 -0800 Maybe this is just another way of running --new-baseline, though? See Bug 141913: run-webkit-tests --new-baseline should not write new result (or delete existing result) if upstream results match