60381 2011-05-06 10:34:18 -0700 NRWT has unexpected failures for MISSING tests when run with --no-pixel 2012-06-08 16:13:49 -0700 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) Unspecified Unspecified RESOLVED WONTFIX P2 Normal --- 64367 1 tony webkit-unassigned abarth dpranke eric ojan oldest_to_newest 399614 0 tony 2011-05-06 10:34:18 -0700 abarth is trying to get the ews chromium linux bots to run layout tests. He's using --no-pixel to start. There are two tests that are marked as MISSING because there are no png files checked in. When running running with --no-pixel, these tests fail text diff (i.e., TEXT failure) and show up as unexpected failures. Maybe when running --no-pixel, if a test is marked as MISSING we still need to check for a png file? Another possibility is to have MISSING imply other failures, but that reduces the incentives to remove MISSING from test_expectations.txt. 399622 1 dpranke 2011-05-06 10:41:54 -0700 Hm. Well, a lame workaround would be to mark the test as TEXT MISSING. I will mull this over further. 399789 2 ojan 2011-05-06 13:52:38 -0700 I've actually wanted us to get rid of MISSING and instead having MISSING-TEXT, MISSING-IMAGE, MISSING-IMAGE+TEXT, MISSING-AUDIO. This would be a step in the direction of having the JSON be a 1:1 map to the TestFailure types. I'd still be OK with allowing people to put MISSING in the test_expectations file and it would just be syntactic sugar for any of the above. We'd be able to remove is_missing_audio, is_missing_text and is_missing_image from the json we output at http://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/layout_tests/layout_package/manager.py#L164 399790 3 ojan 2011-05-06 13:54:32 -0700 I guess that doesn't actually solve this problem. Ugh. Sorry for the noise. 399793 4 dpranke 2011-05-06 14:02:45 -0700 Having thought about this further, I think the lame workaround should be good enough. MISSING is an unusual expectation that is trivially fixed, and I don't think it makes sense to alter the code to do anything about it. What do others think? 399809 5 ojan 2011-05-06 14:21:43 -0700 I agree. We need to work to simplify the test_expectations syntax as much as possible, not complicate it. I'm OK with Tony's first solution of still checking for a png file when running --no-pixel if it's not a dumpAsText test. 399817 6 tony 2011-05-06 14:25:48 -0700 Updating test_expectations.txt is fine with me. I think platform/gtk/fonts/fontconfig-synthetic-bold.html is the only one left now. I poked the other test author and I think a png got checked in for fast/forms/date-input-visible-strings.html. 399822 7 dpranke 2011-05-06 14:34:46 -0700 (In reply to comment #5) > I agree. We need to work to simplify the test_expectations syntax as much as possible, not complicate it. > > I'm OK with Tony's first solution of still checking for a png file when running --no-pixel if it's not a dumpAsText test. Unfortunately NRWT doesn't know if a test is a dumpAsText()/dumpAsMarkup() test. It can't distinguish a test that will never generate PNGs from a test with missing PNGs except insofar as it'll see that there are audio or reftest expectations instead. 436018 8 eric 2011-07-12 15:23:07 -0700 This seems related to bug 64367? 482726 9 eric 2011-10-12 14:35:02 -0700 I assume this is still an issue? 482750 10 dpranke 2011-10-12 14:49:39 -0700 nothing changed in the code to address this bug. However, I'm not clear if this is still considered a bug worth doing anything about. 645116 11 dpranke 2012-06-08 16:13:49 -0700 I'm going to close this as WONTFIX; please re-open or file a new bug if anyone would like to see something different happen here.