96842 2012-09-14 16:59:42 -0700 [chromium] lots of layout test failures on 10.7.4 2012-10-02 18:37:06 -0700 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) Unspecified Unspecified RESOLVED WONTFIX P2 Normal --- 1 dpranke webkit-unassigned chase dpranke nsylvain ojan rsesek thakis tony zmo oldest_to_newest 721254 0 dpranke 2012-09-14 16:59:42 -0700 it appears that things changed in 10.7.4 such that tests that run fine on 10.7.3 produce different baselines on 10.7.4. More details to follow. 722290 1 dpranke 2012-09-17 14:48:53 -0700 fwiw, I have a local platform/chromium-mac-10_7_4 directory with ~160ish layout test results that I think account for most if not all of the failures. I'm not posting this to the bug for now since I haven't really audited all of them, and it's unclear quite yet what to do with this. Per zmo, it seems that we don't necessarily want to just upgrade the bots to 10.7.4 because that'll cause a lot of gpu tests to fail. On the other hand, 10.7.4 is what our customers are running, so arguably we should just suck it up. We can: 1) create a 10.7.4 platform directory and keep it up-to-date by hand 2) spin up a 10.7.4 bot 3) wait for 10.7.5 and hope that fixes the gpu issues 4) suppress the failures as IMAGE (and IMAGE+TEXT etc) so that we don't need to track the baselines 5) ignore the problem and hope it goes away :) Let me know if anyone has any strong leanings here. 722329 2 ojan 2012-09-17 15:42:41 -0700 My vote would be for 1+2. But I'd be fine with any combination of 1, 3, 4 or 5. Seems like we should at least do one of 1 or 4 though. 4 seems like the least amount of effort in the short-term. 722332 3 dpranke 2012-09-17 15:45:26 -0700 I'll run locally w/ my 10.7.4 baselines for a while and see how much churn there is. Maybe updating by hand won't be too bad ... I'd prefer to avoid 2) if we can, and I'm not a bug fan of 4) since I'm trying to move away from suppressions (and 160+ tests is a lot to suppress if we have no reason to expect this to get fixed soon). 722964 4 tony 2012-09-18 12:04:48 -0700 How will the gpu tests fail? Are these real failures that our users are seeing or is it a bug in our test infrastructure? 722974 5 dpranke 2012-09-18 12:22:36 -0700 zmo can confirm but I believe they reflect real bugs in 10.7.4 that our users see. 722980 6 tony 2012-09-18 12:27:32 -0700 (In reply to comment #5) > zmo can confirm but I believe they reflect real bugs in 10.7.4 that our users see. In that case, I would just update the bots to 10.7.4 and either rebaseline or mark them in TestExpectations. The failures are real, so it seems fine for us to mark them as such in TestExpectations. I'm not sure what benefit we get by running the tests on 10.7.3 if that's not what our users are running. 723081 7 dpranke 2012-09-18 15:11:23 -0700 So, after running with a set of 10.7.4-specific baselines for a while, I'm not seeing any real gpu-related weirdness during the layout tests; they seem fairly stable. Given that, and a general consensus that we should probably be testing what our customers are likely running, it seems like we should update the lion layout test bots to 10.7.4 and update the baselines. I am not proposing updating the other chromium bots; I'll let zhenyao worry about those. Anyone disagree? 733305 8 dpranke 2012-10-02 18:37:06 -0700 We've upgraded the canaries to 10.7.5 (and rebaselined), so I'm closing this as WONTFIX ...