Bug 89367
| Summary: | [chromium] Consistently failing layout tests | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Tony Payne <tpayne> |
| Component: | Tools / Tests | Assignee: | Nobody <webkit-unassigned> |
| Status: | RESOLVED WONTFIX | ||
| Severity: | Normal | CC: | abarth |
| Priority: | P2 | ||
| Version: | 528+ (Nightly build) | ||
| Hardware: | Other | ||
| OS: | Linux | ||
Tony Payne
The following tests consistently fail on cr-linux, slowing down the EWS queue:
platform/chromium-linux/fast/text/international/complex-joining-using-gpos.html
fast/text/international/thai-line-breaks.html
| Attachments | ||
|---|---|---|
| Add attachment proposed patch, testcase, etc. |
Adam Barth
This is because I don't have the fonts configured correctly on those machines.
Tony Payne
https://bugs.webkit.org/show_bug.cgi?id=89368
Tony Payne
(In reply to comment #1)
> This is because I don't have the fonts configured correctly on those machines.
If we can't mark the tests as failing, how about fixing the configuration? As mentioned in the patch bug, my experience is that the layout tests always run 3 times for cr-linux, which adds around 20 minutes per run and causes the queue to stay backlogged.
Adam Barth
Here are the most recent jobs from the ec2-cr-linux-01:
Patch 148373 from bug 69360:
1 minute ago (ec2-cr-linux-01) Pass
1 minute ago (ec2-cr-linux-01) Patch does not pass tests [results]
Patch 148370 from bug 89486:
11 minutes ago (ec2-cr-linux-01) Pass
11 minutes ago (ec2-cr-linux-01) Patch does not pass tests [results]
Patch 148378 from bug 89491:
20 minutes ago (ec2-cr-linux-01) Pass
20 minutes ago (ec2-cr-linux-01) Patch does not pass tests [results]
Patch 148375 from bug 89039:
53 minutes ago (ec2-cr-linux-01) Pass
53 minutes ago (ec2-cr-linux-01) Patch does not pass tests [results]
http://queues.webkit.org/queue-status/chromium-ews/bots/ec2-cr-linux-01
In each case, the bot only ran the tests once. Sometimes it runs the tests three times, for example when there's a flaky test or when the patch actually introduces a failure. However, those cases won't be improved by marking these tests as failing.
In general, I agree with you that we should fix the configuration. There's some trickiness last time I investigated the issue because the kernel for "canonical" version of linux we use for tests causes the EC2 disks to lock up. It's likely that situation has improved, but I haven't had the time to investiate recently.
Tony Payne
This is typical of what I see: http://queues.webkit.org/patch/148458
Three runs for the CQ and three runs for cr-linux
Adam Barth
The commit-queue ran the tests twice because the results were flaky. It got different answers with the same binary. In this case because of perf/object-keys.html
The EWS did do the full three test run. I'm not sure why. It could be because the patch before failed and blew out the cache of which tests fail. You're right that there are likely ways we can improve the efficiency of the bots, including by fixing the configuration.
Stephen Chenney
Marked LayoutTest bugs, bugs with Chromium IDs, and some others as WontFix. Test failure bugs still are trackable via TestExpectations or disabled unit tests.