It seems that ImageDiff can get misaligned and timeout after the Python 3 transition.
Created attachment 387566 [details] Patch
The patch appears incomplete. But also, how can this be caused by Python3, does this imply a bug elsewhere.
(In reply to Alexey Proskuryakov from comment #2) > The patch appears incomplete. But also, how can this be caused by Python3, > does this imply a bug elsewhere. The uploaded patch was actually intended to test EWS. The bug is real, though. I think this could be caused by a bug elsewhere (especially since it flakily reproduces), but I can bisect this to the Python 3 compatibility change, and I've made some significant progress towards narrowing down what part of that change actually causes the problem.
It is also possible that the flakiness is caused by actually getting different results (e.g. there is a preceding test that changes color settings permanently, affecting subsequent tests).
(In reply to Alexey Proskuryakov from comment #4) > It is also possible that the flakiness is caused by actually getting > different results (e.g. there is a preceding test that changes color > settings permanently, affecting subsequent tests). The root cause seems to be comparing images with different sizes, though. I've narrowed it down to a few lines of code that have changed in image_diff.py, so I'll have a fix shortly.
if all of these are different sizes, we should also file a bug to get the tests fixed, it's not really OK for them to rely on automatic resizing.
(In reply to Alexey Proskuryakov from comment #6) > if all of these are different sizes, we should also file a bug to get the > tests fixed, it's not really OK for them to rely on automatic resizing. If it is the fault of the tests, then yes, we should/ But I'm not certain that's the root of the problem. I think the root of the problem might actually be ImageDiff getting tripped up when we feed it one image at a time instead of both images at the same time, ImageDiff is reporting one of the images to be exactly twice the size of the other.
Created attachment 387686 [details] Patch
<rdar://problem/58578775>
Committed r254532: <https://trac.webkit.org/changeset/254532>
*** Bug 206179 has been marked as a duplicate of this bug. ***
> If it is the fault of the tests, then yes, we should/ But I'm not certain > that's the root of the problem. I agree that this is not the root of the problem at hand, but the tests are broken too, I don't think that they need to do this, or are allowed to.