Bug 215886 – REGRESSION(r266221) Layout tests step hanging due to slow WaveDiff comparison

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Lauro Moura]

GTK/WPE layout-tests steps are timing out after r266221. Some tests seem to hold the driver alive, and the step fails with those "no console output for 1200 seconds".

For example, running webaudio/ tests (defaults to 5 shards), webaudio/audiobuffersource-loop-points.html (skipped in mac in bug119467) get stuck and the test run does not progress.

The same happens regardless of the number of shards, always getting stuck in this test.

List of webaudio/ tests skipped to allow all other webaudio/ tests to run:

* webaudio/audiobuffersource-loop-points.html
* webaudio/audiobuffersource-playbackrate.html

PS: Not sure if this happens due to some webkitpy bug uncovered by these timeouts.

[Chris Dumez]

Interesting. I had some very slow layout tests runs locally as well until I rebaselined the couple of Web Audio tests that were failing.

You should consider rebaselining the couple of tests that are failing with audio diff to see if it makes the issue go away (Seems like it did on macOS).

[Lauro Moura]

(In reply to Chris Dumez from comment #1)
> Interesting. I had some very slow layout tests runs locally as well until I
> rebaselined the couple of Web Audio tests that were failing.
> 
> You should consider rebaselining the couple of tests that are failing with
> audio diff to see if it makes the issue go away (Seems like it did on macOS).

Indeed rebaselining made the test pass. O_o

In any case, the test suite should not hang because a WAV diff failed, no?

[Chris Dumez]

(In reply to Lauro Moura from comment #2)
> (In reply to Chris Dumez from comment #1)
> > Interesting. I had some very slow layout tests runs locally as well until I
> > rebaselined the couple of Web Audio tests that were failing.
> > 
> > You should consider rebaselining the couple of tests that are failing with
> > audio diff to see if it makes the issue go away (Seems like it did on macOS).
> 
> Indeed rebaselining made the test pass. O_o
> 
> In any case, the test suite should not hang because a WAV diff failed, no?

Yes, I suspect we have a tooling bug. I saw a Python process using 100% CPU constantly.

I was just trying to help you get back in a good state.

[Lauro Moura]

(In reply to Chris Dumez from comment #3)
> 
> I was just trying to help you get back in a good state.

Sure, sure. Thanks.

In the loop-points test, the new test results had small deviations that increased as the frequency increased (looping was right, though).

This caused the WaveDiff class to generate a sample diff for 340k of the 666k samples.

And as the current WaveDiff uses string concat for each sample diff, it ended up being too slow, likely leading the timeout to be triggered on the test controller and it somehow losing track of the runner.

[Lauro Moura]

Created attachment 407429 [details]
Patch

[EWS]

Committed r266272: <https://trac.webkit.org/changeset/266272>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 407429 [details].

[Radar WebKit Bug Importer]

<rdar://problem/67915158>