Bug 188795 – REGRESSION(r234993): Test results page ignores TestExpectations, shows 1800 failures

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Michael Catanzaro]

Since r234993 "Modernize results.html" our test results page has gone crazy. For example:

https://build.webkit.org/results/GTK Linux 64-bit Release (Tests)/r235110 (7847)/results.html

shows 1815 tests that failed pixel/audio diff. But the bot dashboard reports only 13 layout test failures. And looking at the text file, there were indeed only 13 failures:

https://build.webkit.org/builders/GTK Linux 64-bit Release (Tests)/builds/7847/steps/layout-test/logs/stdio

The problem seems to be that the platform TestExpectations file is now ignored, so all the expected failures are showing up as unexpected failures.

It seems this issue only affects GTK: the results for macOS, iOS, Windows, and WPE are all fine.

[Michael Catanzaro]

(In reply to Michael Catanzaro from comment #0)
> Since r234993 "Modernize results.html" our test results page has gone crazy.
> For example:
> 
> https://build.webkit.org/results/GTK Linux 64-bit Release (Tests)/r235110
> (7847)/results.html

Unrelated: it looks like Epiphany's code to percent-encode URLs when copied from the address bar is broken. :)

[Simon Fraser (smfr)]

https://build.webkit.org/results/GTK%20Linux%2064-bit%20Release%20(Tests)/r235110%20(7847)/results.html

[Simon Fraser (smfr)]

It's showing expected failures; I'll fix.

[Simon Fraser (smfr)]

Created attachment 347643 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 347643 [details]
Patch

Clearing flags on attachment: 347643

Committed r235126: <https://trac.webkit.org/changeset/235126>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/43568697>

[Michael Catanzaro]

Thanks Simon!