[Qt] Some tests are flaky with NRWT

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Andras Becsi]

Metabug to track the following flaky tests:

http/tests/misc/object-embedding-svg-delayed-size-negotiation-2.htm = TEXT PASS
fast/forms/textfield-overflow-by-value-update.html = PASS FAIL
fast/notifications/notifications-click-event.html = TEXT PASS
fast/workers/shared-worker-lifecycle.html = TEXT PASS
fast/workers/shared-worker-frame-lifecycle.html = TEXT PASS
fast/workers/worker-close-more.html = TEXT PASS
fast/workers/storage/interrupt-database.html = TEXT PASS
fast/workers/dedicated-worker-lifecycle.html = TEXT PASS
fast/workers/worker-lifecycle.html = TEXT PASS
fast/selectors/unqualified-hover-strict.html = TEXT PASS

[Csaba Osztrogonác]

Created attachment 103762 [details]
.

You can reproduce these fails with ORWT if you run tests in same order 
(first the http tests)and without restarting DumpRenderTree.

I managed to reproduce the fails on r92952 with the following command:
$ Tools/Scripts/old-run-webkit-tests --nthly=50000 --wait-for-httpd `cat tests_run0.txt`

I attached the tests_run0.txt generated by NRWT.

[Csaba Osztrogonác]

It means it isn't an NRWT bug, but a crazy DRT sideeffect bug again ...

[Nandor Huszka]

Created attachment 113856 [details]
I have skipped 2 tests to solve this.

[Csaba Osztrogonác]

Comment on attachment 113856 [details]
I have skipped 2 tests to solve this.

Great digging, r=me.

[Csaba Osztrogonác]

Comment on attachment 113856 [details]
I have skipped 2 tests to solve this.

Ooops, r- because of bad comment. It is absolutely unrelated to r99402.

[Csaba Osztrogonác]

Please reference to the new bugs in skipped list and changelog:
- https://bugs.webkit.org/show_bug.cgi?id=71693
- https://bugs.webkit.org/show_bug.cgi?id=71695

[Nandor Huszka]

Created attachment 114002 [details]
I have skipped 2 tests to solve this.

[Nandor Huszka]

Thanks for the review, I corrected it.

[Csaba Osztrogonác]

Comment on attachment 114002 [details]
I have skipped 2 tests to solve this.

View in context: https://bugs.webkit.org/attachment.cgi?id=114002&action=review

It is better, but we need one more round.

> LayoutTests/ChangeLog:17
> +        * platform/qt/Skipped:
> +        http/tests/notifications/icon-does-not-exist.html
> +        http/tests/xmlhttprequest/workers/abort-exception-assert.html
> +
> +        * platform/qt/test_expectations.txt:

* platform/qt/Skipped: Skip culprit tests
 - http/tests/notifications/icon-does-not-exist.html
 - http/tests/xmlhttprequest/workers/abort-exception-assert.html

* platform/qt/test_expectations.txt: Unskip passing tests.

> LayoutTests/platform/qt/Skipped:2544
> +
> +# [Qt] These made 7 tests timeout
> +# https://bugs.webkit.org/show_bug.cgi?id=64002
> +# https://bugs.webkit.org/show_bug.cgi?id=71693
> +# https://bugs.webkit.org/show_bug.cgi?id=71695
> +http/tests/notifications/icon-does-not-exist.html
> +http/tests/xmlhttprequest/workers/abort-exception-assert.html
> +

I think these entries would be better:

# [Qt] http/tests/notifications/icon-does-not-exist.html makes fast/notifications/notifications-click-event.html fail
# https://bugs.webkit.org/show_bug.cgi?id=71693
http/tests/notifications/icon-does-not-exist.html

# [Qt] http/tests/xmlhttprequest/workers/abort-exception-assert.html makes 6 fast/workers test fail
# https://bugs.webkit.org/show_bug.cgi?id=71695
http/tests/xmlhttprequest/workers/abort-exception-assert.html

[Nandor Huszka]

Created attachment 114004 [details]
I have skipped 2 tests to solve this.

Thank you for the help!

[Csaba Osztrogonác]

Comment on attachment 114004 [details]
I have skipped 2 tests to solve this.

LGTM, r=me

[Csaba Osztrogonác]

Comment on attachment 114004 [details]
I have skipped 2 tests to solve this.

Landed in http://trac.webkit.org/changeset/99536

[Nikolas Zimmermann]

This http/tests/misc/object-embedding-svg-delayed-size-negotiation-2.htm could be fixed once 77099 lands - can anyone check that afterwards?

[Nikolas Zimmermann]

I landed the (In reply to comment #13)
> This http/tests/misc/object-embedding-svg-delayed-size-negotiation-2.htm could be fixed once 77099 lands - can anyone check that afterwards?
Landed in r106001 - awaiting your findings.

[Csaba Osztrogonác]

I think flakey tests aren't related to NRWT, but the order of test running.
All of them should have own bug reports, so I think we can close this one.