Bug 64002

Summary: [Qt] Some tests are flaky with NRWT
Product: WebKit Reporter: Andras Becsi <abecsi>
Component: Tools / TestsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED FIXED    
Severity: Critical CC: hnandor, jbadics, kbalazs, kkristof, ossy, zherczeg, zimmermann
Priority: P1 Keywords: Qt, QtTriaged
Version: 528+ (Nightly build)   
Hardware: All   
OS: Linux   
Bug Depends on: 71693, 71695    
Bug Blocks:    
Attachments:
Description Flags
.
none
I have skipped 2 tests to solve this.
ossy: review-, ossy: commit-queue-
I have skipped 2 tests to solve this.
ossy: review-, ossy: commit-queue-
I have skipped 2 tests to solve this. none

Description Andras Becsi 2011-07-06 09:35:21 PDT
Metabug to track the following flaky tests:

http/tests/misc/object-embedding-svg-delayed-size-negotiation-2.htm = TEXT PASS
fast/forms/textfield-overflow-by-value-update.html = PASS FAIL
fast/notifications/notifications-click-event.html = TEXT PASS
fast/workers/shared-worker-lifecycle.html = TEXT PASS
fast/workers/shared-worker-frame-lifecycle.html = TEXT PASS
fast/workers/worker-close-more.html = TEXT PASS
fast/workers/storage/interrupt-database.html = TEXT PASS
fast/workers/dedicated-worker-lifecycle.html = TEXT PASS
fast/workers/worker-lifecycle.html = TEXT PASS
fast/selectors/unqualified-hover-strict.html = TEXT PASS
Comment 1 Csaba Osztrogonác 2011-08-12 05:08:08 PDT
Created attachment 103762 [details]
.

You can reproduce these fails with ORWT if you run tests in same order 
(first the http tests)and without restarting DumpRenderTree.

I managed to reproduce the fails on r92952 with the following command:
$ Tools/Scripts/old-run-webkit-tests --nthly=50000 --wait-for-httpd `cat tests_run0.txt`

I attached the tests_run0.txt generated by NRWT.
Comment 2 Csaba Osztrogonác 2011-08-12 05:09:35 PDT
It means it isn't an NRWT bug, but a crazy DRT sideeffect bug again ...
Comment 3 Nandor Huszka 2011-11-07 06:31:44 PST
Created attachment 113856 [details]
I have skipped 2 tests to solve this.
Comment 4 Csaba Osztrogonác 2011-11-07 07:52:48 PST
Comment on attachment 113856 [details]
I have skipped 2 tests to solve this.

Great digging, r=me.
Comment 5 Csaba Osztrogonác 2011-11-07 07:54:01 PST
Comment on attachment 113856 [details]
I have skipped 2 tests to solve this.

Ooops, r- because of bad comment. It is absolutely unrelated to r99402.
Comment 6 Csaba Osztrogonác 2011-11-07 08:41:37 PST
Please reference to the new bugs in skipped list and changelog:
- https://bugs.webkit.org/show_bug.cgi?id=71693
- https://bugs.webkit.org/show_bug.cgi?id=71695
Comment 7 Nandor Huszka 2011-11-07 23:47:53 PST
Created attachment 114002 [details]
I have skipped 2 tests to solve this.
Comment 8 Nandor Huszka 2011-11-07 23:48:07 PST
Thanks for the review, I corrected it.
Comment 9 Csaba Osztrogonác 2011-11-08 00:26:43 PST
Comment on attachment 114002 [details]
I have skipped 2 tests to solve this.

View in context: https://bugs.webkit.org/attachment.cgi?id=114002&action=review

It is better, but we need one more round.

> LayoutTests/ChangeLog:17
> +        * platform/qt/Skipped:
> +        http/tests/notifications/icon-does-not-exist.html
> +        http/tests/xmlhttprequest/workers/abort-exception-assert.html
> +
> +        * platform/qt/test_expectations.txt:

* platform/qt/Skipped: Skip culprit tests
 - http/tests/notifications/icon-does-not-exist.html
 - http/tests/xmlhttprequest/workers/abort-exception-assert.html

* platform/qt/test_expectations.txt: Unskip passing tests.

> LayoutTests/platform/qt/Skipped:2544
> +
> +# [Qt] These made 7 tests timeout
> +# https://bugs.webkit.org/show_bug.cgi?id=64002
> +# https://bugs.webkit.org/show_bug.cgi?id=71693
> +# https://bugs.webkit.org/show_bug.cgi?id=71695
> +http/tests/notifications/icon-does-not-exist.html
> +http/tests/xmlhttprequest/workers/abort-exception-assert.html
> +

I think these entries would be better:

# [Qt] http/tests/notifications/icon-does-not-exist.html makes fast/notifications/notifications-click-event.html fail
# https://bugs.webkit.org/show_bug.cgi?id=71693
http/tests/notifications/icon-does-not-exist.html

# [Qt] http/tests/xmlhttprequest/workers/abort-exception-assert.html makes 6 fast/workers test fail
# https://bugs.webkit.org/show_bug.cgi?id=71695
http/tests/xmlhttprequest/workers/abort-exception-assert.html
Comment 10 Nandor Huszka 2011-11-08 00:41:41 PST
Created attachment 114004 [details]
I have skipped 2 tests to solve this.

Thank you for the help!
Comment 11 Csaba Osztrogonác 2011-11-08 00:52:40 PST
Comment on attachment 114004 [details]
I have skipped 2 tests to solve this.

LGTM, r=me
Comment 12 Csaba Osztrogonác 2011-11-09 08:55:40 PST
Comment on attachment 114004 [details]
I have skipped 2 tests to solve this.

Landed in http://trac.webkit.org/changeset/99536
Comment 13 Nikolas Zimmermann 2012-01-26 06:52:41 PST
This http/tests/misc/object-embedding-svg-delayed-size-negotiation-2.htm could be fixed once 77099 lands - can anyone check that afterwards?
Comment 14 Nikolas Zimmermann 2012-01-26 07:06:16 PST
I landed the (In reply to comment #13)
> This http/tests/misc/object-embedding-svg-delayed-size-negotiation-2.htm could be fixed once 77099 lands - can anyone check that afterwards?
Landed in r106001 - awaiting your findings.
Comment 15 Csaba Osztrogonác 2012-02-02 10:15:23 PST
I think flakey tests aren't related to NRWT, but the order of test running.
All of them should have own bug reports, so I think we can close this one.