Bug 66730 – WebSocket layout tests are flaky on Chromium Win

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Peter Kasting]

The following layout test is flaky on Chromium Win, for the release bot on the WebKit builder (see http://test-results.appspot.com/dashboards/flakiness_dashboard.html#group=%40ToT%20-%20webkit.org&showExpectations=true&tests=http%2Ftests%2Fwebsocket%2Ftests%2Fhixie76%2Fhandshake-fail-by-no-connection-header.html ) and the bots on the downstream builder (see http://test-results.appspot.com/dashboards/flakiness_dashboard.html#group=%40DEPS%20-%20chromium.org&showExpectations=true&tests=http%2Ftests%2Fwebsocket%2Ftests%2Fhixie76%2Fhandshake-fail-by-no-connection-header.html ).

http/tests/websocket/tests/hixie76/handshake-fail-by-no-connection-header.html

I don't know why sometimes the handshake doesn't fail; maybe one of you can get somewhere with this?

[Peter Kasting]

More flaky WebSocket tests on Chromium Win:

http/tests/websocket/tests/hybi/handshake-fail-by-no-upgrade-header.html
http/tests/websocket/tests/hybi/masked-frames.html

See e.g. http://test-results.appspot.com/dashboards/flakiness_dashboard.html#group=%40DEPS%20-%20chromium.org&tests=http%2Ftests%2Fwebsocket%2Ftests%2Fhybi%2Fhandshake-fail-by-no-upgrade-header.html%2Chttp%2Ftests%2Fwebsocket%2Ftests%2Fhybi%2Fmasked-frames.html&showExpectations=true

[Mihai Parparita]

http/tests/websocket/tests/hybi/close.html is flaky too:
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#showExpectations=true&tests=http%2Ftests%2Fwebsocket%2Ftests%2Fhybi%2Fclose.html

I'll add it to test_expectations.txt. See also 61767 for more flaky WebSocket tests.

[Takashi Toyoshima]

I found a bug on Chromium side.
I'm preparing a change for this issue.

http://code.google.com/p/chromium/issues/detail?id=98664

[Takashi Toyoshima]

I landed Chromium change to fix a socket termination problem on Windows.
I believe this change will fix many WebSocket close related flaky tests.

[Mihai Parparita]

Looking at http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=http%2Ftests%2Fwebsocket%2Ftests%2Fhybi%2Fclose.html%2Chttp%2Ftests%2Fwebsocket%2Ftests%2Fhybi%2Fhandshake-fail-by-no-upgrade-header.html%2Chttp%2Ftests%2Fwebsocket%2Ftests%2Fhybi%2Fmasked-frames.html%2Chttp%2Ftests%2Fwebsocket%2Ftests%2Fhixie76%2Fhandshake-fail-by-no-connection-header.html

I'm not seeing any failures since http://crrev.com/104852. If things stay green for another day, can you remove the failing expectations from test_expectations.txt?

[Yuta Kitamura]

*** Bug 61767 has been marked as a duplicate of this bug. ***

[Yuta Kitamura]

Created attachment 111576 [details]
Patch

[Yuta Kitamura]

(In reply to comment #7)
> Created an attachment (id=111576) [details]
> Patch

Here's flakiness dashboard link for tests whose test expectation is being removed:
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=http%2Ftests%2Fwebsocket%2Ftests%2Fhixie76%2Fclient-close.html%2Chttp%2Ftests%2Fwebsocket%2Ftests%2Fhybi%2Fclient-close.html%2Chttp%2Ftests%2Fwebsocket%2Ftests%2Fhixie76%2Fhandshake-fail-by-no-connection-header.html%2Chttp%2Ftests%2Fwebsocket%2Ftests%2Fhybi%2Fhandshake-fail-by-no-upgrade-header.html%2Chttp%2Ftests%2Fwebsocket%2Ftests%2Fhybi%2Fmasked-frames.html%2Chttp%2Ftests%2Fwebsocket%2Ftests%2Fhybi%2Fclose.html

These tests stay green these days.

[WebKit Review Bot]

Comment on attachment 111576 [details]
Patch

Clearing flags on attachment: 111576

Committed r97930: <http://trac.webkit.org/changeset/97930>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.