Bug 188640 - [win-ews] No such file or directory: u'/cygdrive/C/cygwin/home/buildbot/WebKit/WebKitBuild/Release/bin32/layout-test-results/pywebsocket.ws.log-out.txt'
Summary: [win-ews] No such file or directory: u'/cygdrive/C/cygwin/home/buildbot/WebKi...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: Other
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Fujii Hironori
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2018-08-15 22:57 PDT by Ryan Haddad
Modified: 2019-10-01 20:45 PDT (History)
11 users (show)

See Also:


Attachments
Patch (1.50 KB, patch)
2019-06-03 13:19 PDT, Alexey Shvayka
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews107 for mac-highsierra-wk2 (2.83 MB, application/zip)
2019-06-03 14:33 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews211 for win-future (14.02 MB, application/zip)
2019-06-03 17:11 PDT, Build Bot
no flags Details
Patch (5.19 KB, patch)
2019-06-04 17:19 PDT, Alexey Shvayka
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews215 for win-future (13.47 MB, application/zip)
2019-06-04 18:44 PDT, Build Bot
no flags Details
Patch (7.49 KB, patch)
2019-06-05 04:32 PDT, Alexey Shvayka
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews211 for win-future (14.09 MB, application/zip)
2019-06-05 08:32 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews212 for win-future (13.73 MB, application/zip)
2019-06-05 08:56 PDT, Build Bot
no flags Details
Patch (7.60 KB, patch)
2019-06-05 15:52 PDT, Alexey Shvayka
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews105 for mac-highsierra-wk2 (2.86 MB, application/zip)
2019-06-05 17:07 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews212 for win-future (13.70 MB, application/zip)
2019-06-05 21:30 PDT, Build Bot
no flags Details
Patch (7.61 KB, patch)
2019-06-06 09:49 PDT, Alexey Shvayka
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews107 for mac-highsierra-wk2 (3.24 MB, application/zip)
2019-06-06 10:41 PDT, Build Bot
no flags Details
Patch (8.32 KB, patch)
2019-10-01 02:32 PDT, Fujii Hironori
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ryan Haddad 2018-08-15 22:57:06 PDT
Windows EWS bots are failing with the following:

Failed to remove stale pywebsocket log files: [Errno 2] No such file or directory: '/cygdrive/C/cygwin/home/buildbot/WebKit/WebKitBuild/Release/bin32/layout-test-results'

IOError raised: [Errno 2] No such file or directory: u'/cygdrive/C/cygwin/home/buildbot/WebKit/WebKitBuild/Release/bin32/layout-test-results/pywebsocket.ws.log-out.txt'
Traceback (most recent call last):
  File "/home/buildbot/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 85, in main
    run_details = run(port, options, args, stderr)
  File "/home/buildbot/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 447, in run
    run_details = manager.run(args)
  File "/home/buildbot/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 240, in run
    needs_http=needs_http, needs_web_platform_test_server=needs_web_platform_test_server, needs_websockets=needs_websockets)
  File "/home/buildbot/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 87, in __init__
    self.start_servers()
  File "/home/buildbot/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 202, in start_servers
    self._port.start_websocket_server()
  File "/home/buildbot/WebKit/Tools/Scripts/webkitpy/port/base.py", line 1029, in start_websocket_server
    server.start()
  File "/home/buildbot/WebKit/Tools/Scripts/webkitpy/layout_tests/servers/http_server_base.py", line 95, in start
    self._prepare_config()
  File "/home/buildbot/WebKit/Tools/Scripts/webkitpy/layout_tests/servers/websocket_server.py", line 118, in _prepare_config
    self._wsout = self._filesystem.open_text_file_for_writing(output_log)
  File "/home/buildbot/WebKit/Tools/Scripts/webkitpy/common/system/filesystem.py", line 239, in open_text_file_for_writing
    return codecs.open(path, 'w', 'utf8')
  File "/usr/lib/python2.7/codecs.py", line 896, in open
    file = __builtin__.open(filename, mode, buffering)
IOError: [Errno 2] No such file or directory: u'/cygdrive/C/cygwin/home/buildbot/WebKit/WebKitBuild/Release/bin32/layout-test-results/pywebsocket.ws.log-out.txt'

https://webkit-queues.webkit.org/results/8876187
Comment 1 Ryan Haddad 2018-08-15 22:57:22 PDT
This persists after a reboot.
Comment 2 Radar WebKit Bug Importer 2018-08-16 08:58:03 PDT
<rdar://problem/43383399>
Comment 3 Per Arne Vollan 2018-08-16 10:26:07 PDT
Temporarily skip websocket tests in <https://trac.webkit.org/changeset/234933/webkit>.
Comment 4 Alexey Shvayka 2019-06-03 13:19:28 PDT
Created attachment 371206 [details]
Patch
Comment 5 Alexey Shvayka 2019-06-03 13:31:29 PDT
I've got the same error when launched `run-webkit-tests` in freshly cloned repository.
Pretty sure that busy port is not the cause of this particular problem.
I am running macOS 10.14.4, Xcode 10.2.1, webkit-gtk 2.4.11_2, cmake 3.14.4.

=== STACK TRACE ===

Failed to remove stale pywebsocket log files: [Errno 2] No such file or directory: '/Users/admin/WebKit/WebKitBuild/Release/layout-test-results'

IOError raised: [Errno 2] No such file or directory: u'/Users/admin/WebKit/WebKitBuild/Release/layout-test-results/pywebsocket.ws.log-out.txt'
Traceback (most recent call last):
  File "/Users/admin/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 91, in main
    run_details = run(port, options, args, stderr)
  File "/Users/admin/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 452, in run
    run_details = manager.run(args)
  File "/Users/admin/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 228, in run
    needs_http=needs_http, needs_web_platform_test_server=needs_web_platform_test_server, needs_websockets=needs_websockets)
  File "/Users/admin/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 87, in __init__
    self.start_servers()
  File "/Users/admin/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py", line 219, in start_servers
    self._port.start_websocket_server()
  File "/Users/admin/WebKit/Tools/Scripts/webkitpy/port/base.py", line 1044, in start_websocket_server
    server.start()
  File "/Users/admin/WebKit/Tools/Scripts/webkitpy/layout_tests/servers/http_server_base.py", line 95, in start
    self._prepare_config()
  File "/Users/admin/WebKit/Tools/Scripts/webkitpy/layout_tests/servers/websocket_server.py", line 118, in _prepare_config
    self._wsout = self._filesystem.open_text_file_for_writing(output_log)
  File "/Users/admin/WebKit/Tools/Scripts/webkitpy/common/system/filesystem.py", line 239, in open_text_file_for_writing
    return codecs.open(path, 'w', 'utf8')
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/codecs.py", line 884, in open
    file = __builtin__.open(filename, mode, buffering)
IOError: [Errno 2] No such file or directory: u'/Users/admin/WebKit/WebKitBuild/Release/layout-test-results/pywebsocket.ws.log-out.txt'
Comment 6 Build Bot 2019-06-03 14:33:54 PDT
Comment on attachment 371206 [details]
Patch

Attachment 371206 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/12365879

New failing tests:
http/wpt/service-workers/service-worker-networkprocess-crash.html
Comment 7 Build Bot 2019-06-03 14:33:55 PDT
Created attachment 371213 [details]
Archive of layout-test-results from ews107 for mac-highsierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews107  Port: mac-highsierra-wk2  Platform: Mac OS X 10.13.6
Comment 8 Build Bot 2019-06-03 17:11:24 PDT
Comment on attachment 371206 [details]
Patch

Attachment 371206 [details] did not pass win-ews (win):
Output: https://webkit-queues.webkit.org/results/12366770

New failing tests:
fast/shadow-dom/svg-use-href-change-in-shadow-tree.html
imported/blink/fast/canvas/bug382588.html
Comment 9 Build Bot 2019-06-03 17:11:26 PDT
Created attachment 371232 [details]
Archive of layout-test-results from ews211 for win-future

The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews211  Port: win-future  Platform: CYGWIN_NT-10.0-17763-3.0.5-338.x86_64-x86_64-64bit
Comment 10 Alex Christensen 2019-06-04 16:42:07 PDT
If you can reproduce the issue and this fixes it, could you also revert https://trac.webkit.org/changeset/234933/webkit in the same patch?
Comment 11 Alexey Shvayka 2019-06-04 17:19:21 PDT
Created attachment 371355 [details]
Patch
Comment 12 Build Bot 2019-06-04 18:44:38 PDT
Comment on attachment 371355 [details]
Patch

Attachment 371355 [details] did not pass win-ews (win):
Output: https://webkit-queues.webkit.org/results/12379424

New failing tests:
http/tests/websocket/tests/hybi/deflate-frame-parameter.html
Comment 13 Build Bot 2019-06-04 18:44:40 PDT
Created attachment 371360 [details]
Archive of layout-test-results from ews215 for win-future

The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews215  Port: win-future  Platform: CYGWIN_NT-10.0-17763-3.0.5-338.x86_64-x86_64-64bit
Comment 14 Alexey Shvayka 2019-06-05 04:32:42 PDT
Created attachment 371386 [details]
Patch
Comment 15 Alexey Shvayka 2019-06-05 04:43:48 PDT
(In reply to Alex Christensen from comment #10)
> If you can reproduce the issue and this fixes it, could you also revert
> https://trac.webkit.org/changeset/234933/webkit in the same patch?

The issue is easy to reproduce with:
  rm -rf WebKitBuild
  Tools/Scripts/build-webkit
  Tools/Scripts/run-webkit-tests --no-build

I've also managed to reproduce "port 8880 is already in use" problem with:
  Tools/Scripts/run-webkit-websocketserver
  ENTER

The latest patch fixes both issues and includes revert of changeset 234933.
CI test fails seem irrelevant.
Comment 16 Build Bot 2019-06-05 08:32:17 PDT
Comment on attachment 371386 [details]
Patch

Attachment 371386 [details] did not pass win-ews (win):
Output: https://webkit-queues.webkit.org/results/12384132

New failing tests:
imported/blink/fast/canvas/canvas-clip-stack-persistence.html
http/tests/websocket/tests/hybi/deflate-frame-parameter.html
imported/blink/fast/canvas/canvas-state-persistence-no-dirty.html
Comment 17 Build Bot 2019-06-05 08:32:20 PDT
Created attachment 371399 [details]
Archive of layout-test-results from ews211 for win-future

The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews211  Port: win-future  Platform: CYGWIN_NT-10.0-17763-3.0.5-338.x86_64-x86_64-64bit
Comment 18 Build Bot 2019-06-05 08:56:10 PDT
Comment on attachment 371386 [details]
Patch

Attachment 371386 [details] did not pass win-ews (win):
Output: https://webkit-queues.webkit.org/results/12384637

New failing tests:
http/tests/websocket/tests/hybi/deflate-frame-parameter.html
Comment 19 Build Bot 2019-06-05 08:56:13 PDT
Created attachment 371402 [details]
Archive of layout-test-results from ews212 for win-future

The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews212  Port: win-future  Platform: CYGWIN_NT-10.0-17763-3.0.5-338.x86_64-x86_64-64bit
Comment 20 Alex Christensen 2019-06-05 10:37:50 PDT
(In reply to Build Bot from comment #18)
> New failing tests:
> http/tests/websocket/tests/hybi/deflate-frame-parameter.html

Yep, this is definitely related.
Comment 21 Alexey Shvayka 2019-06-05 15:52:53 PDT
Created attachment 371443 [details]
Patch
Comment 22 Alex Christensen 2019-06-05 15:58:38 PDT
Comment on attachment 371443 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=371443&action=review

r=me if this passes EWS.

> Tools/ChangeLog:8
> +        1. Properly shutdown WebSocket server so that port 8880 is released.

I don't see this change.
Comment 23 Alexey Shvayka 2019-06-05 16:23:31 PDT
(In reply to Alex Christensen from comment #22)
> I don't see this change.

This change is in `new-run-webkit-websocketserver` script. Does it work for you?

I've updated the test expectations.
In comparison with revision before 234933, 3 more tests now pass expectedly:
  http/tests/websocket/tests/hybi/simple-wss.html
  http/tests/websocket/tests/hybi/websocket-allowed-setting-cookie-as-third-party.html (webkit.org/b/187762)
  http/tests/websocket/tests/hybi/websocket-cookie-overwrite-behavior.html (webkit.org/b/187762)

While 1 new test is failing:
  http/tests/websocket/tests/hybi/deflate-frame-parameter.html
Comment 24 Alex Christensen 2019-06-05 16:29:43 PDT
ok
Comment 25 Build Bot 2019-06-05 17:07:15 PDT
Comment on attachment 371443 [details]
Patch

Attachment 371443 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/12389422

New failing tests:
http/wpt/service-workers/service-worker-networkprocess-crash.html
Comment 26 Build Bot 2019-06-05 17:07:17 PDT
Created attachment 371453 [details]
Archive of layout-test-results from ews105 for mac-highsierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews105  Port: mac-highsierra-wk2  Platform: Mac OS X 10.13.6
Comment 27 Build Bot 2019-06-05 21:30:34 PDT
Comment on attachment 371443 [details]
Patch

Attachment 371443 [details] did not pass win-ews (win):
Output: https://webkit-queues.webkit.org/results/12391593

New failing tests:
http/tests/websocket/tests/hybi/websocket-cookie-overwrite-behavior.html
Comment 28 Build Bot 2019-06-05 21:30:36 PDT
Created attachment 371470 [details]
Archive of layout-test-results from ews212 for win-future

The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews212  Port: win-future  Platform: CYGWIN_NT-10.0-17763-3.0.5-338.x86_64-x86_64-64bit
Comment 29 Alexey Shvayka 2019-06-06 09:49:13 PDT
Created attachment 371505 [details]
Patch
Comment 30 Build Bot 2019-06-06 10:41:51 PDT
Comment on attachment 371505 [details]
Patch

Attachment 371505 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/12396704

New failing tests:
http/wpt/service-workers/service-worker-networkprocess-crash.html
Comment 31 Build Bot 2019-06-06 10:41:53 PDT
Created attachment 371508 [details]
Archive of layout-test-results from ews107 for mac-highsierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews107  Port: mac-highsierra-wk2  Platform: Mac OS X 10.13.6
Comment 32 Alexey Shvayka 2019-06-12 13:20:16 PDT
(In reply to Alexey Shvayka from comment #29)
> Created attachment 371505 [details]
> Patch

I've tweaked the expectations: http/tests/websocket/tests/hybi/websocket-cookie-overwrite-behavior.html test was failing on ews212, while passing on ews211.
Comment 33 Fujii Hironori 2019-10-01 01:28:30 PDT
I'm also suffering from this issue since today.
Here is my error message.

PS C:\webkit\gb> python ./Tools/Scripts/run-webkit-tests --wincairo --release  --no-new-test-results -1
Using port 'wincairo-win10'
Test configuration: <win10, x86_64, release>
Placing test results in C:\webkit\gb\WebKitBuild\Release\bin64\layout-test-results
Using Release build
Pixel tests disabled
Regular timeout: 30000, slow test timeout: 150000
Command line: C:\webkit\gb\WebKitBuild\Release\bin64\DumpRenderTree.exe -

Found 57947 tests; running 16525, skipping 41422.
Failed to remove stale pywebsocket log files: [Error 3] The system cannot find the path specified: u'C:\\webkit\\gb\\WebKitBuild\\Release\\bin64\\layout-test-results\\*.*'

IOError raised: [Errno 2] No such file or directory: u'C:\\webkit\\gb\\WebKitBuild\\Release\\bin64\\layout-test-results\\pywebsocket.ws.log-out.txt'
Traceback (most recent call last):
  File "C:\webkit\gb\Tools\Scripts\webkitpy\layout_tests\run_webkit_tests.py", line 91, in main
    run_details = run(port, options, args, stderr)
  File "C:\webkit\gb\Tools\Scripts\webkitpy\layout_tests\run_webkit_tests.py", line 452, in run
    run_details = manager.run(args)
  File "C:\webkit\gb\Tools\Scripts\webkitpy\layout_tests\controllers\manager.py", line 229, in run
    needs_http=needs_http, needs_web_platform_test_server=needs_web_platform_test_server, needs_websockets=needs_websockets)
  File "C:\webkit\gb\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 87, in __init__
    self.start_servers()
  File "C:\webkit\gb\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 219, in start_servers
    self._port.start_websocket_server()
  File "C:\webkit\gb\Tools\Scripts\webkitpy\port\base.py", line 1037, in start_websocket_server
    server.start()
  File "C:\webkit\gb\Tools\Scripts\webkitpy\layout_tests\servers\http_server_base.py", line 95, in start
    self._prepare_config()
  File "C:\webkit\gb\Tools\Scripts\webkitpy\layout_tests\servers\websocket_server.py", line 118, in _prepare_config
    self._wsout = self._filesystem.open_text_file_for_writing(output_log)
  File "C:\webkit\gb\Tools\Scripts\webkitpy\common\system\filesystem.py", line 240, in open_text_file_for_writing
    return codecs.open(path, 'w', 'utf8')
  File "C:\Python27\lib\codecs.py", line 898, in open
    file = __builtin__.open(filename, mode, buffering)
IOError: [Errno 2] No such file or directory: u'C:\\webkit\\gb\\WebKitBuild\\Release\\bin64\\layout-test-results\\pywebsocket.ws.log-out.txt'
Comment 34 Fujii Hironori 2019-10-01 02:09:18 PDT
In my case, the following conditions causes this issue:
1. check_httpd is failing because Apache service is already started
2. layout-test-results directory is not created yet
Comment 35 Fujii Hironori 2019-10-01 02:23:06 PDT
Comment on attachment 371505 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=371505&action=review

> Tools/ChangeLog:12
> +        (main): Tweak import and use class constants of PyWebSocket directly.

Why is this change needed?
Comment 36 Fujii Hironori 2019-10-01 02:32:01 PDT
Created attachment 379883 [details]
Patch

Rebased onto ToT. Submitted for EWS.
Comment 37 Alexey Shvayka 2019-10-01 12:04:48 PDT
(In reply to Fujii Hironori from comment #35)
> > Tools/ChangeLog:12
> > +        (main): Tweak import and use class constants of PyWebSocket directly.
> 
> Why is this change needed?

This change is not necessary to fix the issue with missing directory, but it solves a couple of FIXMEs.
Comment 38 WebKit Commit Bot 2019-10-01 20:45:46 PDT
Comment on attachment 379883 [details]
Patch

Clearing flags on attachment: 379883

Committed r250593: <https://trac.webkit.org/changeset/250593>
Comment 39 WebKit Commit Bot 2019-10-01 20:45:48 PDT
All reviewed patches have been landed.  Closing bug.