Bug 188092 – [LayoutTests][Win] Tests can fail if long paths are not enabled in registry.

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Basuke Suzuki]

Windows need a special care for path which is longer than 255 (OMG). Some tests are very long and causes error when generating a test result file which is usually longer than original.

i.e. 
C:\Users\basuke\webkit-trunk\LayoutTests\http\tests\storageAccess\equest-and-grant-storage-access-cross-origin-sandboxed-iframe-from-prevalent-domain-with-non-recent-user-interaction-and-try-access-from-right-frame.html
->
C:\Users\basuke\webkit-trunk\WebKitBuild\Release\bin64\layout-test-results\http\tests\storageAccess\request-and-grant-storage-access-cross-origin-sandboxed-iframe-from-prevalent-domain-with-non-recent-user-interaction-and-try-access-from-right-frame-actual.txt

link:
https://docs.microsoft.com/ja-jp/windows/desktop/FileIO/naming-a-file#maximum-path-length-limitation

[Basuke Suzuki]

worker/7: IOError('[Errno 2] No such file or directory: u'C:\\Users\\basuke\\webkit-trunk\\WebKitBuild\\Release\\bin64\\layout-test-results\\http\\tests\\storageAccess\\request-and-grant-storage-access-cross-origin-sandboxed-iframe-from-prevalent-domain-with-non-recent-user-interaction-and-try-access-from-right-frame-actual.txt'') raised:
    File "C:\Users\basuke\webkit-trunk\Tools\Scripts\webkitpy\common\message_pool.py", line 257, in run
      worker.handle(message.name, message.src, *message.args)
    File "C:\Users\basuke\webkit-trunk\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 270, in handle
      self._run_test(test_input, test_list_name)
    File "C:\Users\basuke\webkit-trunk\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 294, in _run_test
      result = self._run_test_with_or_without_timeout(test_input, test_timeout_sec, stop_when_done)
    File "C:\Users\basuke\webkit-trunk\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 337, in _run_test_with_or_without_timeout
      return self._run_test_in_this_thread(test_input, stop_when_done)
    File "C:\Users\basuke\webkit-trunk\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 418, in _run_test_in_this_thread
      return self._run_single_test(self._driver, test_input, stop_when_done)
    File "C:\Users\basuke\webkit-trunk\Tools\Scripts\webkitpy\layout_tests\controllers\layout_test_runner.py", line 422, in _run_single_test
      self._name, driver, test_input, stop_when_done)
    File "C:\Users\basuke\webkit-trunk\Tools\Scripts\webkitpy\layout_tests\controllers\single_test_runner.py", line 46, in run_single_test
      return runner.run()
    File "C:\Users\basuke\webkit-trunk\Tools\Scripts\webkitpy\layout_tests\controllers\single_test_runner.py", line 105, in run
      return self._run_compare_test()
    File "C:\Users\basuke\webkit-trunk\Tools\Scripts\webkitpy\layout_tests\controllers\single_test_runner.py", line 124, in _run_compare_test
      test_result_writer.write_test_result(self._filesystem, self._port, self._results_directory, self._test_name, driver_output, expected_driver_output, test_result.failures)
    File "C:\Users\basuke\webkit-trunk\Tools\Scripts\webkitpy\layout_tests\controllers\test_result_writer.py", line 48, in write_test_result
      failure.write_failure(writer, driver_output, expected_driver_output, port)
    File "C:\Users\basuke\webkit-trunk\Tools\Scripts\webkitpy\layout_tests\models\test_failures.py", line 121, in write_failure
      writer.write_text_files(driver_output.text, expected_driver_output.text)
    File "C:\Users\basuke\webkit-trunk\Tools\Scripts\webkitpy\layout_tests\controllers\test_result_writer.py", line 154, in write_text_files
      self.write_output_files(".txt", actual_text, expected_text)
    File "C:\Users\basuke\webkit-trunk\Tools\Scripts\webkitpy\layout_tests\controllers\test_result_writer.py", line 138, in write_output_files
      self._write_binary_file(actual_filename, output)
    File "C:\Users\basuke\webkit-trunk\Tools\Scripts\webkitpy\layout_tests\controllers\test_result_writer.py", line 111, in _write_binary_file
      self._filesystem.write_binary_file(path, contents)
    File "C:\Users\basuke\webkit-trunk\Tools\Scripts\webkitpy\common\system\filesystem.py", line 228, in write_binary_file
      with file(path, 'wb') as f:

[Basuke Suzuki]

Created attachment 345901 [details]
PATCH

[EWS Watchlist]

Comment on attachment 345901 [details]
PATCH

Attachment 345901 [details] did not pass win-ews (win):
Output: https://webkit-queues.webkit.org/results/8672057

New failing tests:
http/tests/security/canvas-remote-read-remote-video-localhost.html

[EWS Watchlist]

Created attachment 345913 [details]
Archive of layout-test-results from ews200 for win-future

The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews200  Port: win-future  Platform: CYGWIN_NT-6.1-2.9.0-0.318-5-3-x86_64-64bit

[Jonathan Bedard]

Do we need to apply this everywhere in the Filesystem class?  Ie, creating/deleting directory, listing elements in a directory, ect.

[Brent Fulgham]

(In reply to Jonathan Bedard from comment #5)
> Do we need to apply this everywhere in the Filesystem class?  Ie,
> creating/deleting directory, listing elements in a directory, ect.

That's probably the wisest course of action. There must still be some old WinAPI calls getting hit that need to know they are dealing with modern file lengths. So gross.

[Basuke Suzuki]

(In reply to Brent Fulgham from comment #6)
> (In reply to Jonathan Bedard from comment #5)
> > Do we need to apply this everywhere in the Filesystem class?  Ie,
> > creating/deleting directory, listing elements in a directory, ect.
> 
> That's probably the wisest course of action. There must still be some old
> WinAPI calls getting hit that need to know they are dealing with modern file
> lengths. So gross.

The wisest course of action must be just taking away from Windows :), but yes, at the end we have to do to whole those cases. Should I do that on this patch? Or open another bug?

[Jonathan Bedard]

(In reply to Basuke Suzuki from comment #7)
> (In reply to Brent Fulgham from comment #6)
> > (In reply to Jonathan Bedard from comment #5)
> > > Do we need to apply this everywhere in the Filesystem class?  Ie,
> > > creating/deleting directory, listing elements in a directory, ect.
> > 
> > That's probably the wisest course of action. There must still be some old
> > WinAPI calls getting hit that need to know they are dealing with modern file
> > lengths. So gross.
> 
> The wisest course of action must be just taking away from Windows :), but
> yes, at the end we have to do to whole those cases. Should I do that on this
> patch? Or open another bug?

I would be for including it in this change.

[Basuke Suzuki]

(In reply to Jonathan Bedard from comment #8)
> (In reply to Basuke Suzuki from comment #7)
> > (In reply to Brent Fulgham from comment #6)
> > > (In reply to Jonathan Bedard from comment #5)
> > > > Do we need to apply this everywhere in the Filesystem class?  Ie,
> > > > creating/deleting directory, listing elements in a directory, ect.
> > > 
> > > That's probably the wisest course of action. There must still be some old
> > > WinAPI calls getting hit that need to know they are dealing with modern file
> > > lengths. So gross.
> > 
> > The wisest course of action must be just taking away from Windows :), but
> > yes, at the end we have to do to whole those cases. Should I do that on this
> > patch? Or open another bug?
> 
> I would be for including it in this change.

Okay.

[Fujii Hironori]

(In reply to Basuke Suzuki from comment #0)
> link:
> https://docs.microsoft.com/ja-jp/windows/desktop/FileIO/naming-a-file#maximum-path-length-limitation

According to this document, there is a regstory key to opt-in the long path since Windows 10 version 1607.
HKLM\SYSTEM\CurrentControlSet\Control\FileSystem LongPathsEnabled

Did you try?

[Basuke Suzuki]

(In reply to Fujii Hironori from comment #10)
> (In reply to Basuke Suzuki from comment #0)
> > link:
> > https://docs.microsoft.com/ja-jp/windows/desktop/FileIO/naming-a-file#maximum-path-length-limitation
> 
> According to this document, there is a regstory key to opt-in the long path
> since Windows 10 version 1607.
> HKLM\SYSTEM\CurrentControlSet\Control\FileSystem LongPathsEnabled
> 
> Did you try?

After a long silence, I finally figured out this was not supported by Python 2.7. It looks like LongPathsEnabled support is required for app to be enabled in the manifest and it is supported starting from Python 3.6.

https://docs.python.org/3/using/windows.html#removing-the-max-path-limitation

[Alex Christensen]

Comment on attachment 345901 [details]
PATCH

This has been requesting review for more than one year.  If this is still needed, please rebase and re-request review.

[EWS]

Committed r293132 (249834@main): <https://commits.webkit.org/249834@main>

Reviewed commits have been landed. Closing PR #333 and removing active labels.