Bug 72275 – check-webkit-style broken by r99773: "Could not determine the port"

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

As of r100141, check-webkit-style fails even if you haven't made any changes.  Here is the start of the output I see when running on a Mac:

$ Tools/Scripts/check-webkit-style 
WARNING: Could not determine the port for LayoutTests/platform/chromium/test_expectations.txt. Using 'test' port, but platform-specific expectations will fail the check.
ERROR: FAILURES FOR <leopard, x86, release, cpu> in LayoutTests/platform/test/test_expectations.txt
ERROR: Line:10 Path does not exist. fast/css/large-list-of-rules-crash.html
ERROR: Line:11 Path does not exist. fast/dom/Window/window-postmessage-clone-really-deep-array.html
[... lots more errors ...]

I've been doing a binary search (updating Webkit to different revision numbers and trying to run check-webkit-style).  It looks like the problem was introduced between r99700 and r99800.

synced to r99000 : succeeds
synced to r99700 : succeeds
synced to r99800 : fails
synced to r99900 : fails
synced to r100000 : fails
synced to r100118 : fails

I'll keep looking...

I have narrowed it down and confirmed that this started happening as of Eric Seidel's http://trac.webkit.org/changeset/99773 :

synced to r99000 : succeeds
synced to r99700 : succeeds
synced to r99750 : succeeds
synced to r99762 : succeeds
synced to r99770 : succeeds
synced to r99772 : succeeds
synced to r99773 : fails
synced to r99774 : fails
synced to r99775 : fails
synced to r99800 : fails
synced to r99900 : fails
synced to r100000 : fails
synced to r100118 : fails

Eric, do you have a plan to fix this?  Should we roll out your change?  Or what?

[adding sheriff, tonyg]

[Tony Gentilcore]

*** Bug 72282 has been marked as a duplicate of this bug. ***

[Tony Gentilcore]

The rollback for this is not clean at all. It might actually be easiest to let Eric take a crack at this today. If the fix isn't trivial, we can roll back r99791, r99781 and r99773 to get things back to a working state.

[Tony Gentilcore]

[+pkasting, current PST sheriff]

Is Eric Seidel even around today?  I have been unable to summon him on #webkit.

Unless I am mistaken, this problem is causing "webkit-patch upload" to fail for everyone today, and thus developers either (a) force-landing their patches without running the style-checker, or (b) unable to upload any patches for review.

Is there a workaround?

[Tony Gentilcore]

(In reply to comment #7)
> Is Eric Seidel even around today?  I have been unable to summon him on #webkit.
> 
> Unless I am mistaken, this problem is causing "webkit-patch upload" to fail for everyone today

I've been using webkit-patch upload all day today with no problems. It outputs the style errors and prompts y/n to continue. Hit y and everything continues. Style warnings are not meant to be fatal.

Feel free to roll this back yourself if you like, but I have to run for the evening.

(In reply to comment #8)
> > Unless I am mistaken, this problem is causing "webkit-patch upload" to fail for everyone today
> 
> I've been using webkit-patch upload all day today with no problems. It outputs the style errors and prompts y/n to continue. Hit y and everything continues. Style warnings are not meant to be fatal.

OK with me, as long as reviewers feel the same way!  Just wanted to make sure we are all on the same page.

[Eric Seidel (no email)]

looking now.

[Eric Seidel (no email)]

I have a fix, posting shortly.

[Eric Seidel (no email)]

Created attachment 115010 [details]
Patch

[Eric Seidel (no email)]

You should always feel welcome to roll out any patch! :)  I'm sorry this one caused you so much trouble. :(

[Eric Seidel (no email)]

Sorry, I forgot to land this!

[Eric Seidel (no email)]

Committed r100231: <http://trac.webkit.org/changeset/100231>

[Tony Chang]

test-webkitpy appears to be failing on chromium windows.

http://build.webkit.org/builders/Chromium%20Win%20Release%20%28Tests%29/builds/20786/steps/webkitpy-test/logs/stdio


ERROR: test_already_seen_test (webkitpy.style.checkers.test_expectations_unittest.TestExpectationsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "E:\google-windows-2\chromium-win-release-tests\build\Tools\Scripts\webkitpy\style\checkers\test_expectations_unittest.py", line 161, in test_already_seen_test
    "Duplicate or ambiguous expectation. %s  [test/expectations] [5]" % self._test_file)
  File "E:\google-windows-2\chromium-win-release-tests\build\Tools\Scripts\webkitpy\style\checkers\test_expectations_unittest.py", line 82, in assert_lines_lint
    self._error_collector)
  File "E:\google-windows-2\chromium-win-release-tests\build\Tools\Scripts\webkitpy\style\checkers\test_expectations.py", line 79, in __init__
    host._initialize_scm()
  File "E:\google-windows-2\chromium-win-release-tests\build\Tools\Scripts\webkitpy\common\host.py", line 67, in _initialize_scm
    self._scm = default_scm(patch_directories)
  File "E:\google-windows-2\chromium-win-release-tests\build\Tools\Scripts\webkitpy\common\checkout\scm\detection.py", line 61, in default_scm
    scm_system = detect_scm_system(cwd, patch_directories)
  File "E:\google-windows-2\chromium-win-release-tests\build\Tools\Scripts\webkitpy\common\checkout\scm\detection.py", line 79, in detect_scm_system
    return SVN(cwd=absolute_path, patch_directories=patch_directories)
  File "E:\google-windows-2\chromium-win-release-tests\build\Tools\Scripts\webkitpy\common\checkout\scm\svn.py", line 76, in __init__
    SCM.__init__(self, cwd, executive)
  File "E:\google-windows-2\chromium-win-release-tests\build\Tools\Scripts\webkitpy\common\checkout\scm\scm.py", line 63, in __init__
    self.checkout_root = self.find_checkout_root(self.cwd)
  File "E:\google-windows-2\chromium-win-release-tests\build\Tools\Scripts\webkitpy\common\checkout\scm\svn.py", line 108, in find_checkout_root
    uuid = SVN.find_uuid(path)
  File "E:\google-windows-2\chromium-win-release-tests\build\Tools\Scripts\webkitpy\common\checkout\scm\svn.py", line 94, in find_uuid
    return cls.value_from_svn_info(path, 'Repository UUID')
  File "E:\google-windows-2\chromium-win-release-tests\build\Tools\Scripts\webkitpy\common\checkout\scm\svn.py", line 100, in value_from_svn_info
    info_output = Executive().run_command(svn_info_args, cwd=path).rstrip()
  File "E:\google-windows-2\chromium-win-release-tests\build\Tools\Scripts\webkitpy\common\system\executive.py", line 420, in run_command
    close_fds=self._should_close_fds())
  File "E:\google-windows-2\chromium-win-release-tests\build\Tools\Scripts\webkitpy\common\system\executive.py", line 476, in popen
    return subprocess.Popen(*args, **kwargs)
  File "c:\Python26\lib\subprocess.py", line 623, in __init__
    errread, errwrite)
  File "c:\Python26\lib\subprocess.py", line 833, in _execute_child
    startupinfo)
WindowsError: [Error 2] The system cannot find the file specified

[Adam Barth]

Sounds like the windows hacks are failing to engage.

[Eric Seidel (no email)]

Yes, I need to move where the windows-hacks are. :(

I guess I'll add a hack for this hack for now.

[Eric Seidel (no email)]

Might be better to just roll this out.  I won't get to this until late this evening.

[Eric Seidel (no email)]

(In reply to comment #17)
> Sounds like the windows hacks are failing to engage.

I need to move the windows hacks out of the ChromiumWin port and into Host or SCM.  Both are ugly.  But everything involving Windows is ugly.

[Eric Seidel (no email)]

Committed r100421: <http://trac.webkit.org/changeset/100421>

[Eric Seidel (no email)]

Committed r100423: <http://trac.webkit.org/changeset/100423>

(In reply to comment #22)
> Committed r100423: <http://trac.webkit.org/changeset/100423>

I suspect that one of these recent commits broke something else in port selection... it looks like layout tests are not properly handling GPU vs CPU for some platforms.

Sometime between webkit r100298 and r100423 (inclusive), running layout_tests specifying --platform chromium-gpu-cg-mac went from running 245 tests to running 27186 tests.  (The latter is the number of tests I would expect to see run for chromium-cg-mac, not chromium-gpu-cg-mac .)

Check out the "Found:" line from these layout test runs at r100298 and r100423 (you can divine the platform specified for each run from the filename):

chrome-bot@dhcp-172-29-92-177 bash$ grep Found: *webkit100298*/output.txt
20111115-144908-unmodified-webkit100298-build-chromium-gpu-mac/output.txt:2011-11-15 14:49:16,660 43592 printing.py:469 INFO Found:  1322 tests
20111115-145211-unmodified-webkit100298-build-chromium-mac/output.txt:2011-11-15 14:52:31,264 43773 printing.py:469 INFO Found:  27180 tests
20111116-032322-unmodified-webkit100298-build-chromium-gpu-cg-mac/output.txt:2011-11-16 03:23:37,067 52907 printing.py:469 INFO Found:  245 tests
20111116-032416-unmodified-webkit100298-build-chromium-cg-mac/output.txt:2011-11-16 03:24:56,233 53019 printing.py:469 INFO Found:  27180 tests

chrome-bot@dhcp-172-29-92-168 bash$ grep Found: *webkit100423*/output.txt
20111116-041807-unmodified-webkit100423-buildUNKNOWN-chromium-gpu-mac/output.txt:2011-11-16 04:18:17,296 4269 printing.py:469 INFO Found:  1077 tests
20111116-042058-unmodified-webkit100423-buildUNKNOWN-chromium-mac/output.txt:2011-11-16 04:21:51,848 4451 printing.py:469 INFO Found:  27186 tests
20111116-050320-unmodified-webkit100423-buildUNKNOWN-chromium-gpu-cg-mac/output.txt:2011-11-16 05:04:20,744 9929 printing.py:469 INFO Found:  27186 tests
20111116-053248-unmodified-webkit100423-buildUNKNOWN-chromium-cg-mac/output.txt:2011-11-16 05:33:50,132 11927 printing.py:469 INFO Found:  27186 tests

(In reply to comment #23)
> (In reply to comment #22)
> > Committed r100423: <http://trac.webkit.org/changeset/100423>
> 
> I suspect that one of these recent commits broke something else in port selection... it looks like layout tests are not properly handling GPU vs CPU for some platforms.

I believe this latest problem is the result of a webkit change within the range r100350:r100398

Split off as https://bugs.webkit.org/show_bug.cgi?id=72498 ('NRWT not running webkit_gpu_tests properly for Leopard (Mac OS 10.5)')