[webkitscmpy] Cache identifiers in Git checkouts

[Jonathan Bedard]

We need to create an identifier cache fast enough for use by `git log` and `git blame`

[Radar WebKit Bug Importer]

<rdar://problem/77789230>

[Jonathan Bedard]

Created attachment 428204 [details]
Patch

[Jonathan Bedard]

Created attachment 428279 [details]
Patch

[Jonathan Bedard]

Created attachment 428327 [details]
Patch

[Dean Johnson]

Taking this from Jonathan since we've been iterating on this together and I've got a new patch to upload.

[Dean Johnson]

Created attachment 429213 [details]
Add identifiers cache

[Dean Johnson]

Created attachment 429215 [details]
Add identifiers cache v2

Fixed a couple test failures caused by refactoring. Still need to figure out how to correctly handle remotes for these tests:

[1806/1995] webkitscmpy.test.canonicalize_unittest.TestCanonicalize.test_branch_commits failed:
  Traceback (most recent call last):
    File "/Volumes/Data/dean-home/Projects/Safari/OpenSource/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/test/canonicalize_unittest.py", line 199, in test_branch_commits
      self.assertEqual(commit_a.message, 'New commit 1\nIdentifier: 2.3@branch-a')
  AssertionError: 'New commit 1\nIdentifier: 2.4@branch-a' != 'New commit 1\nIdentifier: 2.3@branch-a'
    New commit 1
  - Identifier: 2.4@branch-a?               ^
  + Identifier: 2.3@branch-a?               ^


[1823/1995] webkitscmpy.test.checkout_unittest.TestCheckout.test_checkout_git_svn failed:
  Traceback (most recent call last):
    File "/Volumes/Data/dean-home/Projects/Safari/OpenSource/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/test/checkout_unittest.py", line 62, in test_checkout_git_svn
      self.assertEqual('621652add7fc416099bd2063366cc38ff61afe36', local.Git(self.path).commit().hash)
  AssertionError: '621652add7fc416099bd2063366cc38ff61afe36' != 'fff83bb2d9171b4d9196e977eb0508fd57e7a08d'
  - 621652add7fc416099bd2063366cc38ff61afe36
  + fff83bb2d9171b4d9196e977eb0508fd57e7a08d

And how to handle the broken SVN detection test:

[153/1995] webkitpy.common.checkout.scm.detection_unittest.SCMDetectorTest.test_detect_scm_system failed:
  Traceback (most recent call last):
    File "/Volumes/Data/dean-home/Projects/Safari/OpenSource/Tools/Scripts/webkitpy/common/checkout/scm/detection_unittest.py", line 49, in test_detect_scm_system
      self.assertEqual(
  AssertionError: "MOCK run_command: ['git', 'rev-parse', '--is-inside-work-tree'], cwd=/\n" != "MOCK run_command: ['svn', 'info'], cwd=/\nMOCK run_command: [[49 chars]=/\n"
  + MOCK run_command: ['svn', 'info'], cwd=/
    MOCK run_command: ['git', 'rev-parse', '--is-inside-work-tree'], cwd=/

[Dean Johnson]

Created attachment 429217 [details]
Identifiers cache v2

[Jonathan Bedard]

Created attachment 432420 [details]
Patch

[Jonathan Bedard]

Comment on attachment 432420 [details]
Patch

A high-level comment: This ends up being a 3-4x improvement on `git-webkit` commands in most cases. Where it's really important, though, is in `git log` and `git blame`, which will come in a later patch. Without this cache, implementations of those two commands would be extremely slow

[Jonathan Bedard]

Created attachment 432528 [details]
Patch

Comment on attachment 432528 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=432528&action=review

r=me

> Tools/Scripts/libraries/webkitscmpy/webkitscmpy/local/git.py:51
> +            if os.path.exists(self.path):

Maybe we can do
``
if not os.path.exists(self.path):
    return
``
to avoid indentation.

[Jonathan Bedard]

Created attachment 432642 [details]
Patch for landing

[EWS]

Committed r279445 (239301@main): <https://commits.webkit.org/239301@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 432642 [details].

[Jonathan Bedard]

Reopening to attach new patch.

[Jonathan Bedard]

Created attachment 432729 [details]
Patch

Comment on attachment 432729 [details]
Patch

r=me

[Jonathan Bedard]

Created attachment 432732 [details]
Patch

[Jonathan Bedard]

Committed r279488 (239340@main): <https://commits.webkit.org/239340@main>