Bug 275693 – KeyError: '8e2703a62c3ac7282b9f96744a53646a89b59c9a' when running 'git-webkit pr'

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Michael Catanzaro]

I can't use 'git-webkit pr' anymore. Not sure what's going wrong, but it thinks something is very special about 8e2703a62c3ac7282b9f96744a53646a89b59c9a (which is 279551@main).

$ git-webkit -v pr
INFO:webkitscmpy:Using committed changes...
INFO:webkitscmpy:Rebasing 'eng/WPEGTK-IPC-socket-should-use-SOCK_CLOEXEC-on-Linux' on 'main'...
From https://github.com/WebKit/WebKit
 * branch                      main       -> FETCH_HEAD
Current branch eng/WPEGTK-IPC-socket-should-use-SOCK_CLOEXEC-on-Linux is up to date.
Traceback (most recent call last):
  File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/git-webkit", line 63, in <module>
    sys.exit(program.main(
             ^^^^^^^^^^^^^
  File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/program/__init__.py", line 185, in main
    return parsed.main(
           ^^^^^^^^^^^^
  File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/program/pull_request.py", line 769, in main
    return cls.create_pull_request(repository, args, branch_point)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/program/pull_request.py", line 424, in create_pull_request
    if repository.pull(rebase=True, branch=branch_point.branch, remote=source_remote):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/local/git.py", line 1237, in pull
    self.cache.clear(self.branch)
  File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/local/git.py", line 225, in clear
    self._fill(branch)
  File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/local/git.py", line 84, in _fill
    branch_point = int(self._hash_to_identifiers[self._ordered_commits[branch][0]].split('@')[0])
                       ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/nested_fuzzy_dict.py", line 53, in __getitem__
    raise KeyError(keyname)
KeyError: '8e2703a62c3ac7282b9f96744a53646a89b59c9a'

[Radar WebKit Bug Importer]

<rdar://problem/130680688>

[Sam Sneddon [:gsnedders]]

We really need to handle errors like this within webkitscmpy, but "rm $( git rev-parse --git-common-dir )/identifiers.json" should fix this (by forcing the cache to be recreated).

[Michael Catanzaro]

I can confirm that works. Nice!

[Alexey Proskuryakov]

Did you happen to keep the cache, to identify what was wrong?

[Michael Catanzaro]

Unfortunately no, but I can guess what caused it: Ctrl+C. I do that often enough and probably shouldn't.

[Sam Sneddon [:gsnedders]]

(In reply to Alexey Proskuryakov from comment #4)
> Did you happen to keep the cache, to identify what was wrong?

With similar issues, it's historically not been super helpful having the cache — like, it merely shows us that "yes, this is in fact broken", but doesn't tell us _how_ it got into that state.

If it's caused by running Ctrl+C while updating the cache, especially happens while we're blocked on a subprocess, that's something that we should be handling when the subprocess exits with SIGINT.

IMO, this is still actionable (both via checking we're handling that reasonably and by falling back to regenerating the cache when we do get a KeyError/IndexError here), thus -> REOPENED.

[Alexey Proskuryakov]

Let's file a new clean bug with a plan though. "KeyError: '8e2703a62c3ac7282b9f96744a53646a89b59c9a' when running 'git-webkit pr'" is no longer accurate.

[Michael Catanzaro]

Hitting this again now. Looks the same as before, only difference is the commit hash is different:

Traceback (most recent call last):
  File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/git-webkit", line 63, in <module>
    sys.exit(program.main(
             ~~~~~~~~~~~~^
        identifier_template=is_webkit_filter('Canonical link: https://commits.webkit.org/{}'),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        classifier=is_webkit_filter(classifier()),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ))
    ^
  File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/program/__init__.py", line 184, in main
    return parsed.main(
           ~~~~~~~~~~~^
        args=parsed,
        ^^^^^^^^^^^^
    ...<5 lines>...
        canonical_svn=canonical_svn,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/program/pull_request.py", line 776, in main
    return cls.create_pull_request(repository, args, branch_point)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/program/pull_request.py", line 439, in create_pull_request
    if repository.pull(rebase=True, branch=branch_point.branch, remote=source_remote):
       ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/local/git.py", line 1258, in pull
    self.cache.clear(self.branch)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/local/git.py", line 222, in clear
    self._fill(branch)
    ~~~~~~~~~~^^^^^^^^
  File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/local/git.py", line 84, in _fill
    branch_point = int(self._hash_to_identifiers[self._ordered_commits[branch][0]].split('@')[0])
                       ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mcatanzaro/Projects/WebKit/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/nested_fuzzy_dict.py", line 53, in __getitem__
    raise KeyError(keyname)
KeyError: '405782ff2e808caa3d518d03fc5de7b15de8098b'

This time I know not to delete the cache. I presume I used Ctrl+C again, and that we need to catch the KeyError and rebuild the cache.

Alternatively or additionally, we could try catching and handling KeyboardInterrupt to prevent things from going wrong in the first place.

[Michael Catanzaro]

Created attachment 476638 [details]
Michael's identifiers.json

[Michael Catanzaro]

Pull request: https://github.com/WebKit/WebKit/pull/50305

[EWS]

Committed 299639@main (22f2cf717f6c): <https://commits.webkit.org/299639@main>

Reviewed commits have been landed. Closing PR #50305 and removing active labels.