Bug 220736 - REGRESSION(r271506): webkit-patch keyring integration is broken on Linux
Summary: REGRESSION(r271506): webkit-patch keyring integration is broken on Linux
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: PC Linux
: P2 Normal
Assignee: Michael Catanzaro
URL:
Keywords: InRadar
Depends on: 220744
Blocks:
  Show dependency treegraph
 
Reported: 2021-01-19 10:18 PST by Michael Catanzaro
Modified: 2021-10-07 18:55 PDT (History)
5 users (show)

See Also:


Attachments
Patch (1.64 KB, patch)
2021-01-19 11:23 PST, Michael Catanzaro
no flags Details | Formatted Diff | Diff
Patch (1.39 KB, patch)
2021-01-19 11:37 PST, Michael Catanzaro
no flags Details | Formatted Diff | Diff
Patch (1.39 KB, patch)
2021-01-19 11:39 PST, Michael Catanzaro
no flags Details | Formatted Diff | Diff
Patch (1.39 KB, patch)
2021-01-19 11:39 PST, Michael Catanzaro
no flags Details | Formatted Diff | Diff
Patch (2.03 KB, patch)
2021-01-19 11:44 PST, Michael Catanzaro
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Catanzaro 2021-01-19 10:18:33 PST
webkit-patch is no longer able to look up my Bugzilla password from the system keyring, instead prompting me to enter my password each time. I don't know why yet... probably related to r271506, but I need to investigate more.
Comment 1 Michael Catanzaro 2021-01-19 11:12:09 PST
Debug patch:

diff --git a/Tools/Scripts/webkitpy/common/net/credentials.py b/Tools/Scripts/webkitpy/common/net/credentials.py
index ef98ee15b3c4..c8781e5d5232 100644
--- a/Tools/Scripts/webkitpy/common/net/credentials.py
+++ b/Tools/Scripts/webkitpy/common/net/credentials.py
@@ -161,8 +161,8 @@ class Credentials(object):
         if username and not password and self._keyring and use_stored_credentials:
             try:
                 password = self._keyring.get_password(self.host, username)
-            except:
-                pass
+            except Exception as e:
+                print('failed to get password from keyring: {}'.format(str(e)))
 
         if not password:
             password = user.prompt_password("%s password for %s: " % (self.host, username))

It prints:

$ webkit-patch upload 220736
Total errors found: 0 in 1 files
failed to get password from keyring: No recommended backend was available. Install a recommended 3rd party backend package; or, install the keyrings.alt package if you want to use the non-recommended backends. See https://pypi.org/project/keyring for details.
bugs.webkit.org password for mcatanzaro@gnome.org:

So somehow python-keyring is no longer able to use python-secretservice, which is bad.
Comment 2 Michael Catanzaro 2021-01-19 11:20:32 PST
> So somehow python-keyring is no longer able to use python-secretservice,
> which is bad.

Er, it's called python-secretstorage. That is required for system keyring integration on Linux. Historically, our autoinstalled python-keyring has been able to use the system copy of python-secretstorage. Maybe the updated version is not?

Anyway, if I revert to our previous version of python-keyring, the bug disappears. The bug also goes away if I upgrade to the latest stable version of python-keyring. I'm not sure why exactly version 18.0.1 was selected, but it seems to be a bad version. CC: Jonathan in case this particular version was selected for some important reason.
Comment 3 Michael Catanzaro 2021-01-19 11:23:16 PST
Created attachment 417891 [details]
Patch
Comment 4 Michael Catanzaro 2021-01-19 11:30:23 PST
Comment on attachment 417891 [details]
Patch

OK, webkitpy EWS is red. I didn't realize we still support python2. The new version of python-keyring does not support python2. So I need to find an older version that still supports python2, is new enough for the GitHub code to work, and doesn't exhibit this secretstorage bug. Hi Jonathan, can you comment regarding why you upgraded python-keyring? I assume you needed a newer version of the library for... something? What exactly was required?

Then I'm not sure what is wrong with the python3 test. ModuleNotFoundError: No module named 'monotonic' immediately after installing python-monotonic. Hm.
Comment 5 Michael Catanzaro 2021-01-19 11:32:21 PST
(In reply to Michael Catanzaro from comment #2)
> I'm not sure why exactly version 18.0.1 was selected

This was simply the last version to support python2.
Comment 6 Michael Catanzaro 2021-01-19 11:37:32 PST Comment hidden (obsolete)
Comment 7 Michael Catanzaro 2021-01-19 11:39:19 PST Comment hidden (obsolete)
Comment 8 Michael Catanzaro 2021-01-19 11:39:41 PST Comment hidden (obsolete)
Comment 9 Michael Catanzaro 2021-01-19 11:41:39 PST
(In reply to Michael Catanzaro from comment #2)
> I'm not sure why exactly version 18.0.1 was selected, but
> it seems to be a bad version.

It seems versions 12 is the first version that is broken. So version 11 is the last version that supports python2 and works properly on Linux. I wonder if this is new enough for Jonathan's GitHub support that landed in r271506.
Comment 10 Michael Catanzaro 2021-01-19 11:44:29 PST
Created attachment 417897 [details]
Patch
Comment 11 Michael Catanzaro 2021-01-19 11:46:38 PST
webkitpy bot is green. Jonathan, can you please check to ensure I haven't broken your GitHub credentials work?
Comment 12 Jonathan Bedard 2021-01-19 12:34:08 PST
(In reply to Michael Catanzaro from comment #11)
> webkitpy bot is green. Jonathan, can you please check to ensure I haven't
> broken your GitHub credentials work?

I didn't need to update the library, I assumed (erroneously, clearly), that a newer version would be better. Before we land this change, I'd like to land another change that will help some of our bots out, because otherwise changing the library version on them will require manual intervention. Is it Ok if this change waits a few hours before landing?
Comment 13 Michael Catanzaro 2021-01-19 12:39:03 PST
Sure. I can't land it now anyway, since commit-queue is taking the day off, and I don't have an SVN account anymore.

If you could give r+ when you're ready, that would be great.
Comment 14 Jonathan Bedard 2021-01-19 12:44:52 PST
(In reply to Michael Catanzaro from comment #13)
> Sure. I can't land it now anyway, since commit-queue is taking the day off,
> and I don't have an SVN account anymore.

yeah, that's the other thing I'm working on right now :)

> 
> If you could give r+ when you're ready, that would be great.
Comment 15 Jonathan Bedard 2021-01-19 13:00:52 PST
Comment on attachment 417897 [details]
Patch

Marking this cq- right now so no one accidentally commits it before our infrastructure is ready, in particular, we need https://bugs.webkit.org/show_bug.cgi?id=220744 first.
Comment 16 EWS 2021-01-19 17:47:18 PST
Committed r271634: <https://trac.webkit.org/changeset/271634>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 417897 [details].
Comment 17 Radar WebKit Bug Importer 2021-01-19 17:48:14 PST
<rdar://problem/73382360>
Comment 18 Philippe Normand 2021-01-27 01:21:15 PST
Still borken BTW, when running webkit-patch in the Flatpak SDK runtime...
Comment 19 Jonathan Bedard 2021-01-27 07:51:59 PST
(In reply to Philippe Normand from comment #18)
> Still borken BTW, when running webkit-patch in the Flatpak SDK runtime...

What exactly does broken mean in this case? Seems likely it's something about keyring being unable to find some sort of key store.
Comment 20 Michael Catanzaro 2021-01-27 07:59:44 PST
(In reply to Jonathan Bedard from comment #19)
> What exactly does broken mean in this case? Seems likely it's something
> about keyring being unable to find some sort of key store.

Yup, the problem is python-keyring is totally unable to find the system keyring. It's as if it thinks the system keyring doesn't exist. I did not debug it to figure out why, since the problem I was hitting was definitely fixed in the latest version.
Comment 21 Michael Catanzaro 2021-01-27 08:01:02 PST
(In reply to Michael Catanzaro from comment #1)
> $ webkit-patch upload 220736
> Total errors found: 0 in 1 files
> failed to get password from keyring: No recommended backend was available.
> Install a recommended 3rd party backend package; or, install the
> keyrings.alt package if you want to use the non-recommended backends. See
> https://pypi.org/project/keyring for details.
> bugs.webkit.org password for mcatanzaro@gnome.org:

Looks like it's failing to discover its own backends. It's supposed to use python-secretstorage to talk to gnome-keyring via libsecret, but for whatever reason it doesn't realize python-secretstorage is installed.
Comment 22 Philippe Normand 2021-01-27 08:05:38 PST
I had this error:

failed to get password from keyring: No recommended backend was available. Install a recommended 3rd party backend package; or, install the keyrings.alt package if you want to use the non-recommended backends. See https://pypi.org/project/keyring for details.
Comment 23 Jonathan Bedard 2021-01-27 08:40:50 PST
(In reply to Philippe Normand from comment #22)
> I had this error:
> 
> failed to get password from keyring: No recommended backend was available.
> Install a recommended 3rd party backend package; or, install the
> keyrings.alt package if you want to use the non-recommended backends. See
> https://pypi.org/project/keyring for details.

Oh, I wonder if it checks for the module instead of importing it, which would by-pass our auto-install clever-ness....or the "if sys.platform == 'linux':" is false in webkitscmpy's __init__.py