Bug 229690 - [git-webkit] Relocate contributors.json
Summary: [git-webkit] Relocate contributors.json
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Jonathan Bedard
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-08-30 16:09 PDT by Jonathan Bedard
Modified: 2021-10-13 16:38 PDT (History)
10 users (show)

See Also:


Attachments
Patch (169.97 KB, patch)
2021-08-30 16:22 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Part 1 (146.31 KB, patch)
2021-08-31 10:15 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Part 2 (2.25 KB, patch)
2021-08-31 10:28 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Part 2 (3.60 KB, patch)
2021-08-31 10:37 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Part 3 (2.92 KB, patch)
2021-08-31 10:46 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Part 4 (2.84 KB, patch)
2021-08-31 10:55 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Part 3 (2.91 KB, patch)
2021-08-31 11:03 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Part 4 (3.76 KB, patch)
2021-08-31 11:53 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Part 5 (6.73 KB, patch)
2021-08-31 12:13 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Patch (134.55 KB, patch)
2021-08-31 15:22 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Patch (134.46 KB, patch)
2021-08-31 16:02 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Patch (160.97 KB, patch)
2021-09-01 08:23 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Part 3 (3.46 KB, patch)
2021-09-01 16:17 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Part 4 (2.92 KB, patch)
2021-09-01 16:24 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Part 5 (3.75 KB, patch)
2021-09-01 16:30 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Patch (6.72 KB, patch)
2021-09-01 16:33 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Part 5 (3.95 KB, patch)
2021-09-01 16:50 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Part 6 (6.72 KB, patch)
2021-09-01 16:56 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Part 7 (131.97 KB, patch)
2021-09-01 17:04 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Part 3 (3.83 KB, patch)
2021-09-02 09:03 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Part 4 (2.23 KB, patch)
2021-09-02 09:09 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Part 5 (3.97 KB, patch)
2021-09-02 09:12 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Part 6 (3.92 KB, patch)
2021-09-02 14:04 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Part 7 (131.99 KB, patch)
2021-09-02 17:22 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Patch for landing (131.97 KB, patch)
2021-10-07 15:23 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Bedard 2021-08-30 16:09:48 PDT
contributors.json should be relocated outside of webkitpy, because it's applicable to the entire repository, and used by tooling, not part of tooling.
Comment 1 Radar WebKit Bug Importer 2021-08-30 16:10:20 PDT
<rdar://problem/82552403>
Comment 2 Jonathan Bedard 2021-08-30 16:22:21 PDT
Created attachment 436826 [details]
Patch
Comment 3 Jonathan Bedard 2021-08-30 16:36:48 PDT
Pull-request: https://github.com/WebKit/WebKit/pull/6
Comment 4 Ryosuke Niwa 2021-08-30 17:20:07 PDT
Can we split off the inactive contributors from the top-level JSON while we're at it?
Comment 5 Ryosuke Niwa 2021-08-30 17:20:31 PDT
I also wonder if yaml would be a better fit for this file anyway. JSON is just so verbose!
Comment 6 Alexey Proskuryakov 2021-08-30 18:36:47 PDT
JSON is easier to parse in JavaScript, so probably not (bugs.webkit.org does this).

Not quite sure how to best identify inactive contributors programmatically, but yes, would be nice to have fewer autocomplete suggestions in CC lists.
Comment 7 Aakash Jain 2021-08-31 08:19:33 PDT
We can also consider changing the formatting to make it compact, similar to https://commits.webkit.org/240315@main (r280725)
Comment 8 Jonathan Bedard 2021-08-31 08:38:17 PDT
(In reply to Aakash Jain from comment #7)
> We can also consider changing the formatting to make it compact, similar to
> https://commits.webkit.org/240315@main (r280725)

Compacting contributors.json is a bit more difficult than you might think, because we have a script that re-writes contributors.json with pretty-print.
Comment 9 Jonathan Bedard 2021-08-31 10:15:40 PDT
Created attachment 436901 [details]
Part 1
Comment 10 Jonathan Bedard 2021-08-31 10:28:21 PDT
Created attachment 436904 [details]
Part 2
Comment 11 Jonathan Bedard 2021-08-31 10:37:47 PDT
Created attachment 436907 [details]
Part 2
Comment 12 Jonathan Bedard 2021-08-31 10:46:26 PDT
Created attachment 436908 [details]
Part 3
Comment 13 Jonathan Bedard 2021-08-31 10:55:49 PDT
Created attachment 436912 [details]
Part 4
Comment 14 Jonathan Bedard 2021-08-31 11:03:00 PDT
Created attachment 436914 [details]
Part 3
Comment 15 Jonathan Bedard 2021-08-31 11:53:15 PDT
Created attachment 436921 [details]
Part 4
Comment 16 Jonathan Bedard 2021-08-31 12:13:50 PDT
Created attachment 436925 [details]
Part 5
Comment 17 Aakash Jain 2021-08-31 12:23:21 PDT
Comment on attachment 436901 [details]
Part 1

View in context: https://bugs.webkit.org/attachment.cgi?id=436901&action=review

> ChangeLog:9
> +        * metadata/contributors.json: Copied from Tools/Scripts/webkitpy/common/config/contributors.json, made into list.

As I mentioned on slack, I don't think this is a good idea to modify the file and move it in single commit. As a reviewer, it's extremely hard for me to verify whether there were any mistakes while copying and editing this large file. 

There should be one commit which just move/copy the file without any changes in contributors.json at all (and required changes in other files), so that there isn't need to review the contents of contributors.json. There should be separate commit to change the contents of contributors.json
Comment 18 Jonathan Bedard 2021-08-31 12:51:25 PDT
Comment on attachment 436901 [details]
Part 1

View in context: https://bugs.webkit.org/attachment.cgi?id=436901&action=review

>> ChangeLog:9
>> +        * metadata/contributors.json: Copied from Tools/Scripts/webkitpy/common/config/contributors.json, made into list.
> 
> As I mentioned on slack, I don't think this is a good idea to modify the file and move it in single commit. As a reviewer, it's extremely hard for me to verify whether there were any mistakes while copying and editing this large file. 
> 
> There should be one commit which just move/copy the file without any changes in contributors.json at all (and required changes in other files), so that there isn't need to review the contents of contributors.json. There should be separate commit to change the contents of contributors.json

I understand the sentiment (although the file was generated by validate-committer-lists, as happens when a user modifies contributors.json normally, not manually edited), but the mechanics of editing a file like contributors.json, which is accessed by services that aren't version locked, means that we need a way to determine if we're looking at a new or old version of contributors.json. The way that webkit.org, bugs.webkit.org, ewe-build.webkit.org and webkit-bot all access contributors.json is directly from it's tip-of-tree source via either svn.webkit.org or GitHub. If we don't tie the format change of contributors.json to the rename of the file, these services can't tell if they're dealing with the new file format, or the old one
Comment 19 Jonathan Bedard 2021-08-31 15:16:06 PDT
Aakash and I talked about this via slack, our rough plan is this:
1. Copy existing contributors.json
2. Refactor contributors.json
3-6. Adopt new contributors.json in services
Comment 20 Jonathan Bedard 2021-08-31 15:22:33 PDT
Created attachment 436953 [details]
Patch
Comment 21 Aakash Jain 2021-08-31 15:55:48 PDT
Comment on attachment 436953 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=436953&action=review

> ChangeLog:9
> +        * metadata/contributors.json: Copied from Tools/Scripts/webkitpy/common/config/contributors.json.

was it copied through svn cp?
Comment 22 Jonathan Bedard 2021-08-31 16:02:50 PDT
Created attachment 436958 [details]
Patch
Comment 23 Jonathan Bedard 2021-08-31 16:17:33 PDT
Comment on attachment 436958 [details]
Patch

Sitting on this change until tomorrow
Comment 24 EWS 2021-09-01 08:10:20 PDT
Committed r281850 (241182@main): <https://commits.webkit.org/241182@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 436958 [details].
Comment 25 Jonathan Bedard 2021-09-01 08:23:33 PDT
Reopening to attach new patch.
Comment 26 Jonathan Bedard 2021-09-01 08:23:35 PDT
Created attachment 437031 [details]
Patch
Comment 27 Aakash Jain 2021-09-01 09:35:34 PDT
Since multiple different improvements are being made, I feel like the improvements should be done in separate bug(s) (with an explanation for that change). Making improvement to contributors.json with a title 'Relocate contributors.json' doesn't feel right.
Comment 28 Jonathan Bedard 2021-09-01 09:42:21 PDT
(In reply to Aakash Jain from comment #27)
> Since multiple different improvements are being made, I feel like the
> improvements should be done in separate bug(s) (with an explanation for that
> change). Making improvement to contributors.json with a title 'Relocate
> contributors.json' doesn't feel right.

I think this goes back to what we were discussing yesterday: the conversion from a dictionary to a list is part of the relocation when it comes to services. This is why I made a pull-request (https://github.com/WebKit/WebKit/pull/6) as well, so you can see why the changes to services need to combine the dictionary -> list change.

If we try and separate the refactor of contributors.json, this is going to be nonsense. We have a sequence of changes which depend on one another, a patch series lets us clearly communicate that dependency chain.
Comment 29 dewei_zhu 2021-09-01 15:29:34 PDT
Comment on attachment 437031 [details]
Patch

r=me
Comment 30 EWS 2021-09-01 15:46:37 PDT
Committed r281883 (241213@main): <https://commits.webkit.org/241213@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 437031 [details].
Comment 31 Jonathan Bedard 2021-09-01 16:17:10 PDT
Reopening to attach new patch.
Comment 32 Jonathan Bedard 2021-09-01 16:17:11 PDT
Created attachment 437078 [details]
Part 3
Comment 33 Jonathan Bedard 2021-09-01 16:24:53 PDT
Created attachment 437079 [details]
Part 4
Comment 34 Jonathan Bedard 2021-09-01 16:30:06 PDT
Created attachment 437081 [details]
Part 5
Comment 35 Jonathan Bedard 2021-09-01 16:33:31 PDT
Created attachment 437082 [details]
Patch
Comment 36 Jonathan Bedard 2021-09-01 16:50:48 PDT
Created attachment 437085 [details]
Part 5
Comment 37 Jonathan Bedard 2021-09-01 16:56:03 PDT
Created attachment 437086 [details]
Part 6
Comment 38 Jonathan Bedard 2021-09-01 17:04:33 PDT
Created attachment 437087 [details]
Part 7
Comment 39 Jonathan Bedard 2021-09-02 09:03:32 PDT
Created attachment 437157 [details]
Part 3
Comment 40 Aakash Jain 2021-09-02 09:07:12 PDT
Comment on attachment 437157 [details]
Part 3

rs=me
Comment 41 Jonathan Bedard 2021-09-02 09:09:07 PDT
Created attachment 437158 [details]
Part 4
Comment 42 Jonathan Bedard 2021-09-02 09:12:15 PDT
Created attachment 437159 [details]
Part 5
Comment 43 Aakash Jain 2021-09-02 09:20:03 PDT
Comment on attachment 437158 [details]
Part 4

View in context: https://bugs.webkit.org/attachment.cgi?id=437158&action=review

rs=me with a comment

> Websites/bugs.webkit.org/committers-autocomplete.js:27
> +    var COMMITTERS_URL = 'https://svn.webkit.org/repository/webkit/trunk/metadata/contributors.json';

we should start reading it from github (like ews). from https://raw.githubusercontent.com/WebKit/WebKit/main/metadata/contributors.json
Comment 44 Jonathan Bedard 2021-09-02 09:21:41 PDT
(In reply to Aakash Jain from comment #43)
> Comment on attachment 437158 [details]
> Part 4
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=437158&action=review
> 
> rs=me with a comment
> 
> > Websites/bugs.webkit.org/committers-autocomplete.js:27
> > +    var COMMITTERS_URL = 'https://svn.webkit.org/repository/webkit/trunk/metadata/contributors.json';
> 
> we should start reading it from github (like ews). from
> https://raw.githubusercontent.com/WebKit/WebKit/main/metadata/contributors.
> json

Agreed, but that seemed like something for a different change, this change is already more than we want it to be, I think
Comment 45 Aakash Jain 2021-09-02 09:33:11 PDT
Comment on attachment 437159 [details]
Part 5

View in context: https://bugs.webkit.org/attachment.cgi?id=437159&action=review

> Tools/CISupport/ews-build/steps.py:907
> +        repo_root = os.path.dirname(os.path.dirname(os.path.dirname(cwd)))

It's fine for now, but this method of determining repo root based on location of current file seems fragile, would be good to calculate it in some other reliable method.
Comment 46 EWS 2021-09-02 09:43:01 PDT
Committed r281932 (241241@main): <https://commits.webkit.org/241241@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 437157 [details].
Comment 47 Jonathan Bedard 2021-09-02 10:13:34 PDT
Still a few more changes to land.
Comment 48 EWS 2021-09-02 10:32:59 PDT
Committed r281936 (241245@main): <https://commits.webkit.org/241245@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 437158 [details].
Comment 49 Jonathan Bedard 2021-09-02 11:08:35 PDT
Still more changes to land.
Comment 50 EWS 2021-09-02 11:31:18 PDT
Committed r281938 (241247@main): <https://commits.webkit.org/241247@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 437159 [details].
Comment 51 Jonathan Bedard 2021-09-02 14:04:44 PDT
Reopening to attach new patch.
Comment 52 Jonathan Bedard 2021-09-02 14:04:46 PDT
Created attachment 437192 [details]
Part 6
Comment 53 Yusuke Suzuki 2021-09-02 14:09:19 PDT
Comment on attachment 437192 [details]
Part 6

r=me Please change the running webkitbot code in the server.
Comment 54 EWS 2021-09-02 15:00:04 PDT
Committed r281956 (241263@main): <https://commits.webkit.org/241263@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 437192 [details].
Comment 55 Jonathan Bedard 2021-09-02 17:22:32 PDT
Reopening to attach new patch.
Comment 56 Jonathan Bedard 2021-09-02 17:22:34 PDT
Created attachment 437222 [details]
Part 7
Comment 57 Jonathan Bedard 2021-09-02 17:23:47 PDT
Comment on attachment 437222 [details]
Part 7

Last one!
Comment 58 dewei_zhu 2021-10-07 14:42:33 PDT
Comment on attachment 437157 [details]
Part 3

r=me
Comment 59 dewei_zhu 2021-10-07 14:44:00 PDT
Comment on attachment 437222 [details]
Part 7

r=me
Comment 60 Jonathan Bedard 2021-10-07 15:23:24 PDT
Created attachment 440546 [details]
Patch for landing
Comment 61 EWS 2021-10-07 15:55:41 PDT
Committed r283750 (242672@main): <https://commits.webkit.org/242672@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 440546 [details].