36193 2010-03-16 14:21:43 -0700 git.webkit.org repository is missing svn revision r49890 2010-03-19 20:12:11 -0700 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) All All RESOLVED FIXED P2 Normal --- 1 bolsinga webkit-unassigned abecsi cjerdonek ddkilzer dpranke gustavo hamaji mrowe simon.fraser wsiegrist oldest_to_newest 200433 0 bolsinga 2010-03-16 14:21:43 -0700 Here's my git config: sonic:WebKit bolsinga$ cat .git/config [core] repositoryformatversion = 0 filemode = true bare = false logallrefupdates = true editor = ./WebKitTools/Scripts/commit-log-editor autocrlf = false [remote "origin"] url = git://git.webkit.org/WebKit.git fetch = +refs/heads/*:refs/remotes/origin/* [branch "master"] remote = origin merge = refs/heads/master [svn-remote "svn"] url = http://svn.webkit.org/repository/webkit fetch = trunk:refs/remotes/trunk When I git svn rebase, I see: First, rewinding head to replay your work on top of it... Applying: 2009-10-20 Mikhail Naganov <mnaganov@chromium.org> error: patch failed: WebCore/ChangeLog:1 error: WebCore/ChangeLog: patch does not apply error: patch failed: WebCore/bindings/v8/V8Binding.h:61 error: WebCore/bindings/v8/V8Binding.h: patch does not apply error: patch failed: WebCore/inspector/front-end/BottomUpProfileDataGridTree.js:29 error: WebCore/inspector/front-end/BottomUpProfileDataGridTree.js: patch does not apply error: patch failed: WebCore/inspector/front-end/ProfileDataGridTree.js:126 error: WebCore/inspector/front-end/ProfileDataGridTree.js: patch does not apply error: patch failed: WebCore/inspector/front-end/TopDownProfileDataGridTree.js:33 error: WebCore/inspector/front-end/TopDownProfileDataGridTree.js: patch does not apply Using index info to reconstruct a base tree... Falling back to patching base and 3-way merge... Auto-merging WebCore/ChangeLog CONFLICT (content): Merge conflict in WebCore/ChangeLog Auto-merging WebCore/bindings/v8/V8Binding.h CONFLICT (content): Merge conflict in WebCore/bindings/v8/V8Binding.h Auto-merging WebCore/inspector/front-end/BottomUpProfileDataGridTree.js CONFLICT (content): Merge conflict in WebCore/inspector/front-end/BottomUpProfileDataGridTree.js Auto-merging WebCore/inspector/front-end/ProfileDataGridTree.js Auto-merging WebCore/inspector/front-end/TopDownProfileDataGridTree.js CONFLICT (content): Merge conflict in WebCore/inspector/front-end/TopDownProfileDataGridTree.js Failed to merge in the changes. Patch failed at 0001 2009-10-20 Mikhail Naganov <mnaganov@chromium.org> When you have resolved this problem run "git rebase --continue". If you would prefer to skip this patch, instead run "git rebase --skip". To restore the original branch and stop rebasing run "git rebase --abort". rebase refs/remotes/trunk: command returned error: 1 David Kilzer mentioned there was a missing svn commit. He said it was r49890. It is in remote/trunk, but not remotes/origin/master. FWIW, the svn commit above is r50141. 200437 1 ddkilzer 2010-03-16 14:25:38 -0700 I will send a message to webkit-dev to ask how developers want to handle this issue. 200441 2 bolsinga 2010-03-16 14:27:30 -0700 If I: sonic:WebKit bolsinga$ git checkout -b NewMaster remotes/trunk sonic:WebKit bolsinga$ git svn rebase Things are good. 200445 3 ddkilzer 2010-03-16 14:35:54 -0700 I'm in favor of fixing git.webkit.org, even though it will cause issues when pulling from the repository again. (Maybe we could move master to an old-master branch so the revisions are still there, but then fix master to include the missing revision again.) 200447 4 ddkilzer 2010-03-16 14:36:54 -0700 (In reply to comment #2) > If I: > > sonic:WebKit bolsinga$ git checkout -b NewMaster remotes/trunk > sonic:WebKit bolsinga$ git svn rebase > > Things are good. This will break the next time you do a git-pull from git.webkit.org again. (It will force a merge, I think, but I haven't tried it.) 200491 5 dpranke 2010-03-16 16:07:47 -0700 +1 to fixing the repo; I'm not sure how having a git repo that is different from the svn repo would be acceptable? 200518 6 simon.fraser 2010-03-16 17:05:52 -0700 We should fix the git repo. 200689 7 cjerdonek 2010-03-17 04:51:11 -0700 (In reply to comment #3) > I'm in favor of fixing git.webkit.org, even though it will cause issues when > pulling from the repository again. I support fixing it even though I don't fully understand the down-side for doing so. Will pulling for the first time after the fix be a time-intensive interruption for users (especially if we have many branches)? 200759 8 gustavo 2010-03-17 07:15:11 -0700 (In reply to comment #7) > (In reply to comment #3) > > I'm in favor of fixing git.webkit.org, even though it will cause issues when > > pulling from the repository again. > > I support fixing it even though I don't fully understand the down-side for > doing so. Will pulling for the first time after the fix be a time-intensive > interruption for users (especially if we have many branches)? The downside is that trees that use the git repository as a base will all be broken, and their current state will have to be recreated somehow. For instance, Debian uses the git repository to maintain webkitgtk packages, and we gtk+ porters maintain a git repository to track our "stable" branch. It'll be a pain for me if this is done, but I guess we'll survive. 200784 9 ddkilzer 2010-03-17 07:59:48 -0700 (In reply to comment #8) > The downside is that trees that use the git repository as a base will all be > broken, and their current state will have to be recreated somehow. For > instance, Debian uses the git repository to maintain webkitgtk packages, and we > gtk+ porters maintain a git repository to track our "stable" branch. > > It'll be a pain for me if this is done, but I guess we'll survive. I'm planning on moving the current master to "broken-master", doing a hard reset of master back before the missing commit, then letting it catch up to svn trunk again. The broken-master branch would not be updated after it was made. I'll send a warning on IRC before making this change. On the client side, you'd also want to create a local "broken-master" branch (maybe connecting it via remote tracking branch), do a hard reset of your master branch to before the missing commit, then re-pull from git.webkit.org. After that comes the hard part of deciding whether to migrate existing branches from broken-master to master, or just leaving them as-is. Please update this bug with any helpful techniques you find while doing this. 200862 10 cjerdonek 2010-03-17 10:56:14 -0700 (In reply to comment #9) > I'm planning on moving the current master to "broken-master", doing a hard > reset of master back before the missing commit, then letting it catch up to svn > trunk again. The broken-master branch would not be updated after it was made. > I'll send a warning on IRC before making this change. Can you also comment on this report when it is done catching up? Thanks. 200989 11 ddkilzer 2010-03-17 15:14:41 -0700 (In reply to comment #10) > (In reply to comment #9) > > I'm planning on moving the current master to "broken-master", doing a hard > > reset of master back before the missing commit, then letting it catch up to svn > > trunk again. The broken-master branch would not be updated after it was made. > > I'll send a warning on IRC before making this change. > > Can you also comment on this report when it is done catching up? Thanks. Sure. Waiting to coordinate with Bill on the update. 201221 12 ddkilzer 2010-03-18 01:46:17 -0700 Bill and I will probably start working on this between 7 and 8 AM PDT (GMT-0700) today. In the meantime, it appears git.webkit.org has developed another issue: $ git pull origin && git svn fetch remote: error: failed to unpack compressed delta at offset 167772124 from ./objects/pack/pack-48166dba7808af6c8cbc7b74dada2c456e2ba411.pack remote: error: failed to read object be61563cc282b7c036d656ec05df6d5d66be4be7 at offset 167772121 from ./objects/pack/pack-48166dba7808af6c8cbc7b74dada2c456e2ba411.pack remote: fatal: object be61563cc282b7c036d656ec05df6d5d66be4be7 is corrupted remote: aborting due to possible repository corruption on the remote side. fatal: protocol error: bad pack header $ git show be61563cc282b7c036d656ec05df6d5d66be4be7 tree be61563cc282b7c036d656ec05df6d5d66be4be7 Mac-compatible-font-fallback.css run-webkit-tests-epilogue.html run-webkit-tests-prologue.html 201331 13 ddkilzer 2010-03-18 09:16:22 -0700 (In reply to comment #11) > Sure. Waiting to coordinate with Bill on the update. We've started the process of fixing the history on the master. When it's complete (in 1-2 hours), Bill will push a repaired git repository with a new master branch, and the old (broken) master branch renamed to "broken-master". (He or I will post a comment here when that's done.) If you have pulled from the broken master previously, you'll want to do the following: 1. Rename master to broken-master: $ git branch -m master broken-master 2. Create a new master starting at r49889 (the commit before the missing r49890 commit): $ git checkout -b master 81e1876cbf3bc239eca66e1a48367b9b277d66e1 3. Run your usual git-pull or git-fetch command to pull the new master branch down. After that, it's up to you whether you want to rebase your local branches against the new master branch or leave them based on broken-master. -- While we were working on this, we came up with a possible cause and fix for the occasional corruption that happens (see Comment #12) when the git.webkit.org repository is updated (via git-push using an svn post-commit hook). Time will tell if we've fixed the issue. 201367 14 cjerdonek 2010-03-18 10:13:41 -0700 (In reply to comment #13) > After that, it's up to you whether you want to rebase your local branches > against the new master branch or leave them based on broken-master. Thanks, David. To be explicit, I assume the following command would work for each topic branch we want to rebase (assuming the difference between topic-branch and broken-master consists only of our local changes)? > git rebase --onto master broken-master topic-branch 201428 15 ddkilzer 2010-03-18 11:29:40 -0700 (In reply to comment #14) > (In reply to comment #13) > > After that, it's up to you whether you want to rebase your local branches > > against the new master branch or leave them based on broken-master. > > Thanks, David. To be explicit, I assume the following command would work for > each topic branch we want to rebase (assuming the difference between > topic-branch and broken-master consists only of our local changes)? > > > git rebase --onto master broken-master topic-branch The fixed repository is available on git.webkit.org. I will test some of my local topic branches to see if the above command works as expected. 201438 16 ddkilzer 2010-03-18 11:36:43 -0700 (In reply to comment #15) > (In reply to comment #14) > > (In reply to comment #13) > > > After that, it's up to you whether you want to rebase your local branches > > > against the new master branch or leave them based on broken-master. > > > > Thanks, David. To be explicit, I assume the following command would work for > > each topic branch we want to rebase (assuming the difference between > > topic-branch and broken-master consists only of our local changes)? > > > > > git rebase --onto master broken-master topic-branch > > The fixed repository is available on git.webkit.org. The corruption is still present. Bill is going to repush the repository. 201458 17 ddkilzer 2010-03-18 12:18:09 -0700 (In reply to comment #13) > If you have pulled from the broken master previously, you'll want to do the > following: > > 1. Rename master to broken-master: > $ git branch -m master broken-master > > 2. Create a new master starting at r49889 (the commit before the missing r49890 > commit): > $ git checkout -b master 81e1876cbf3bc239eca66e1a48367b9b277d66e1 2.5. Edit .git/config to change references to "broken-master" back to master, e.g.: [branch "master"] remote = origin merge = refs/heads/master > 3. Run your usual git-pull or git-fetch command to pull the new master branch > down. I ran "git pull origin" and it updated the master branch as expected. More feedback to follow. 201464 18 ddkilzer 2010-03-18 12:26:56 -0700 (In reply to comment #14) > Thanks, David. To be explicit, I assume the following command would work for > each topic branch we want to rebase (assuming the difference between > topic-branch and broken-master consists only of our local changes)? > > git rebase --onto master broken-master topic-branch This is exactly how you want to merge topic branches from broken-master to master. Works like a dream, especially with resolve-ChangeLogs as the merge driver. :) 201488 19 ddkilzer 2010-03-18 12:55:38 -0700 Hopefully no one else did this, but I actually reset my svn history in my git repository so that it would match git.webkit.org. (I did this by editing a ref file, deleting the git-svn rev map, then doing a git-svn-fetch to repopulate the rev map.) In case you did that, too, here's how to undo it. NOTE: It may be quicker just to re-clone the git.webkit.org repository and follow the instructions here: <http://trac.webkit.org/wiki/UsingGitWithWebKit#Checkout> 1. Look at your .git/config file to determine where the tracking information is stored for the svn repository: [svn-remote "svn"] url = http://svn.webkit.org/repository/webkit fetch = trunk:refs/remotes/trunk 2. Remove the svn tracking directory (git will rebuild this on the next svn fetch). Mine is "trunk" because that's what it's called in the svn-remote section above: $ rm -rf .git/svn/trunk 3. If .git/refs/remotes/trunk exists, edit it and change its head commit back to r49889 (81e1876cbf3bc239eca66e1a48367b9b277d66e1). If .git/info/refs exist, edit it there instead. OPTIONAL: Instead of using the commitish for r49889, you could use the latest commitish on master. 4. Run "git svn fetch" to recreate the rev map and update back to trunk again. NOTE: I edited other entries in .git/info/refs, created .git/refs/remotes/trunk (pasted the commitish at the head of master), and edited .git/refs/remotes/origin/HEAD and .git/refs/remotes/origin/master to contain master's latest commitish. Not sure if all of that was necessary, but it didn't hurt. 201491 20 ddkilzer 2010-03-18 12:56:47 -0700 Marking this as RESOLVED/FIXED, but please continue to add comments as necessary. 201837 21 cjerdonek 2010-03-19 06:41:09 -0700 Here is a bit of info from my experience. To see whether your branch has the commit for r49890, you can use any of the following techniques: git log --before=2009-10-21 (and page down) git log 3d3c0aff8a1d6bb6aca0dbe273d0559f870a9886 (everything <= 49891) git log --grep="49890" (just the commit itself) To see if your git-svn history has it, you can use: git svn log -r 49889:49891 I ran into the following unusual behavior after running "git pull origin" in comment 17. For some reason, every git-svn command became extremely slow for me. The git-svn commands I was trying would give no output and then finish after on the order of 5 minutes. Even something as simple as git-svn-log would take this long. The process would consume around 40% of my machine's CPU during this time. This made it difficult to figure out what was wrong! My git-svn history was missing r49890 during this time. This is probably because I set up my git checkout for the first time back in January, using the instructions here: http://trac.webkit.org/wiki/UsingGitWithWebKit#Checkout (January 2010 was after the missing commit in October 2009.) Those suggestions say to "re-use the history that we've already cloned from git.webkit.org [which must have been missing the commit] rather than trying to fetch it all from Subversion" and then to use the SVN repository for subsequent commits. Maybe the slowness was because git-svn was trying to compute the entire SVN history on the fly every time or something because it was out of synch. Running "rm -rf .git/svn" did not speed it up. What finally sped it up was running-- git svn reset -r 49889 This command may be a substitute for the manual edits suggested in comment 19. Then, after running "git svn fetch" (which took quite a while), I was back to normal. 201844 22 gustavo 2010-03-19 07:10:10 -0700 I think it's important to send email to webkit-dev telling people what they should do, as reading through all comments here will be error prone =). 202152 23 ddkilzer 2010-03-19 20:12:11 -0700 (In reply to comment #22) > I think it's important to send email to webkit-dev telling people what they > should do, as reading through all comments here will be error prone =). The wiki would be the best place to put the directions, but I don't have a feel for how many people are still having issues. I don't want to spend time writing up instructions for a one-time fix if no one is going to use them.