inspired by bug141946 Sometimes the repository of the bots has conflicts (who knows how is it possible), sometimes untracked orphaned files cause issues. I think Tools/BuildSlaveSupport/clean-build should solve this issues too.
See Bug 141946 Comment #25 and Bug 142083 Comment #19 for how to clean up these two particular issues. In summary, we probably want to write a script that runs 'svn revert --recursive' on the entire tree, then does an 'svn stat' to find "orphaned" files/directories with status "?", then remove those. Or we could switch the bots to check out from git.webkit.org, then use git-revert and git-clean without having to write a separate script, and get the added benefit of faster updates from trunk/master.
(In reply to comment #1) > See Bug 141946 Comment #25 and Bug 142083 Comment #19 for how to clean up > these two particular issues. > > In summary, we probably want to write a script that runs 'svn revert > --recursive' on the entire tree, then does an 'svn stat' to find "orphaned" > files/directories with status "?", then remove those. Sounds good. > Or we could switch the bots to check out from git.webkit.org, then use > git-revert and git-clean without having to write a separate script, and get > the added benefit of faster updates from trunk/master. With git we would loose the human readable svn revision number, which is undesirable. And git would be overkiller for the ARM bots.
Or a third, but longer term option: - switch to server side SVN step: bug90075 - use it for proper fresh build on demand: bug90075 We can do it once all slaves upgraded to at least 0.8.5, but it seems there are many slaves still use very old buildslave version: https://build.webkit.org/buildslaves - Apple Mavericks bots: 0.8.3 - EFL ARM bots: 0.8.3 (not an issue, I can easiy upgrade them) - GTK bots: 0.7.12
Found some old (didn't check file dates) files that had been sitting in the svn tree on some Mavericks internal bots as well (that should have been cleaned up): $ svn stat . D C Source/WebKit2/Scripts/webkit2 > local unversioned, incoming add upon update D Source/WebKit2/Scripts/webkit2/LegacyMessageReceiver-expected.cpp D Source/WebKit2/Scripts/webkit2/LegacyMessages-expected.h D Source/WebKit2/Scripts/webkit2/MessageReceiver-expected.cpp D Source/WebKit2/Scripts/webkit2/MessageReceiverSuperclass-expected.cpp D Source/WebKit2/Scripts/webkit2/Messages-expected.h D Source/WebKit2/Scripts/webkit2/MessagesSuperclass-expected.h D Source/WebKit2/Scripts/webkit2/__init__.py D Source/WebKit2/Scripts/webkit2/messages.py D Source/WebKit2/Scripts/webkit2/messages_unittest.py D Source/WebKit2/Scripts/webkit2/model.py D Source/WebKit2/Scripts/webkit2/parser.py D Source/WebKit2/Scripts/webkit2/test-legacy-messages.in D Source/WebKit2/Scripts/webkit2/test-messages.in D Source/WebKit2/Scripts/webkit2/test-superclass-messages.in ? layout-test-results ? layout-test-results.zip Summary of conflicts: Tree conflicts: 1
(In reply to comment #2) > (In reply to comment #1) > > See Bug 141946 Comment #25 and Bug 142083 Comment #19 for how to clean up > > these two particular issues. > > > > In summary, we probably want to write a script that runs 'svn revert > > --recursive' on the entire tree, then does an 'svn stat' to find "orphaned" > > files/directories with status "?", then remove those. > > Sounds good. > > > Or we could switch the bots to check out from git.webkit.org, then use > > git-revert and git-clean without having to write a separate script, and get > > the added benefit of faster updates from trunk/master. > > With git we would loose the human readable svn revision number, > which is undesirable. And git would be overkiller for the ARM > bots. No, I'm talking about using the existing git-svn repository which still contains all the svn revision number happiness, and none of the svn performance issues when checking out WebKit from scratch.
Cleaning up git-svn checkouts seems like a much bigger trouble than cleaning up svn FWIW - just look at EWS configuration, which is super tricky. That said, we seem to have it working well enough on EWS now.
(In reply to comment #5) > > With git we would loose the human readable svn revision number, > > which is undesirable. And git would be overkiller for the ARM > > bots. > > No, I'm talking about using the existing git-svn repository which still > contains all the svn revision number happiness, and none of the svn > performance issues when checking out WebKit from scratch. Not at all, buildbot can't extract svn revision number. If we switched to git, we would loose svn revision number on build.webkit.org which would break flakiness dashboard, WebKit perf monitor and bot watcher's dashboard too. What kind of SVN performance issue? A new checkout/clone from sratch is much more faster with SVN than GIT and consumes much less CPU time.
Just to have some statistics, how fast/slow are svn and git Cloning from official Apple servers: ------------------------------------- $ time git clone git://git.webkit.org/WebKit.git real 28m41.117s ---> 5.4x slower than svn user 27m23.484s ---> 20.5x more consumed CPU time than svn sys 2m14.190s ---> 3.2x more consumed kernel time than svn $ time svn checkout http://svn.webkit.org/repository/webkit/trunk WebKit real 5m4.963s user 1m20.516s sys 0m42.239s Cloning from our internal mirrors: ----------------------------------- $ time git clone git://<our-internal-git-mirror> real 11m52.672s ---> 6.9x slower than svn user 26m45.524s ---> 25.1x more consumed CPU time than svn sys 2m10.426s ---> 3.6x more consumed kernel time than svn $ time svn checkout svn://<our-internal-svn-mirror>/webkit/trunk WebKit real 1m43.370s user 1m4.520s sys 0m36.122s
(In reply to comment #8) > Just to have some statistics, how fast/slow are svn and git > > Cloning from official Apple servers: > ------------------------------------- > $ time git clone git://git.webkit.org/WebKit.git > real 28m41.117s ---> 5.4x slower than svn > user 27m23.484s ---> 20.5x more consumed CPU time than svn > sys 2m14.190s ---> 3.2x more consumed kernel time than svn > > $ time svn checkout http://svn.webkit.org/repository/webkit/trunk WebKit > real 5m4.963s > user 1m20.516s > sys 0m42.239s > > Cloning from our internal mirrors: > ----------------------------------- > $ time git clone git://<our-internal-git-mirror> > real 11m52.672s ---> 6.9x slower than svn > user 26m45.524s ---> 25.1x more consumed CPU time than svn > sys 2m10.426s ---> 3.6x more consumed kernel time than svn > > $ time svn checkout svn://<our-internal-svn-mirror>/webkit/trunk WebKit > real 1m43.370s > user 1m4.520s > sys 0m36.122s Do EWS bots need the full git history? What about --depth=1?
Created attachment 249118 [details] WIP
(In reply to comment #10) > Created attachment 249118 [details] > WIP Patch seems to work on my linux env, both SVN and GIT. Not sure how it is supposed to get tested though. Any feedback appreciated.
Comment on attachment 249118 [details] WIP The patch looks great!
> $ time svn checkout http://svn.webkit.org/repository/webkit/trunk WebKit > real 5m4.963s Is this on Linux or Mac? What is your svn client version? It's way faster than I expected it to be, especially from Europe.
(In reply to comment #13) > > $ time svn checkout http://svn.webkit.org/repository/webkit/trunk WebKit > > real 5m4.963s > > Is this on Linux or Mac? What is your svn client version? > > It's way faster than I expected it to be, especially from Europe. It is Linux, Ubuntu 14.04 with the stock SVN 1.8.8.
Created attachment 249226 [details] Fixing difference between SVN and GIT for folders and adding unit test
(In reply to comment #15) > Created attachment 249226 [details] > Fixing difference between SVN and GIT for folders and adding unit test While adding unit test, it appears that many SCM unit tests are broken on my env. No assertion failures but errors.
Comment on attachment 249226 [details] Fixing difference between SVN and GIT for folders and adding unit test Clearing flags on attachment: 249226 Committed r182119: <http://trac.webkit.org/changeset/182119>
All reviewed patches have been landed. Closing bug.
Re-opened since this is blocked by bug 143196
Created attachment 249692 [details] Fixing syntax
Created attachment 249693 [details] Fixing syntax
Comment on attachment 249693 [details] Fixing syntax Rejecting attachment 249693 [details] from commit-queue. Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.appspot.com', '--bot-id=webkit-cq-02', 'validate-changelog', '--check-oops', '--non-interactive', 249693, '--port=mac']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit ChangeLog entry in Tools/ChangeLog is not at the top of the file. Full output: http://webkit-queues.appspot.com/results/6541139016417280
Created attachment 249714 [details] Rebasing according 143135 and testing whether WebKitBuild dir exists before deleting it
Attachment 249714 [details] did not pass style-queue: ERROR: Tools/Scripts/webkitpy/common/checkout/scm/scm.py:208: [SCM.discard_untracked_files] Instance of 'SCM' has no 'untracked_files' member [pylint/E1101] [5] Total errors found: 1 in 5 files If any of these errors are false positives, please file a bug against check-webkit-style.
Created attachment 249803 [details] Landing with additional test
Comment on attachment 249803 [details] Landing with additional test Clearing flags on attachment: 249803 Committed r182177: <http://trac.webkit.org/changeset/182177>
Does this script run on builders and/or testers? It doesn't seem like it does.
(In reply to comment #28) > Does this script run on builders and/or testers? It doesn't seem like it > does. AFAIUI, this script is skipped by default as it removes WebKitBuild folder. I triggered this script when forcing a clean build on two builders (gtk and apple) this morning and it runned fine in both cases.
I see. Maybe we need something that really cleans up everything other than WebKitBuild (and maybe WebKitLibraries), and that runs every time.
(In reply to comment #30) > I see. Maybe we need something that really cleans up everything other than > WebKitBuild (and maybe WebKitLibraries), and that runs every time. Maybe we can cleanup Git/SVN state and remove untracked files?
(In reply to comment #31) > (In reply to comment #30) > > I see. Maybe we need something that really cleans up everything other than > > WebKitBuild (and maybe WebKitLibraries), and that runs every time. > > Maybe we can cleanup Git/SVN state and remove untracked files? Removing untracked files would mean removing dynamically downloaded stuff like python modules, wpt modules, all pyc... and also WebKitBuild. Even if we skip WebKitBuild removal, this may be too expensive to do that for all patches. We could trigger that clean-up whenever applying patches fail. Is there any other case where that would be useful?
We had cases where build failed because of stale files, yes.
Re-opened since this is blocked by bug 143314
(In reply to comment #34) > Re-opened since this is blocked by bug 143314 This patch can be landed again. I plan to do it tomorrow.
Comment on attachment 249803 [details] Landing with additional test Clearing flags on attachment: 249803 Committed r194696: <http://trac.webkit.org/changeset/194696>