Summary: | commit-queue keeps crashing | ||||||||
---|---|---|---|---|---|---|---|---|---|
Product: | WebKit | Reporter: | Eric Seidel (no email) <eric> | ||||||
Component: | Tools / Tests | Assignee: | Nobody <webkit-unassigned> | ||||||
Status: | RESOLVED DUPLICATE | ||||||||
Severity: | Normal | CC: | abarth, evan, levin, steveblock | ||||||
Priority: | P2 | ||||||||
Version: | 528+ (Nightly build) | ||||||||
Hardware: | PC | ||||||||
OS: | OS X 10.5 | ||||||||
Attachments: |
|
Description
Eric Seidel (no email)
2009-10-28 09:55:08 PDT
This appears to be a python bug. I've made a reduced test case. Created attachment 42122 [details]
Script demonstrating python 2.5.1 (or mac os x?) bug
On Linux (the "import site" bits are when I'm repeatedly hitting ctl-C). Python 2.5.2 % ./t ...............................................................................'import site' failed; use -v for traceback .........'import site' failed; use -v for traceback ............'import site' failed; use -v for traceback ...........'import site' failed; use -v for traceback ....................'import site' failed; use -v for traceback .......... Bah. I just don't know how exec works. I didn't realize it carried file handles through. Created attachment 42123 [details]
Patch v1
Comment on attachment 42123 [details]
Patch v1
File handle leaks = bad.
Comment on attachment 42123 [details]
Patch v1
Going to make a better patch.
It would appear that even with my fix, Python itself is still leaking 3 file descriptors on every exec(): Python 57958 eseidel 58r REG 14,2 144580 466329 /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/Resources/HITo olbox.rsrc Python 57958 eseidel 59r REG 14,2 490410 9627854 /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/Resources/Engl ish.lproj/Localized.rsrc Python 57958 eseidel 60r REG 14,2 86770 481232 /System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac/errors.rsrc.df.rsrc I expect this may be a mac-only problem. I wrote a function: def check_for_file_leak(message): log("before %s" % message) lsof_output = SCM.run_command(['lsof', '-p', str(os.getpid())]) if lsof_output.count('HIToolbox'): log(message) error(lsof_output) log("after %s" % message) and sprinkled it throughout the code. It seems these 3 leaking files are opened by mechanize during: self.browser = Browser() in Bugzilla.__init__() I've not tried to trace through the mechanize code to see where the leaks come from. Why don't we fork and exec and then throw away the old version of ourselves? If that doesn't work, can we use a trampoline executable? Tracking down every leak seems fragile. Here is a stand-alone version of the same function: def check_for_file_leak(message): import os import sys import subprocess print >> sys.stderr, "before %s" % message process = subprocess.Popen(['lsof', '-p', str(os.getpid())], stdout=subprocess.PIPE, stderr=subprocess.STDOUT) lsof_output = process.communicate()[0].rstrip() if lsof_output.count('HIToolbox'): print >> sys.stderr, message print >> sys.stderr, lsof_output exit(1) print >> sys.stderr, "after %s" % message We could definitely use a trampoline executable. That may be the simplest solution. bugzilla-tool commit-queue is already a trampoline executable of sorts. It's just no longer a shell script, instead it's implemented in bugzilla-tool itself. I think I might just get rid of the "don't need to restart bugzilla-tool to notice committers.py changes" feature for now. |