Bug 88586 - rebaseline from garden-o-matic leaves N processes each time it is run
Summary: rebaseline from garden-o-matic leaves N processes each time it is run
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Tony Chang
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-06-07 15:47 PDT by Tony Chang
Modified: 2012-06-11 10:40 PDT (History)
5 users (show)

See Also:


Attachments
Patch (2.87 KB, patch)
2012-06-08 12:24 PDT, Tony Chang
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ec2-cr-linux-04 (699.89 KB, application/zip)
2012-06-08 17:03 PDT, WebKit Review Bot
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Tony Chang 2012-06-07 15:47:18 PDT
I get 16 left over python processes each time I run Rebaseline from garden-o-matic.  It looks like we never close our multiprocess.Pools in run_in_parallel.

http://docs.python.org/library/multiprocessing.html#multiprocessing.pool.multiprocessing.Pool.close

Changing run_in_parallel to:

        pool = multiprocessing.Pool(processes=processes)
        results = pool.map(_run_command_thunk, command_lines_and_cwds)
        pool.close()
        return results

Seems to work, but I get an exception for each command:

Exception RuntimeError: RuntimeError('cannot join current thread',) in <Finalize object, dead> ignored
Comment 1 Dirk Pranke 2012-06-07 16:06:02 PDT
oh, I didn't know about close(). I thought map() effectively auto-closed.

Does calling pool.join() after pool.close() fix your terminate issue?
Comment 2 Tony Chang 2012-06-07 16:22:32 PDT
(In reply to comment #1)
> oh, I didn't know about close(). I thought map() effectively auto-closed.
> 
> Does calling pool.join() after pool.close() fix your terminate issue?

Yes, that seems to work.
Comment 3 Tony Chang 2012-06-08 12:24:36 PDT
Created attachment 146628 [details]
Patch
Comment 4 Dirk Pranke 2012-06-08 12:34:26 PDT
Comment on attachment 146628 [details]
Patch

Thanks! I was wondering how to test this ... I didn't know there was a multiprocessing.active_children(), either.
Comment 5 WebKit Review Bot 2012-06-08 17:03:53 PDT
Comment on attachment 146628 [details]
Patch

Attachment 146628 [details] did not pass chromium-ews (chromium-xvfb):
Output: http://queues.webkit.org/results/12919635

New failing tests:
svg/text/text-vkern.svg
svg/custom/use-forward-refs.svg
svg/zoom/page/absolute-sized-document-no-scrollbars.svg
svg/clip-path/clipper-placement-issue.svg
svg/custom/gradient-stop-corner-cases.svg
svg/zoom/page/relative-sized-document-scrollbars.svg
svg/text/text-hkern-on-vertical-text.svg
svg/zoom/page/zoom-svg-as-relative-image.html
svg/zoom/page/zoom-background-image-tiled.html
svg/custom/svgpolyparser-extra-space.svg
svg/text/text-hkern.svg
svg/custom/dynamic-empty-path.svg
svg/custom/broken-internal-references.svg
svg/css/arrow-with-shadow.svg
svg/css/background-image-svg.html
svg/custom/use-on-clip-path-with-transformation.svg
svg/text/text-vkern-on-horizontal-text.svg
Comment 6 WebKit Review Bot 2012-06-08 17:03:57 PDT
Created attachment 146665 [details]
Archive of layout-test-results from ec2-cr-linux-04

The attached test failures were seen while running run-webkit-tests on the chromium-ews.
Bot: ec2-cr-linux-04  Port: <class 'webkitpy.common.config.ports.ChromiumXVFBPort'>  Platform: Linux-2.6.35-28-virtual-x86_64-with-Ubuntu-10.10-maverick
Comment 7 WebKit Review Bot 2012-06-11 10:39:59 PDT
Comment on attachment 146628 [details]
Patch

Clearing flags on attachment: 146628

Committed r119986: <http://trac.webkit.org/changeset/119986>
Comment 8 WebKit Review Bot 2012-06-11 10:40:04 PDT
All reviewed patches have been landed.  Closing bug.