rebaseline from garden-o-matic leaves N processes each time it is run

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Tony Chang]

I get 16 left over python processes each time I run Rebaseline from garden-o-matic.  It looks like we never close our multiprocess.Pools in run_in_parallel.

http://docs.python.org/library/multiprocessing.html#multiprocessing.pool.multiprocessing.Pool.close

Changing run_in_parallel to:

        pool = multiprocessing.Pool(processes=processes)
        results = pool.map(_run_command_thunk, command_lines_and_cwds)
        pool.close()
        return results

Seems to work, but I get an exception for each command:

Exception RuntimeError: RuntimeError('cannot join current thread',) in <Finalize object, dead> ignored

[Dirk Pranke]

oh, I didn't know about close(). I thought map() effectively auto-closed.

Does calling pool.join() after pool.close() fix your terminate issue?

[Tony Chang]

(In reply to comment #1)
> oh, I didn't know about close(). I thought map() effectively auto-closed.
> 
> Does calling pool.join() after pool.close() fix your terminate issue?

Yes, that seems to work.

[Tony Chang]

Created attachment 146628 [details]
Patch

[Dirk Pranke]

Comment on attachment 146628 [details]
Patch

Thanks! I was wondering how to test this ... I didn't know there was a multiprocessing.active_children(), either.

[WebKit Review Bot]

Comment on attachment 146628 [details]
Patch

Attachment 146628 [details] did not pass chromium-ews (chromium-xvfb):
Output: http://queues.webkit.org/results/12919635

New failing tests:
svg/text/text-vkern.svg
svg/custom/use-forward-refs.svg
svg/zoom/page/absolute-sized-document-no-scrollbars.svg
svg/clip-path/clipper-placement-issue.svg
svg/custom/gradient-stop-corner-cases.svg
svg/zoom/page/relative-sized-document-scrollbars.svg
svg/text/text-hkern-on-vertical-text.svg
svg/zoom/page/zoom-svg-as-relative-image.html
svg/zoom/page/zoom-background-image-tiled.html
svg/custom/svgpolyparser-extra-space.svg
svg/text/text-hkern.svg
svg/custom/dynamic-empty-path.svg
svg/custom/broken-internal-references.svg
svg/css/arrow-with-shadow.svg
svg/css/background-image-svg.html
svg/custom/use-on-clip-path-with-transformation.svg
svg/text/text-vkern-on-horizontal-text.svg

[WebKit Review Bot]

Created attachment 146665 [details]
Archive of layout-test-results from ec2-cr-linux-04

The attached test failures were seen while running run-webkit-tests on the chromium-ews.
Bot: ec2-cr-linux-04  Port: <class 'webkitpy.common.config.ports.ChromiumXVFBPort'>  Platform: Linux-2.6.35-28-virtual-x86_64-with-Ubuntu-10.10-maverick

[WebKit Review Bot]

Comment on attachment 146628 [details]
Patch

Clearing flags on attachment: 146628

Committed r119986: <http://trac.webkit.org/changeset/119986>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.