205280 2019-12-16 09:59:43 -0800 Python 3: Multiprocessing doesn't do well when nested 2019-12-16 15:41:32 -0800 1 1 1 Unclassified WebKit Tools / Tests WebKit Nightly Build Unspecified Unspecified NEW https://bugs.webkit.org/show_bug.cgi?id=205291 https://bugs.webkit.org/show_bug.cgi?id=184986 P2 Normal --- 1 jbedard webkit-unassigned ap oldest_to_newest 1598940 0 jbedard 2019-12-16 09:59:43 -0800 Not quite clear why. We end up with some weird race conditions, I spent a few hours trying to get to the bottom of it with no luck. I can reproduce the problem except in unit testing, so I'm inclined to defer investigation since run-webkit-tests works fine in Python 3 despite this bug. 1598984 1 ap 2019-12-16 11:06:40 -0800 Which start method are we using with Python 3? 1598990 2 jbedard 2019-12-16 11:13:04 -0800 (In reply to Alexey Proskuryakov from comment #1) > Which start method are we using with Python 3? Not sure what you mean by this. For testing, I actually changed the shebang, but the code to run enable run-webkit-tests in Python3 isn't up for review quite yet, polishing that patch now. I'll be referencing this bug in that patch. 1599160 3 ap 2019-12-16 15:28:22 -0800 In Python 3, one can choose the underlying method for how multiprocessing works, it's called "start method". See <https://docs.python.org/3/library/multiprocessing.html>. As this bug is about multiprocessing having some strange behavior on tests with Python 3, it's relevant which of the methods you are using. I'm guessing that it is 'spawn' because that's the default. Since 'fork' was the only one that Python 2 supported, that would be the 1:1 replacement, but of course we shouldn't be using that on macOS. 1599176 4 jbedard 2019-12-16 15:41:32 -0800 (In reply to Alexey Proskuryakov from comment #3) > In Python 3, one can choose the underlying method for how multiprocessing > works, it's called "start method". See > <https://docs.python.org/3/library/multiprocessing.html>. > > As this bug is about multiprocessing having some strange behavior on tests > with Python 3, it's relevant which of the methods you are using. I'm > guessing that it is 'spawn' because that's the default. Since 'fork' was the > only one that Python 2 supported, that would be the 1:1 replacement, but of > course we shouldn't be using that on macOS. I wasn't aware of this. I haven't modified the start method used, so yes, it should be using spawn. Seems possible that the spawn/fork difference is the root of our problem, but I'm still at a loss as to why I saw a race condition.