Bug 133397 – jsc can make a successful exit even after we've already started crashing and we've started displaying the stack dump

• Component: JavaScriptCore
• Status: NEW

[Filip Pizlo]

That's a bit weird.  Probably the solution is to have all of the WTFCrash() thingies flag that we're experiencing badness so that jsc.cpp will know not to exit with a success code.  Of course that won't cover all modes of testing but it will make testing with JSC a bit more sane.

[Mark Lam]

Is there a report step for this?

[Filip Pizlo]

(In reply to comment #1)
> Is there a report step for this?

What do you mean?

[Mark Lam]

(In reply to comment #2)
> (In reply to comment #1)
> > Is there a report step for this?
> 
> What do you mean?

I meant what were you running when you observed this issue?

[Filip Pizlo]

(In reply to comment #3)
> (In reply to comment #2)
> > (In reply to comment #1)
> > > Is there a report step for this?
> > 
> > What do you mean?
> 
> I meant what were you running when you observed this issue?

I was running tests with local changes that led to crashes.

[Alexey Proskuryakov]

Is there something special about jsc, or are all Darwin command line tools expected to have this problem?

WTFCrash just calls __builtin_trap(), unless there is something wrong done via WTFSetCrashHook(), or WTFReportBacktrace() does an exit() of its own somehow.

[Filip Pizlo]

(In reply to comment #5)
> Is there something special about jsc, or are all Darwin command line tools expected to have this problem?
> 
> WTFCrash just calls __builtin_trap(), unless there is something wrong done via WTFSetCrashHook(), or WTFReportBacktrace() does an exit() of its own somehow.

It's nothing special about jsc and it's not a function of __buildin_trap(), but JavaScriptCore is particularly susceptible to this problem and the way that we use the jsc shell to run JavaScriptCore tests exacerbates the problem.

The issue is that we can finish running a test while we are still compiling code.  This kind of interleaving is somewhat common in some tests and that's kind of intentional - the tests are basically checking that concurrent JIT works and so some compilations will still be on-going when the test finishes.

Weird stuff happens when the concurrent JIT crashes via WTFCrash() and the jsc shell is about to exit.  WTFCrash() appears to take sufficiently long that the exit() on the main thread happens before WTFCrash() can finish on the concurrent JIT thread.  For the bug I was debugging just now, this appeared to happen systematically: all tests would exit with a successful exit code while being half-way through dumping stack in WTFCrash().

This is a particularity of:

- Concurrent JIT.  Concurrent JIT is somewhat unique in that we have tests that will systematically exit with the concurrent JIT still running, and we are OK with this.  Hence concurrent JIT crashes may happen while we are exiting.

- jsc testing.  This doesn't have to do with JavaScriptCore or even the jsc shell but with how we run tests using jsc.  Instead of running a bunch of tests in one process (like RWT), we have one shell invocation per test.  This means we exit() a lot.

- WTFCrash() being slow.

Of course, you could also imagine a crash that starts happening "after" the process exits, so you might say that it's of course natural for a process to also exit while the crash is on-going.  But because of the combination of the above, I've found that for some compiler bugs, it ends up being *particularly* likely that we will exit exactly as the crashing thread is executing inside WTFCrash()'s stack dumping code.  Whenever a test exits "successfully" with stderr output consisting of a partial stack trace, it causes nothing but grief for the person doing the testing - arguably it might even be more useful for the test to have exited with no output at all, and it would *definitely* have been much more useful if the test finished dumping stack before exiting.  Most useful of all would be if the test actually crashed instead of exiting, which would be a less ambiguous way of reporting the test outcome than "success with error output".

This bug tracks finding a workaround that reduces the likelihood of such test outcomes.  I believe that this is a particularity of how we run JSC tests.  The 'jsc' shell is currently engineered solely to support our testing so it is appropriate to put the workaround there.  The least insane workaround is probably to have WTFCrash() set a global flag as soon as it is invoked, and to have the jsc shell pause (by busy-spinning or something else) instead of exiting if the flag is set.

[Alexey Proskuryakov]

Thank you, the proposed solution makes sense to me. Sounds like WTFReportBacktrace() takes long enough for main thread to exit already.