run-javascriptcore-tests is failing on Windows WebKit2 bots. See the URL for an example. These tests aren't failing on any other bots. We should figure out what's different about these bots.
<rdar://problem/9149165>
This is now the only thing keeping the Windows WebKit2 bots from being green!
Geoff Garen helped me to discover that, at least in one case, one of the "failing" tests was actually crashing. I'm going to see if I can reproduce the crash in a debugger on one of the buildslaves.
I can reproduce the failure by running run-javascriptcore-tests on the buildslave. I haven't yet been able to repro a crash, though.
Created attachment 86236 [details] Assertion failure in Heap::destroy I wasn't able to get a crash in a Release build on the buildslave. But I was able to get an assertion failure in a Debug build on the buildslave.
The assertion failure is the same one from bug 52156.
You can see, for example, from <http://build.webkit.org/builders/Windows%207%20Release%20%28WebKit2%20Tests%29/builds/4322/steps/jscore-test/logs/actual.html>: > Testcase ecma_2/LexicalConventions/regexp-literals-001.js failed > [ Previous Failure | Next Failure | Top of Page ] > Expected exit code 0, got 126 Presumably this means the test crashed.
Here's the backtrace of the assertion failure: > JavaScriptCore.dll!JSC::Heap::destroy() Line 71 + 0x30 bytes C++ jsc.exe!cleanupGlobalData(JSC::JSGlobalData * globalData=0x0299cd28) Line 370 C++ jsc.exe!main(int argc=6, char * * argv=0x0299af60) Line 362 + 0x9 bytes C++ jsc.exe!__tmainCRTStartup() Line 597 + 0x17 bytes C kernel32.dll!_BaseProcessStart@4() + 0x23 bytes
Some info from IRC: <ggaren> aroben: what is the value of m_operationInProgress? <aroben> ggaren: NoOperation <ggaren> aroben: what are m_globalData->m_refCount and m_globalData->dynamicGlobalObject? <aroben> ggaren: 2 and 0x022a0128 <ggaren> aroben: and is m_globalData->m_deletionHasBegun false? <aroben> ggaren: yes <ggaren> aroben: inside main, what is the value of res? <aroben> ggaren: 3 <ggaren> aroben: i have a guess that res was set to 3 by the TRY / EXCEPT clause. if so, we've caught the bug too late. <aroben> ggaren: makes sense <aroben> ggaren: I can remove the try/except <aroben> ggaren: I see a comment that say try/except is only there to block the normal crash report mechanisms <aroben> ggaren: seems like eventually we'd like to save crash logs instead of just using a magic 3 return code <ggaren> aroben: inside jsc, all access (save 1 irrelevant place) to dynamicGlobalObject is managed by an RAII object: DynamicGlobalObjectScope. so, now i'm even more sure of my guess.
Removing the try/except resulted in a CRASH() with the following backtrace: > JavaScriptCore.dll!WTF::fastRealloc(void * p=0x7eb00040, unsigned int n=14785173) Line 371 + 0x5 bytes C++ JavaScriptCore.dll!JSC::AssemblerBuffer::grow(int extraCapacity=0) Line 175 + 0x13 bytes C++ JavaScriptCore.dll!JSC::AssemblerBuffer::ensureSpace(int space=16) Line 61 C++ JavaScriptCore.dll!JSC::X86Assembler::X86InstructionFormatter::oneByteOp(JSC::X86Assembler::OneByteOpcodeID opcode=OP_CALL_rel32) Line 1692 C++ JavaScriptCore.dll!JSC::X86Assembler::call() Line 1221 C++ JavaScriptCore.dll!JSC::MacroAssemblerX86::call() Line 125 + 0xe bytes C++ JavaScriptCore.dll!JSC::JITStubCall::call() Line 176 C++ JavaScriptCore.dll!JSC::JITStubCall::call(unsigned int dst=1) Line 196 C++ JavaScriptCore.dll!JSC::JIT::emit_op_resolve_with_base(JSC::Instruction * currentInstruction=0x7bc495f0) Line 1231 C++ JavaScriptCore.dll!JSC::JIT::privateCompileMainPass() Line 300 + 0xc bytes C++ JavaScriptCore.dll!JSC::JIT::privateCompile(JSC::MacroAssemblerCodePtr * functionEntryArityCheck=0x00000000) Line 484 C++ JavaScriptCore.dll!JSC::JIT::compile(JSC::JSGlobalData * globalData=0x0299cd28, JSC::CodeBlock * codeBlock=0x57fb19c0, JSC::MacroAssemblerCodePtr * functionEntryArityCheck=0x00000000, void * offsetBase=0x00000000) Line 183 + 0x26 bytes C++ JavaScriptCore.dll!JSC::EvalExecutable::compileInternal(JSC::ExecState * exec=0x02b20038, JSC::ScopeChainNode * scopeChainNode=0x028c0128) Line 121 + 0x20 bytes C++ JavaScriptCore.dll!JSC::EvalExecutable::compile(JSC::ExecState * exec=0x02b20038, JSC::ScopeChainNode * scopeChainNode=0x028c0128) Line 221 + 0x10 bytes C++ JavaScriptCore.dll!JSC::EvalCodeCache::get(JSC::ExecState * exec=0x02b20038, JSC::ScriptExecutable * owner=0x03420180, bool inStrictContext=false, const JSC::UString & evalSource={f('0') f('1') f('2') f('3') f('4') f('5') f('6') f('7') f('8') f('9') f('10') f('11') f('12') f('13') f('14') f('15') f('16') f('17') f('18') f('19') f('20') f('21') f('22') f('23') f('24') f('25') f('26') f('27') f('28') f('29') f('30') f('31') f('32') f('33') f('34') f('35') f('36') f('37') f('38') f('39') f('40') f('41') f('42') f('43') f('44') f('45') f('46') f('47') f('48') f('49') f('50'), JSC::ScopeChainNode * scopeChain=0x028c0128, JSC::JSValue & exceptionValue={...}) Line 57 + 0x10 bytes C++ JavaScriptCore.dll!JSC::Interpreter::callEval(JSC::ExecState * callFrame=0x02b20038, JSC::RegisterFile * registerFile=0x02ac8fcc, JSC::Register * argv=0x02b20040, int argc=2, int registerOffset=9) Line 412 + 0x34 bytes C++ JavaScriptCore.dll!cti_op_call_eval(void * * args=0x0012fc98) Line 3118 C++ JavaScriptCore.dll!@cti_op_create_this@4() + 0x1df bytes C++ JavaScriptCore.dll!JSC::JITCode::execute(JSC::RegisterFile * registerFile=0x02ac8fcc, JSC::ExecState * callFrame=0x02b20038, JSC::JSGlobalData * globalData=0x0299cd28) Line 77 + 0x22 bytes C++ JavaScriptCore.dll!JSC::Interpreter::execute(JSC::ProgramExecutable * program=0x03420180, JSC::ExecState * callFrame=0x022a01a0, JSC::ScopeChainNode * scopeChain=0x028c0128, JSC::JSObject * thisObj=0x022a0128) Line 773 + 0x25 bytes C++ JavaScriptCore.dll!JSC::evaluate(JSC::ExecState * exec=0x022a01a0, JSC::ScopeChainNode * scopeChain=0x028c0128, const JSC::SourceCode & source={...}, JSC::JSValue thisValue={...}) Line 69 C++ jsc.exe!runWithScripts(GlobalObject * globalObject=0x022a0128, const WTF::Vector<Script,0> & scripts=[2]({isFile=true argument=0x0299afca "./js1_5/shell.js" },{isFile=true argument=0x0299afde "./js1_5/Regress/regress-159334.js" }), bool dump=false) Line 402 + 0x52 bytes C++ jsc.exe!jscmain(int argc=6, char * * argv=0x0299af60, JSC::JSGlobalData * globalData=0x0299cd28) Line 540 + 0x11 bytes C++ jsc.exe!main(int argc=6, char * * argv=0x0299af60) Line 359 + 0x11 bytes C++ jsc.exe!__tmainCRTStartup() Line 597 + 0x17 bytes C kernel32.dll!_BaseProcessStart@4() + 0x23 bytes
The test that causes the crash is js1_5/Regress/regress-159334.js.
The crashing call is in AssemblerBuffer::grow: m_buffer = static_cast<char*>(fastRealloc(m_buffer, m_capacity)); m_capacity is 14785173.
In EvalCodeCache::get, evalSource's length is 1461754.
I should note that I have full page heap enabled (which is basically the Windows equivalent of GuardMalloc).
::GetLastError() says: 0x00000008 Not enough storage is available to process this command.
Turning off full page heap makes the crash go away. Presumably the extra memory incurred by full page heap is causing the allocation to fail. Maybe this has all been a red herring.
(In reply to comment #7) > You can see, for example, from <http://build.webkit.org/builders/Windows%207%20Release%20%28WebKit2%20Tests%29/builds/4322/steps/jscore-test/logs/actual.html>: > > > Testcase ecma_2/LexicalConventions/regexp-literals-001.js failed > > [ Previous Failure | Next Failure | Top of Page ] > > Expected exit code 0, got 126 > > Presumably this means the test crashed. Google seems to indicate that 126 means "specified module could not be found".
Every failure I've seen on the bots recently has been due to exit code 126.
It turns out this failure is specific to Windows 7 SP1. Windows XP, and Windows 7 without SP1 do not have this bug.
I don't see any crash dumps from jsc.exe on one of the failing bots, so I don't think this is manifesting as a crash. (Which corroborates what Adam indicated above about the crash he was seeing being caused by out of memory from full page heap).
There is no Windows WebKit2 port, and Windows bots are happy now.