Bug 154964

Summary: Slow JSC stress tests times out in 32 bit debug mode
Product: WebKit Reporter: Csaba Osztrogonác <ossy>
Component: New BugsAssignee: Csaba Osztrogonác <ossy>
Severity: Critical CC: ap, commit-queue, fpizlo, ossy, rniwa
Priority: P1    
Version: Other   
Hardware: Unspecified   
OS: Unspecified   
Bug Depends on:    
Bug Blocks: 154858    
Description Flags
Patch none

Description Csaba Osztrogonác 2016-03-03 03:29:42 PST
http://trac.webkit.org/changeset/197442 disabled FTL JIT testing 
on 32 bit bots, because there is no FTL JIT on any 32 bit platforms.

But 25-35 tests started to fail on these bots:
- https://build.webkit.org/builders/Apple%20El%20Capitan%2032-bit%20JSC%20%28BuildAndTest%29/builds/1658
- https://build.webkit.org/builders/Apple%20Yosemite%2032-bit%20JSC%20%28BuildAndTest%29

The problem is that these tests seems to be very slow in debug mode,
but they didn't fail previously because jsc-stress-tests script
duplicated the timeout in case of FTL JIT is enabled:

if $enableFTL and ENV["JSCTEST_timeout"]
    # Currently, using the FTL is a performance regression particularly in real
    # (i.e. non-loopy) benchmarks. Account for this in the timeout.
    ENV["JSCTEST_timeout"] = (ENV["JSCTEST_timeout"].to_i * 2).to_s

It seems we should duplicate timeout in debug mode too.
Comment 1 Csaba Osztrogonác 2016-03-03 03:39:24 PST
My first idea was to replace $enableFtl condition with
($enableFTL or !$ifJSCArgIsntProvidedAreWeReleaseBuild),
but unfortunately run-javascriptcore-tests doesn't pass
through the --debug option to run-jsc-stress-tests.

I don't have time to investigate this bug, feel free to pick it up.

But maybe the simplest fix is to set JSCTEST_timeout 
environment on these bots to a high enough number.
Comment 2 Alexey Proskuryakov 2016-03-03 09:23:26 PST
Will fix by rolling out r197442. There is no reason to keep such a regression.
Comment 3 Alexey Proskuryakov 2016-03-03 09:26:55 PST

*** This bug has been marked as a duplicate of bug 154967 ***
Comment 4 Csaba Osztrogonác 2016-03-03 09:31:30 PST
Rolling out r197442 only hides this problem. The tests are still slow, relying
on increased timeout set by --ftl-jit option is incorrect and should be fixed.
Comment 5 Alexey Proskuryakov 2016-03-05 15:21:33 PST
This bug used to be a duplicate, because it said that the tests failed on bots. That is fixed via bug 154967.

You changed the title after the fact.
Comment 6 Csaba Osztrogonác 2016-03-06 11:34:27 PST
(In reply to comment #5)
> This bug used to be a duplicate, because it said that the tests failed on
> bots. That is fixed via bug 154967.
> You changed the title after the fact.

I agree, the bug mentioned in the original title of the bug was "solved"
(papered over) by the rollout. But The description of this bug is clear
and the bug is still valid and should be fixed in the near future. The
original title was confusing, I just changed it to match with the desription.

It's not reasonable to rely on running useless FTL JIT tests (+50% runtime)
just to increase the JSCTEST_timeout on 32 bit debug JSC tester bots. We
should increase the timeout in debug mode in this bug and then we can
disable FTL JIT testing on all 32 bit bots in bug154858.
Comment 7 Csaba Osztrogonác 2016-05-10 08:38:16 PDT
Created attachment 278495 [details]
Comment 8 Michael Catanzaro 2016-05-17 08:36:32 PDT
Comment on attachment 278495 [details]

(Please commit when you're around to watch the bots.)
Comment 9 WebKit Commit Bot 2016-05-19 05:02:16 PDT
Comment on attachment 278495 [details]

Clearing flags on attachment: 278495

Committed r201158: <http://trac.webkit.org/changeset/201158>
Comment 10 WebKit Commit Bot 2016-05-19 05:02:21 PDT
All reviewed patches have been landed.  Closing bug.