Bug 129155 – [Win][LLINT] Many JSC stress test failures.

• Status: RESOLVED
• Resolution: FIXED
• Component: JavaScriptCore

When running the stress tests on Windows with LLINT enabled, about 25% of the tests fail.

Created attachment 224879 [details]
Patch

[Michael Saboff]

Comment on attachment 224879 [details]
Patch

r=me

(In reply to comment #2)
> (From update of attachment 224879 [details])
> r=me

Thanks for reviewing :)

With this patch, I get the following results from the stress tests:

** The following JSC stress test failures have been introduced:
        mozilla-tests.yaml/ecma/FunctionObjects/15.3.1.1-3.js.mozilla
        mozilla-tests.yaml/ecma/FunctionObjects/15.3.1.1-3.js.mozilla-llint
        mozilla-tests.yaml/ecma/FunctionObjects/15.3.1.1-3.js.mozilla-baseline
        mozilla-tests.yaml/ecma/FunctionObjects/15.3.1.1-3.js.mozilla-dfg-eager-no-cjit-validate-phases
        mozilla-tests.yaml/ecma/FunctionObjects/15.3.2.1-3.js.mozilla
        mozilla-tests.yaml/ecma/FunctionObjects/15.3.2.1-3.js.mozilla-llint
        mozilla-tests.yaml/ecma/FunctionObjects/15.3.2.1-3.js.mozilla-baseline
        mozilla-tests.yaml/ecma/FunctionObjects/15.3.2.1-3.js.mozilla-dfg-eager-no-cjit-validate-phases
        mozilla-tests.yaml/ecma/FunctionObjects/15.3.5-1.js.mozilla
        mozilla-tests.yaml/ecma/FunctionObjects/15.3.5-1.js.mozilla-llint
        mozilla-tests.yaml/ecma/FunctionObjects/15.3.5-1.js.mozilla-baseline
        mozilla-tests.yaml/ecma/FunctionObjects/15.3.5-1.js.mozilla-dfg-eager-no-cjit-validate-phases
        mozilla-tests.yaml/ecma_3/Date/15.9.5.6.js.mozilla
        mozilla-tests.yaml/ecma_3/Date/15.9.5.6.js.mozilla-llint
        mozilla-tests.yaml/ecma_3/Date/15.9.5.6.js.mozilla-baseline
        mozilla-tests.yaml/ecma_3/Date/15.9.5.6.js.mozilla-dfg-eager-no-cjit-validate-phases
        mozilla-tests.yaml/ecma/Array/15.4.5.2-2.js.mozilla
        mozilla-tests.yaml/ecma/Array/15.4.5.2-2.js.mozilla-llint
        mozilla-tests.yaml/ecma/Array/15.4.5.2-2.js.mozilla-baseline
        mozilla-tests.yaml/ecma/Array/15.4.5.2-2.js.mozilla-dfg-eager-no-cjit-validate-phases
        mozilla-tests.yaml/js1_5/Regress/regress-191633.js.mozilla
        mozilla-tests.yaml/js1_5/Regress/regress-191633.js.mozilla-llint
        mozilla-tests.yaml/js1_5/Regress/regress-191633.js.mozilla-baseline
        mozilla-tests.yaml/js1_5/Regress/regress-191633.js.mozilla-dfg-eager-no-cjit-validate-phases
        mozilla-tests.yaml/js1_5/Regress/regress-192414.js.mozilla
        mozilla-tests.yaml/js1_5/Regress/regress-192414.js.mozilla-llint
        mozilla-tests.yaml/js1_5/Regress/regress-192414.js.mozilla-baseline
        mozilla-tests.yaml/js1_5/Regress/regress-192414.js.mozilla-dfg-eager-no-cjit-validate-phases
        mozilla-tests.yaml/js1_5/String/regress-179068.js.mozilla
        mozilla-tests.yaml/js1_5/String/regress-179068.js.mozilla-llint
        mozilla-tests.yaml/js1_5/String/regress-179068.js.mozilla-baseline
        mozilla-tests.yaml/js1_5/String/regress-179068.js.mozilla-dfg-eager-no-cjit-validate-phases
        stress/ftl-arithcos.js.default
        stress/ftl-arithcos.js.no-llint
        stress/ftl-arithcos.js.always-trigger-copy-phase
        stress/ftl-arithcos.js.no-cjit-validate-phases
        stress/ftl-arithcos.js.dfg-eager
        stress/ftl-arithcos.js.dfg-eager-no-cjit-validate
        jsc-layout-tests.yaml/js/script-tests/basic-map.js.layout
        jsc-layout-tests.yaml/js/script-tests/basic-map.js.layout-no-cjit
        jsc-layout-tests.yaml/js/script-tests/excessive-comma-usage.js.layout
        jsc-layout-tests.yaml/js/script-tests/excessive-comma-usage.js.layout-no-cjit
        jsc-layout-tests.yaml/js/script-tests/excessive-comma-usage.js.layout-dfg-eager-no-cjit
        jsc-layout-tests.yaml/js/script-tests/function-apply-many-args.js.layout
        jsc-layout-tests.yaml/js/script-tests/function-apply-many-args.js.layout-no-llint
        jsc-layout-tests.yaml/js/script-tests/function-apply-many-args.js.layout-no-cjit
        jsc-layout-tests.yaml/js/script-tests/function-apply-many-args.js.layout-dfg-eager-no-cjit
        jsc-layout-tests.yaml/js/script-tests/function-apply.js.layout
        jsc-layout-tests.yaml/js/script-tests/function-apply.js.layout-no-llint
        jsc-layout-tests.yaml/js/script-tests/function-apply.js.layout-no-cjit
        jsc-layout-tests.yaml/js/script-tests/function-apply.js.layout-dfg-eager-no-cjit
        jsc-layout-tests.yaml/js/script-tests/reentrant-caching.js.layout-no-llint
        jsc-layout-tests.yaml/js/script-tests/reentrant-caching.js.layout-dfg-eager-no-cjit
        jsc-layout-tests.yaml/js/script-tests/sort-stability.js.layout
        jsc-layout-tests.yaml/js/script-tests/sort-stability.js.layout-no-llint
        jsc-layout-tests.yaml/js/script-tests/sort-stability.js.layout-no-cjit
        jsc-layout-tests.yaml/js/script-tests/sort-stability.js.layout-dfg-eager-no-cjit

Results for JSC stress tests:
    57 failures found.

When running with the C Loop LLINT, I get these results:

** The following JSC stress test failures have been introduced:
        mozilla-tests.yaml/ecma_3/Date/15.9.5.6.js.mozilla
        mozilla-tests.yaml/ecma_3/Date/15.9.5.6.js.mozilla-llint
        mozilla-tests.yaml/ecma_3/Date/15.9.5.6.js.mozilla-baseline
        mozilla-tests.yaml/ecma_3/Date/15.9.5.6.js.mozilla-dfg-eager-no-cjit-validate-phases
        stress/ftl-arithcos.js.default
        stress/ftl-arithcos.js.no-llint
        stress/ftl-arithcos.js.always-trigger-copy-phase
        stress/ftl-arithcos.js.no-cjit-validate-phases
        stress/ftl-arithcos.js.dfg-eager
        stress/ftl-arithcos.js.dfg-eager-no-cjit-validate
        jsc-layout-tests.yaml/js/script-tests/basic-map.js.layout
        jsc-layout-tests.yaml/js/script-tests/basic-map.js.layout-no-llint
        jsc-layout-tests.yaml/js/script-tests/basic-map.js.layout-no-cjit
        jsc-layout-tests.yaml/js/script-tests/basic-map.js.layout-dfg-eager-no-cjit
        jsc-layout-tests.yaml/js/script-tests/sort-stability.js.layout
        jsc-layout-tests.yaml/js/script-tests/sort-stability.js.layout-no-llint
        jsc-layout-tests.yaml/js/script-tests/sort-stability.js.layout-no-cjit
        jsc-layout-tests.yaml/js/script-tests/sort-stability.js.layout-dfg-eager-no-cjit

Results for JSC stress tests:
    18 failures found.

So there seems to be some regressions when using the x86 backend compared to the C Loop backend, I will look into those next.

[Mark Lam]

(In reply to comment #5)
> Results for JSC stress tests:
>     18 failures found.
> 
> So there seems to be some regressions when using the x86 backend compared to the C Loop backend, I will look into those next.

Beware of https://bugs.webkit.org/show_bug.cgi?id=129099 which may contribute some failures.  Note: some stress tests will turn on the DFG even if you don’t have it on by default.  The only way it doesn’t get turned on is if the DFG is not built in.  The C loop LLINT build benefits from that.

(In reply to comment #6)
> (In reply to comment #5)
> > Results for JSC stress tests:
> >     18 failures found.
> > 
> > So there seems to be some regressions when using the x86 backend compared to the C Loop backend, I will look into those next.
> 
> Beware of https://bugs.webkit.org/show_bug.cgi?id=129099 which may contribute some failures.  Note: some stress tests will turn on the DFG even if you don’t have it on by default.  The only way it doesn’t get turned on is if the DFG is not built in.  The C loop LLINT build benefits from that.

Ok, so maybe I should run a test with DFG and LLINT enabled?

[Mark Lam]

(In reply to comment #7)
> (In reply to comment #6)
> > (In reply to comment #5)
> > > Results for JSC stress tests:
> > >     18 failures found.
> > > 
> > > So there seems to be some regressions when using the x86 backend compared to the C Loop backend, I will look into those next.
> > 
> > Beware of https://bugs.webkit.org/show_bug.cgi?id=129099 which may contribute some failures.  Note: some stress tests will turn on the DFG even if you don’t have it on by default.  The only way it doesn’t get turned on is if the DFG is not built in.  The C loop LLINT build benefits from that.
> 
> Ok, so maybe I should run a test with DFG and LLINT enabled?

Just to clarify: the LLINT is always enabled.  It’s either the C loop LLINT or the ASM LLINT.

The DFG can be disabled at runtime, but some jsc stress tests will turn it back on because that is the purpose of those tests i.e. to test the DFG.  If you run them with the DFG not built in the binary (like the C loop build), then the DFG won’t come into play.

To remove the effects of the DFG, don’t build the DFG into the code.

(In reply to comment #8)
> (In reply to comment #7)
> > (In reply to comment #6)
> > > (In reply to comment #5)
> > > > Results for JSC stress tests:
> > > >     18 failures found.
> > > > 
> > > > So there seems to be some regressions when using the x86 backend compared to the C Loop backend, I will look into those next.
> > > 
> > > Beware of https://bugs.webkit.org/show_bug.cgi?id=129099 which may contribute some failures.  Note: some stress tests will turn on the DFG even if you don’t have it on by default.  The only way it doesn’t get turned on is if the DFG is not built in.  The C loop LLINT build benefits from that.
> > 
> > Ok, so maybe I should run a test with DFG and LLINT enabled?
> 
> Just to clarify: the LLINT is always enabled.  It’s either the C loop LLINT or the ASM LLINT.
> 
> The DFG can be disabled at runtime, but some jsc stress tests will turn it back on because that is the purpose of those tests i.e. to test the DFG.  If you run them with the DFG not built in the binary (like the C loop build), then the DFG won’t come into play.
> 
> To remove the effects of the DFG, don’t build the DFG into the code.

Ok, I haven't been building with DFG, only LLINT (x86 backend).

Not a committer, cq+?  :)

[WebKit Commit Bot]

Comment on attachment 224879 [details]
Patch

Clearing flags on attachment: 224879

Committed r164494: <http://trac.webkit.org/changeset/164494>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Csaba Osztrogonác]

(In reply to comment #11)
> (From update of attachment 224879 [details])
> Clearing flags on attachment: 224879
> 
> Committed r164494: <http://trac.webkit.org/changeset/164494>

It broke zillion tests on 32 bit platforms. :-/

Here is the log from the GTK bot:
 http://build.webkit.org/builders/GTK%20Linux%2032-bit%20Release/builds/41425

But you can easily reproduce the  failures on Mac too.

(In reply to comment #13)
> (In reply to comment #11)
> > (From update of attachment 224879 [details] [details])
> > Clearing flags on attachment: 224879
> > 
> > Committed r164494: <http://trac.webkit.org/changeset/164494>
> 
> It broke zillion tests on 32 bit platforms. :-/
> 
> Here is the log from the GTK bot:
>  http://build.webkit.org/builders/GTK%20Linux%2032-bit%20Release/builds/41425
> 
> But you can easily reproduce the  failures on Mac too.

Sorry for this, I will look into it, and create a new patch soon.

[Mark Lam]

(In reply to comment #14)
> (In reply to comment #13)
> > (In reply to comment #11)
> > > (From update of attachment 224879 [details] [details] [details])
> > > Clearing flags on attachment: 224879
> > > 
> > > Committed r164494: <http://trac.webkit.org/changeset/164494>
> > 
> > It broke zillion tests on 32 bit platforms. :-/
> > 
> > Here is the log from the GTK bot:
> >  http://build.webkit.org/builders/GTK%20Linux%2032-bit%20Release/builds/41425
> > 
> > But you can easily reproduce the  failures on Mac too.
> 
> Sorry for this, I will look into it, and create a new patch soon.

peavo, if you think it will take longer than 1 hour to find the fix, we should roll out this change first until you can fix it.  Please advise.

(In reply to comment #15)
> (In reply to comment #14)
> > (In reply to comment #13)
> > > (In reply to comment #11)
> > > > (From update of attachment 224879 [details] [details] [details] [details])
> > > > Clearing flags on attachment: 224879
> > > > 
> > > > Committed r164494: <http://trac.webkit.org/changeset/164494>
> > > 
> > > It broke zillion tests on 32 bit platforms. :-/
> > > 
> > > Here is the log from the GTK bot:
> > >  http://build.webkit.org/builders/GTK%20Linux%2032-bit%20Release/builds/41425
> > > 
> > > But you can easily reproduce the  failures on Mac too.
> > 
> > Sorry for this, I will look into it, and create a new patch soon.
> 
> peavo, if you think it will take longer than 1 hour to find the fix, we should roll out this change first until you can fix it.  Please advise.

I think I can post a fix within one hour.

[Mark Lam]

FYI, the fix for the regression was address in https://bugs.webkit.org/show_bug.cgi?id=129272.  Fix landed in r164612: <http://trac.webkit.org/changeset/164612>