Bug 204894 – EWS: JSC-Tests-EWS should run with debug instead of release

• Status: RESOLVED
• Resolution: WONTFIX
• Component: Tools / Tests

[Jonathan Bedard]

This is a result of a conversation with Mark, who indicated that a debug EWS is probably more valuable than then a release one.

[Jonathan Bedard]

Created attachment 384912 [details]
Patch

[Alexey Proskuryakov]

Comment on attachment 384912 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=384912&action=review

> Tools/ChangeLog:3
> +        EWS: JSC-Tests-EWS should run with debug instead of release

I'm somewhat skeptical, because historically issues on debug JSC tests were often discarded or deprioritized.

Are we seeing many debug-only regressions on JSC tests that are getting fixed?

[Alexey Proskuryakov]

<rdar://problem/57643755>

[Mark Lam]

I found out last night that the debug build runs ridiculously slow.  Let me do something to make it complete in a more reasonable time before we apply this change.

(In reply to Alexey Proskuryakov from comment #2)
> Comment on attachment 384912 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=384912&action=review
> 
> > Tools/ChangeLog:3
> > +        EWS: JSC-Tests-EWS should run with debug instead of release
> 
> I'm somewhat skeptical, because historically issues on debug JSC tests were
> often discarded or deprioritized.
> 
> Are we seeing many debug-only regressions on JSC tests that are getting
> fixed?

With a debug build, we check for missing exception checks, and various GC bugs.  These bugs will go unnoticed if we don't run with the debug build.  That said, let me put together a debug build configuration that makes more sense to run in the EWS first.

Stay tuned.

[Alexey Proskuryakov]

If there is one that the JSC team is willing to fully defend, we should run it in post-commit testing too, and unskip tests that are marked as skipped in debug.

[Mark Lam]

After https://bugs.webkit.org/show_bug.cgi?id=205279 lands, there's one more change that needs to be made in order to have the tests complete in a reasonable amount of time: for the EWS, I recommend only running the default configuration of the tests.  That will get us a lot of mileage.

To only run the default configuration, you can pass --filter "\.default" to the run-javascriptcore-tests command line in addition to the --debug flag.

[Mark Lam]

Comment on attachment 384912 [details]
Patch

Please also add the command line option --filter '\.default'.

[Filip Pizlo]

Comment on attachment 384912 [details]
Patch

This isn't a great idea.  Debug testing takes longer, experiences races that are different from the release ones, and has important behavioral changes (object layout, synchronization) that make it quite different from release.

If we test one thing it should be release.

If we test two things it should be two different kinds of release builds.

[Jonathan Bedard]

After talking with various JSC folks, seems like this is a 'WON'T FIX', although we might modify release builds to do more debug-y things, that seems like it should be another bug.