Introduce a way to run benchmarks and JSRegress as stress tests with different jsc command-line options

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Filip Pizlo]

Patch forthcoming.

[Filip Pizlo]

Created attachment 210680 [details]
the patch

[Geoffrey Garen]

Comment on attachment 210680 [details]
the patch

rs=me

[Mark Hahnenberg]

Comment on attachment 210680 [details]
the patch

View in context: https://bugs.webkit.org/attachment.cgi?id=210680&action=review

r=me too with a couple comments

> Tools/Scripts/run-javascriptcore-tests:281
>      print "\n** The following fast/js test failures have been introduced:\n";

s/failures/crashes

> Tools/Scripts/run-jsc-stress-tests:45
> +begin
> +    require 'shellwords'
> +    
> +    def shellescape(text)
> +        Shellwords.shellescape(text)
> +    end
> +rescue => e
> +    $stderr.puts "Warning: no shellwords, will emulate: #{e.inspect}"
> +    def shellescape(text)
> +        text
> +    end
> +end
> +
> +def mysys(*cmd)
> +    commandArray = cmd.each{|value| shellescape(value.to_s)}.join(' ')
> +    $stderr.puts ">> #{commandArray}"
> +    riase unless system(*cmd)
> +end

You said in person that this is all dead code.

[Csaba Osztrogonác]

Yay, I like this new script. ;) Thank you very much for it.
With  this script we will be able easily catch new regressions 
early in different JIT tiers.

Just out of curiosity, are you planning to extend running 
stress tests to use_JIT=false and use_DFGJIT=false too?

[Filip Pizlo]

(In reply to comment #4)
> Yay, I like this new script. ;) Thank you very much for it.
> With  this script we will be able easily catch new regressions 
> early in different JIT tiers.
> 
> Just out of curiosity, are you planning to extend running 
> stress tests to use_JIT=false and use_DFGJIT=false too?

Yeah we should do something like this.

I don't recommend doing this for all tests, though.  SunSpider and V8 run fairly slow with the JITs disabled.  At some point you have to trade off test run time for coverage.

What I think we should do, is make new test commands available like:

//@ runLLInt
//@ runBaselineJIT

That do what you say.  So, if you want to write a test that tests any of those configurations you just add that kind of comment to the top of the test.

I also think we'll want to introduce a new test directory (like, Source/JavaScriptCore/tests/stress?) and run tests from there.

[Filip Pizlo]

Landed in http://trac.webkit.org/changeset/155200

[Benjamin Poulain]

jscore-test fails on the bots after this patch.

Can you please have a look?

[Filip Pizlo]

(In reply to comment #7)
> jscore-test fails on the bots after this patch.
> 
> Can you please have a look?

Yup, looking.

[Filip Pizlo]

(In reply to comment #8)
> (In reply to comment #7)
> > jscore-test fails on the bots after this patch.
> > 
> > Can you please have a look?
> 
> Yup, looking.

Should be fixed in http://trac.webkit.org/changeset/155216.  I verified that this new script runs on the bots which have an old version of Ruby.