Bug 120808 - Introduce a way to run benchmarks and JSRegress as stress tests with different jsc command-line options
Summary: Introduce a way to run benchmarks and JSRegress as stress tests with differen...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: All All
: P2 Normal
Assignee: Filip Pizlo
URL:
Keywords:
Depends on:
Blocks: 120696 120809 120831
  Show dependency treegraph
 
Reported: 2013-09-05 15:58 PDT by Filip Pizlo
Modified: 2013-09-06 15:04 PDT (History)
14 users (show)

See Also:


Attachments
the patch (30.72 KB, patch)
2013-09-05 16:31 PDT, Filip Pizlo
ggaren: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Filip Pizlo 2013-09-05 15:58:39 PDT
Patch forthcoming.
Comment 1 Filip Pizlo 2013-09-05 16:31:54 PDT
Created attachment 210680 [details]
the patch
Comment 2 Geoffrey Garen 2013-09-05 16:35:31 PDT
Comment on attachment 210680 [details]
the patch

rs=me
Comment 3 Mark Hahnenberg 2013-09-05 16:43:19 PDT
Comment on attachment 210680 [details]
the patch

View in context: https://bugs.webkit.org/attachment.cgi?id=210680&action=review

r=me too with a couple comments

> Tools/Scripts/run-javascriptcore-tests:281
>      print "\n** The following fast/js test failures have been introduced:\n";

s/failures/crashes

> Tools/Scripts/run-jsc-stress-tests:45
> +begin
> +    require 'shellwords'
> +    
> +    def shellescape(text)
> +        Shellwords.shellescape(text)
> +    end
> +rescue => e
> +    $stderr.puts "Warning: no shellwords, will emulate: #{e.inspect}"
> +    def shellescape(text)
> +        text
> +    end
> +end
> +
> +def mysys(*cmd)
> +    commandArray = cmd.each{|value| shellescape(value.to_s)}.join(' ')
> +    $stderr.puts ">> #{commandArray}"
> +    riase unless system(*cmd)
> +end

You said in person that this is all dead code.
Comment 4 Csaba Osztrogonác 2013-09-05 21:35:03 PDT
Yay, I like this new script. ;) Thank you very much for it.
With  this script we will be able easily catch new regressions 
early in different JIT tiers.

Just out of curiosity, are you planning to extend running 
stress tests to use_JIT=false and use_DFGJIT=false too?
Comment 5 Filip Pizlo 2013-09-06 11:48:08 PDT
(In reply to comment #4)
> Yay, I like this new script. ;) Thank you very much for it.
> With  this script we will be able easily catch new regressions 
> early in different JIT tiers.
> 
> Just out of curiosity, are you planning to extend running 
> stress tests to use_JIT=false and use_DFGJIT=false too?

Yeah we should do something like this.

I don't recommend doing this for all tests, though.  SunSpider and V8 run fairly slow with the JITs disabled.  At some point you have to trade off test run time for coverage.

What I think we should do, is make new test commands available like:

//@ runLLInt
//@ runBaselineJIT

That do what you say.  So, if you want to write a test that tests any of those configurations you just add that kind of comment to the top of the test.

I also think we'll want to introduce a new test directory (like, Source/JavaScriptCore/tests/stress?) and run tests from there.
Comment 6 Filip Pizlo 2013-09-06 11:50:29 PDT
Landed in http://trac.webkit.org/changeset/155200
Comment 7 Benjamin Poulain 2013-09-06 14:30:18 PDT
jscore-test fails on the bots after this patch.

Can you please have a look?
Comment 8 Filip Pizlo 2013-09-06 14:57:45 PDT
(In reply to comment #7)
> jscore-test fails on the bots after this patch.
> 
> Can you please have a look?

Yup, looking.
Comment 9 Filip Pizlo 2013-09-06 15:04:08 PDT
(In reply to comment #8)
> (In reply to comment #7)
> > jscore-test fails on the bots after this patch.
> > 
> > Can you please have a look?
> 
> Yup, looking.

Should be fixed in http://trac.webkit.org/changeset/155216.  I verified that this new script runs on the bots which have an old version of Ruby.