Bug 103943 – nrwt: derive per-thread, per-worker stats from result summaries

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Dirk Pranke]

nrwt: derive per-thread, per-worker stats from result summaries

[Dirk Pranke]

Created attachment 177367 [details]
Patch

[Ojan Vafai]

Comment on attachment 177367 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=177367&action=review

> Tools/ChangeLog:21
> +        Also, this changes the stats so that we report only the time taken
> +        to actually run the test, rather than the time taken to run the test
> +        plus run matching reftests, compute diffs, etc. It's unclear which
> +        number is actually more interesting, but I suspect most people
> +        expect the stats to report the former; we can always add the latter
> +        back in if needed.

I think for the individual test timings we care about the time actually spent in DRT. But for the aggregate numbers we care about the whole time. For example, all your numbers for the time spent in workers or per-directory will now not include time spent doing the other work, no? That seems less than ideal.

[Dirk Pranke]

(In reply to comment #2)
> (From update of attachment 177367 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=177367&action=review
> 
> > Tools/ChangeLog:21
> > +        Also, this changes the stats so that we report only the time taken
> > +        to actually run the test, rather than the time taken to run the test
> > +        plus run matching reftests, compute diffs, etc. It's unclear which
> > +        number is actually more interesting, but I suspect most people
> > +        expect the stats to report the former; we can always add the latter
> > +        back in if needed.
> 
> I think for the individual test timings we care about the time actually spent in DRT. But for the aggregate numbers we care about the whole time. For example, all your numbers for the time spent in workers or per-directory will now not include time spent doing the other work, no? That seems less than ideal.

Yeah, you're right. I was being a little lazy. I will collect both numbers.

[Dirk Pranke]

Created attachment 177522 [details]
collect both test_run_time and total_run_time

[Ojan Vafai]

Comment on attachment 177522 [details]
collect both test_run_time and total_run_time

View in context: https://bugs.webkit.org/attachment.cgi?id=177522&action=review

> Tools/Scripts/webkitpy/layout_tests/models/test_results.py:54
> +        self.total_run_time = 0  # The time taken to run the test plus any references, compute diffs, etc.

Where do we set this?

[Dirk Pranke]

(In reply to comment #5)
> (From update of attachment 177522 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=177522&action=review
> 
> > Tools/Scripts/webkitpy/layout_tests/models/test_results.py:54
> > +        self.total_run_time = 0  # The time taken to run the test plus any references, compute diffs, etc.
> 
> Where do we set this?


Here, next to where we set the worker name and shard name:

> Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py:304
> +        result.total_test_time = time.time() - start

[Ojan Vafai]

total_test_time != total_run_time

[Dirk Pranke]

Created attachment 177535 [details]
fix typos, lint

[Dirk Pranke]

Created attachment 177537 [details]
fix typos, lint, verify manually

[Dirk Pranke]

(In reply to comment #7)
> total_test_time != total_run_time

Whoops. Fixed now. Clearly I need some tests for this and don't have them, but if it's okay with you I'd like to address that in a separate patch. I verified things look right manually.

[Dirk Pranke]

Committed r136579: <http://trac.webkit.org/changeset/136579>