Bug 77024 – Web Inspector: tests in PerformanceTests/inspector/ are timing out

• Component: Web Inspector (Deprecated)
• Status: RESOLVED FIXED

[Ryosuke Niwa]

See http://build.webkit.org/builders/Chromium%20Mac%20Release%20%28Perf%29/builds/6/steps/perf-test/logs/stdio

Running inspector/console-300-lines.html (8 of 14)
timeout: PerformanceTests/inspector/console-300-lines.html
FAILED

/Users/chrome-bot/Desktop/WebKit-BuildSlave/chromium-mac-perf-tests/build/PerformanceTests/inspector/first-open-elements.html killing driver
Running inspector/first-open-elements.html (9 of 14)
timeout: PerformanceTests/inspector/first-open-elements.html
FAILED

/Users/chrome-bot/Desktop/WebKit-BuildSlave/chromium-mac-perf-tests/build/PerformanceTests/inspector/first-open-resources.html killing driver
Running inspector/first-open-resources.html (10 of 14)
timeout: PerformanceTests/inspector/first-open-resources.html
FAILED

/Users/chrome-bot/Desktop/WebKit-BuildSlave/chromium-mac-perf-tests/build/PerformanceTests/inspector/first-open-scripts.html killing driver
Running inspector/first-open-scripts.html (11 of 14)
timeout: PerformanceTests/inspector/first-open-scripts.html
FAILED

/Users/chrome-bot/Desktop/WebKit-BuildSlave/chromium-mac-perf-tests/build/PerformanceTests/inspector/inspector-startup-time.html killing driver
Running inspector/inspector-startup-time.html (12 of 14)
timeout: PerformanceTests/inspector/inspector-startup-time.html
FAILED

/Users/chrome-bot/Desktop/WebKit-BuildSlave/chromium-mac-perf-tests/build/PerformanceTests/inspector/network-append-30-requests.html killing driver
Running inspector/network-append-30-requests.html (13 of 14)
timeout: PerformanceTests/inspector/network-append-30-requests.html
FAILED

/Users/chrome-bot/Desktop/WebKit-BuildSlave/chromium-mac-perf-tests/build/PerformanceTests/inspector/show-panel.html killing driver
Running inspector/show-panel.html (14 of 14)
timeout: PerformanceTests/inspector/show-panel.html
FAILED

[Ryosuke Niwa]

Skipped them in http://trac.webkit.org/changeset/106312 temporarily since run-perf-tests will spend 4+ minutes on each test when a test times out.

[Ilya Tikhonovsky]

The root of problem is that we have no Inspector's tests support on Chromium Mac platform.

[Ryosuke Niwa]

(In reply to comment #2)
> The root of problem is that we have no Inspector's tests support on Chromium Mac platform.

That sounds really bad. Can we add it?

[Ilya Tikhonovsky]

(In reply to comment #3)
> (In reply to comment #2)
> > The root of problem is that we have no Inspector's tests support on Chromium Mac platform.
> 
> That sounds really bad. Can we add it?

I'll try to do that.

Is it possible to disable the tests only for mac platform?

[Ryosuke Niwa]

(In reply to comment #4)
> I'll try to do that.
> 
> Is it possible to disable the tests only for mac platform?

Not at the moment :( layout-test-style skipped list or test_expectations.txt won't work here because we don't have a platform directory. But we can safely assume all perf bots are on Mac for now since I haven't heard anything about adding win/linux bot from the infrastructure team although I'm also intending to ping Qt/GTK folks.

It might be a good idea to add --force option so that you can run skipped tests locally though.

[Zoltan Horvath]

(In reply to comment #5)

> It might be a good idea to add --force option so that you can run skipped tests locally though.

I filed it: bug #90279

[Zoltan Horvath]

Some tests are not timeouting for me on Qt anymore:

Pass:
Running inspector/console-300-lines.html (1 of 7)
Running inspector/first-open-elements.html (2 of 7)
Running inspector/first-open-resources.html (3 of 7)
Running inspector/inspector-startup-time.html (6 of 7)

Timeout:
Running inspector/heap-snapshot-advanced.html (4 of 7)
Running inspector/heap-snapshot.html (5 of 7)

Fail:
Running inspector/native-memory-snapshot.html (7 of 7)

Ilya, was there any progress on Mac-Chromium inspector support?

[Ilya Tikhonovsky]

Created attachment 178204 [details]
Patch

[Ilya Tikhonovsky]

(In reply to comment #7)
> Some tests are not timeouting for me on Qt anymore:
> 
> Pass:
> Running inspector/console-300-lines.html (1 of 7)
> Running inspector/first-open-elements.html (2 of 7)
> Running inspector/first-open-resources.html (3 of 7)
> Running inspector/inspector-startup-time.html (6 of 7)
> 
> Timeout:
> Running inspector/heap-snapshot-advanced.html (4 of 7)
> Running inspector/heap-snapshot.html (5 of 7)
> 
> Fail:
> Running inspector/native-memory-snapshot.html (7 of 7)
> 
> Ilya, was there any progress on Mac-Chromium inspector support?

I've fixed the problem with chromium-mac but heap-snapshot* tests wouldn't work in non-chromium WebKit ports.

[Zoltan Horvath]

(In reply to comment #9)
>
> I've fixed the problem with chromium-mac but heap-snapshot* tests wouldn't work in non-chromium WebKit ports.

Great, thanks! 

Can you explain me the situation with the heap-snapshot tests? (What do we need to do to get these work on non-chromium ports?)

[Ryosuke Niwa]

Comment on attachment 178204 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=178204&action=review

> PerformanceTests/ChangeLog:8
> +        It was fixed in downstream patch r171707.

Does it work on non-Chromium ports? run-perf-tests doesn’t support per-port skipped list. If any of the tests don’t pass on non-Chromium ports, then we have to skip them on bots.
Also, we might want to consider adopting the perf. test framework used by other perf. tests instead of using chromium perf-bot style output.
runner.js has improved considerably in the last six months.

[Ryosuke Niwa]

Comment on attachment 178204 [details]
Patch

It appears that we shouldn’t land this patch since some of the tests wouldn’t run on non-Chromium ports.

[Ilya Tikhonovsky]

(In reply to comment #11)
> (From update of attachment 178204 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=178204&action=review
> 
> > PerformanceTests/ChangeLog:8
> > +        It was fixed in downstream patch r171707.
> 
> Does it work on non-Chromium ports? run-perf-tests doesn’t support per-port skipped list. If any of the tests don’t pass on non-Chromium ports, then we have to skip them on bots.
> Also, we might want to consider adopting the perf. test framework used by other perf. tests instead of using chromium perf-bot style output.
> runner.js has improved considerably in the last six months.

Is there a chance to support port specific skip list or even support current TestExpectation format?

(In reply to comment #10)
> (In reply to comment #9)
> >
> > I've fixed the problem with chromium-mac but heap-snapshot* tests wouldn't work in non-chromium WebKit ports.
> 
> Great, thanks! 
> 
> Can you explain me the situation with the heap-snapshot tests? (What do we need to do to get these work on non-chromium ports?)


You need to implement detailed heap snapshot for JSC engine.

It is a bit complex task. Each memory fragment separately allocated by JSC should be reported as a snapshot node with id, size, type, name and edges count. Also all the links from the fragment to other fragments should be reported as snapshot edges with type, name and to_node_id.

Id consistency: The node id that was assigned to a memory fragment when you took a snapshot has to be the same in the second snapshot even if the memory fragment was moved by GC.

Also you can report an array of strings and their indexes can be used as node names and edge names.

[Ryosuke Niwa]

(In reply to comment #13)
> (In reply to comment #11)
> > (From update of attachment 178204 [details] [details])
> > View in context: https://bugs.webkit.org/attachment.cgi?id=178204&action=review
> > 
> > > PerformanceTests/ChangeLog:8
> > > +        It was fixed in downstream patch r171707.
> > 
> > Does it work on non-Chromium ports? run-perf-tests doesn’t support per-port skipped list. If any of the tests don’t pass on non-Chromium ports, then we have to skip them on bots.
> > Also, we might want to consider adopting the perf. test framework used by other perf. tests instead of using chromium perf-bot style output.
> > runner.js has improved considerably in the last six months.
> 
> Is there a chance to support port specific skip list or even support current TestExpectation format?

No. We don’t have platforms directory so I’m not even sure how we’d support that, and adding platforms directly just to support Skipped/TestExpectations files seems rather silly.

[Ryosuke Niwa]

Created attachment 203321 [details]
Delete tests for now

[Andreas Kling]

Comment on attachment 203321 [details]
Delete tests for now

rs=me

[WebKit Commit Bot]

Comment on attachment 203321 [details]
Delete tests for now

Clearing flags on attachment: 203321

Committed r150955: <http://trac.webkit.org/changeset/150955>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.