Bug 174443

Summary: Add BenchmarkRunner patches with WebDriver support for each testing plan
Product: WebKit Reporter: Matthew Stewart <matthew_r_stewart>
Component: New BugsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED FIXED    
Severity: Normal CC: buildbot, commit-queue, dewei_zhu, glenn, slewis
Priority: P2    
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Bug Depends on: 174390    
Bug Blocks: 174056    
Attachments:
Description Flags
Patch
buildbot: commit-queue-
Archive of layout-test-results from ews122 for ios-simulator-wk2
none
Add BenchmarkRunner patches with WebDriver support for each testing plan
slewis: review+, commit-queue: commit-queue-
patch none

Description Matthew Stewart 2017-07-12 16:46:28 PDT
Add BenchmarkRunner patches with WebDriver support for each testing plan
Comment 1 Matthew Stewart 2017-07-12 16:51:04 PDT
Created attachment 315308 [details]
Patch
Comment 2 Build Bot 2017-07-12 16:53:40 PDT
Attachment 315308 [details] did not pass style-queue:


ERROR: Tools/Scripts/webkitpy/benchmark_runner/benchmark_builder.py:39:  [BenchmarkBuilder.__enter__] Instance of 'BenchmarkBuilder' has no '_driver' member  [pylint/E1101] [5]
Total errors found: 1 in 35 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 3 Stephanie Lewis 2017-07-12 17:18:24 PDT
Comment on attachment 315308 [details]
Patch

This looks rubber stampable to me but I'd like Dewei to weigh in.
Comment 4 Build Bot 2017-07-12 18:36:03 PDT
Comment on attachment 315308 [details]
Patch

Attachment 315308 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/4110662

New failing tests:
storage/websql/execute-sql-rowsAffected.html
Comment 5 Build Bot 2017-07-12 18:36:05 PDT
Created attachment 315325 [details]
Archive of layout-test-results from ews122 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews122  Port: ios-simulator-wk2  Platform: Mac OS X 10.12.5
Comment 6 dewei_zhu 2017-07-17 17:38:21 PDT
Comment on attachment 315308 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=315308&action=review

> Tools/Scripts/webkitpy/benchmark_runner/benchmark_builder.py:44
> +            if self._driver == 'websocket':
> +                if 'websocket_benchmark_patch' in self._plan:
> +                    self._apply_patch(self._plan['websocket_benchmark_patch'])
> +            else:
> +                if 'webdriver_benchmark_patch' in self._plan:
> +                    self._apply_patch(self._plan['webdriver_benchmark_patch'])

It would be clearer:
patch_file_key = "{driver_name}_benchmark_patch".format(driver_name=self._driver)
if patch_file_key in self._plan:
    self._apply_patch(self._plan[patch_file_key])
Comment 7 Matthew Stewart 2017-07-18 19:35:37 PDT
Created attachment 315878 [details]
Add BenchmarkRunner patches with WebDriver support for each testing plan
Comment 8 dewei_zhu 2017-07-18 20:57:37 PDT
Comment on attachment 315878 [details]
Add BenchmarkRunner patches with WebDriver support for each testing plan

View in context: https://bugs.webkit.org/attachment.cgi?id=315878&action=review

> Tools/ChangeLog:6
> +        Reviewed by Dewei Zhu.

r=me
Comment 9 WebKit Commit Bot 2017-07-24 17:16:06 PDT
Comment on attachment 315878 [details]
Add BenchmarkRunner patches with WebDriver support for each testing plan

Rejecting attachment 315878 [details] from commit-queue.

Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-01', 'validate-changelog', '--check-oops', '--non-interactive', 315878, '--port=mac']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

Dewei Zhu found in /Volumes/Data/EWS/WebKit/Tools/ChangeLog does not appear to be a valid reviewer according to contributors.json.
/Volumes/Data/EWS/WebKit/Tools/ChangeLog neither lists a valid reviewer nor contains the string "Unreviewed" or "Rubber stamp" (case insensitive).

Full output: http://webkit-queues.webkit.org/results/4181291
Comment 10 Matthew Stewart 2017-07-24 18:01:54 PDT
Created attachment 316335 [details]
patch
Comment 11 WebKit Commit Bot 2017-07-24 18:57:07 PDT
The commit-queue encountered the following flaky tests while processing attachment 316335 [details]:

http/tests/security/cross-origin-xsl-BLOCKED.html bug 51054 (authors: abarth@webkit.org, jochen@chromium.org, and rniwa@webkit.org)
The commit-queue is continuing to process your patch.
Comment 12 WebKit Commit Bot 2017-07-24 18:58:32 PDT
Comment on attachment 316335 [details]
patch

Clearing flags on attachment: 316335

Committed r219854: <http://trac.webkit.org/changeset/219854>
Comment 13 WebKit Commit Bot 2017-07-24 18:58:34 PDT
All reviewed patches have been landed.  Closing bug.