Bug 174056

Summary: WIP - RunBenchmark WebDriver testing
Product: WebKit Reporter: Matthew Stewart <matthew_r_stewart>
Component: New BugsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED FIXED    
Severity: Normal CC: bburg, buildbot, dean_johnson, dewei_zhu, glenn, slewis
Priority: P2    
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Bug Depends on: 174331, 174390, 174443, 174445    
Bug Blocks:    
Attachments:
Description Flags
Patch slewis: review-

Description Matthew Stewart 2017-06-30 16:57:55 PDT
WIP - RunBenchmark WebDriver testing
Comment 1 Matthew Stewart 2017-06-30 17:02:20 PDT
Created attachment 314327 [details]
Patch
Comment 2 Build Bot 2017-06-30 17:05:50 PDT
Attachment 314327 [details] did not pass style-queue:


ERROR: Tools/Scripts/webkitpy/webdriver_benchmark_runner/browser_driver/osx_safari_driver.py:36:  [OSXSafariDriver.close_browsers] Undefined variable 'sys'  [pylint/E0602] [5]
ERROR: Tools/Scripts/webkitpy/webdriver_benchmark_runner/browser_driver/osx_safari_driver.py:36:  [OSXSafariDriver.close_browsers] Instance of 'OSXSafariDriver' has no '_process' member  [pylint/E1101] [5]
ERROR: Tools/Scripts/webkitpy/webdriver_benchmark_runner/utils.py:51:  [write_defaults] No name 'NSUserDefaults' in module 'Foundation'  [pylint/E0611] [5]
ERROR: Tools/Scripts/webkitpy/webdriver_benchmark_runner/browser_driver/linux_browser_driver.py:92:  [LinuxBrowserDriver._launch_driver] Method should have "self" as first argument  [pylint/E0213] [5]
ERROR: Tools/Scripts/webkitpy/webdriver_benchmark_runner/browser_driver/osx_chrome_driver.py:19:  missing whitespace around operator  [pep8/E225] [5]
ERROR: Tools/Scripts/webkitpy/webdriver_benchmark_runner/browser_driver/osx_chrome_driver.py:35:  expected 2 blank lines, found 1  [pep8/E302] [5]
ERROR: Tools/Scripts/webkitpy/webdriver_benchmark_runner/browser_driver/osx_chrome_driver.py:38:  missing whitespace around operator  [pep8/E225] [5]
ERROR: Tools/Scripts/webkitpy/webdriver_benchmark_runner/browser_driver/osx_firefox_driver.py:19:  missing whitespace around operator  [pep8/E225] [5]
ERROR: Tools/Scripts/webkitpy/webdriver_benchmark_runner/browser_driver/osx_firefox_driver.py:31:  expected 2 blank lines, found 1  [pep8/E302] [5]
ERROR: Tools/Scripts/webkitpy/webdriver_benchmark_runner/browser_driver/osx_firefox_driver.py:34:  missing whitespace around operator  [pep8/E225] [5]
ERROR: Tools/Scripts/webkitpy/webdriver_benchmark_runner/browser_driver/osx_browser_driver.py:20:  [OSXBrowserDriver.prepare_env] No name 'CGWarpMouseCursorPosition' in module 'Quartz'  [pylint/E0611] [5]
ERROR: Tools/Scripts/webkitpy/webdriver_benchmark_runner/browser_driver/osx_browser_driver.py:46:  [OSXBrowserDriver._screen_size] No name 'NSScreen' in module 'AppKit'  [pylint/E0611] [5]
Total errors found: 12 in 44 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 3 dewei_zhu 2017-06-30 17:58:19 PDT
Comment on attachment 314327 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=314327&action=review

> Tools/ChangeLog:7
> +

A lot of files are duplicated. We should try to figure out a way to reuse the code from run-benchmark.

> Tools/Scripts/webkitpy/webdriver_benchmark_runner/README.md:1
> +# Benchmark Runner 

Looks like this file need to be updated.

> Tools/Scripts/webkitpy/webdriver_benchmark_runner/benchmark_runner.py:19
> +'''from selenium import webdriver
> +from selenium.webdriver.chrome.options import Options'''

We should remove commented code.

> Tools/Scripts/webkitpy/webdriver_benchmark_runner/browser_driver/linux_browser_driver.py:32
> +from selenium import webdriver

Is this used?
Comment 4 BJ Burg 2017-07-03 10:44:08 PDT
Comment on attachment 314327 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=314327&action=review

> Tools/Scripts/webkitpy/webdriver_benchmark_runner/benchmark_runner.py:74
> +            url = 'file://' + web_root + '/' + self._plan_name + '/' + test_file

This would look a lot better using a template/format string.

> Tools/Scripts/webkitpy/webdriver_benchmark_runner/browser_driver/osx_safari_driver.py:30
> +        self._launch_driver(build_dir=browser_build_path, app_name="Safari.app", url=url, args=["-HomePage", "about:blank", "-WarnAboutFraudulentWebsites", "0", "-ExtensionsEnabled", "0", "-ShowStatusBar", "0", "-NewWindowBehavior", "1", "-NewTabBehavior", "1"], driver=driver)

If these are needed to avoid hangs while running WebDriver tests, you should file a bug against safaridriver. Hangs should never happen during a WebDriver test.

> Tools/Scripts/webkitpy/webdriver_benchmark_runner/browser_driver/osx_safari_driver.py:39
> +    def _maximize_window(cls):

You can do this via WebDriver API. Why is this needed?