Bug 188280 - Explain test name matching in run-api-tests help
Summary: Explain test name matching in run-api-tests help
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Jonathan Bedard
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-08-02 15:24 PDT by Jonathan Bedard
Modified: 2018-08-23 09:40 PDT (History)
8 users (show)

See Also:


Attachments
Patch (1.86 KB, patch)
2018-08-02 15:43 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews205 for win-future (12.88 MB, application/zip)
2018-08-03 02:33 PDT, EWS Watchlist
no flags Details
Patch (4.35 KB, patch)
2018-08-20 15:54 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews206 for win-future (12.91 MB, application/zip)
2018-08-21 17:51 PDT, EWS Watchlist
no flags Details
Patch for landing (4.44 KB, patch)
2018-08-23 09:00 PDT, Jonathan Bedard
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Bedard 2018-08-02 15:24:35 PDT
After discussion on <https://bugs.webkit.org/show_bug.cgi?id=187893>, it seems that the help message in run-api-tests does not adequately explain how test matching works.
Comment 1 Jonathan Bedard 2018-08-02 15:43:52 PDT
Created attachment 346424 [details]
Patch
Comment 2 Daniel Bates 2018-08-02 19:23:44 PDT
Comment on attachment 346424 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=346424&action=review

> Tools/Scripts/webkitpy/api_tests/run_api_tests.py:128
> +                    'format <binary>.<suite>.<test> or <suite>.<test>. Note that in the case where a binary is not ' +

What are the binaries?
Comment 3 EWS Watchlist 2018-08-03 02:33:19 PDT
Comment on attachment 346424 [details]
Patch

Attachment 346424 [details] did not pass win-ews (win):
Output: https://webkit-queues.webkit.org/results/8747593

New failing tests:
imported/blink/transitions/unprefixed-transform.html
Comment 4 EWS Watchlist 2018-08-03 02:33:30 PDT
Created attachment 346466 [details]
Archive of layout-test-results from ews205 for win-future

The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews205  Port: win-future  Platform: CYGWIN_NT-6.1-2.9.0-0.318-5-3-x86_64-64bit
Comment 5 Jonathan Bedard 2018-08-03 16:02:33 PDT
(In reply to Daniel Bates from comment #2)
> Comment on attachment 346424 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=346424&action=review
> 
> > Tools/Scripts/webkitpy/api_tests/run_api_tests.py:128
> > +                    'format <binary>.<suite>.<test> or <suite>.<test>. Note that in the case where a binary is not ' +
> 
> What are the binaries?

If you would like the binary names in the help message, can we land <https://bugs.webkit.org/show_bug.cgi?id=187893> first so I don't have to duplicate them? That way the help message could grab from the static arrays, ensuring that if new binaries are added, the help message would automatically pick them up.
Comment 6 Jonathan Bedard 2018-08-20 15:54:20 PDT
Created attachment 347553 [details]
Patch
Comment 7 EWS Watchlist 2018-08-21 17:51:24 PDT
Comment on attachment 347553 [details]
Patch

Attachment 347553 [details] did not pass win-ews (win):
Output: https://webkit-queues.webkit.org/results/8937142

New failing tests:
http/tests/security/canvas-remote-read-remote-video-blocked-no-crossorigin.html
Comment 8 EWS Watchlist 2018-08-21 17:51:36 PDT
Created attachment 347744 [details]
Archive of layout-test-results from ews206 for win-future

The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews206  Port: win-future  Platform: CYGWIN_NT-6.1-2.9.0-0.318-5-3-x86_64-64bit
Comment 9 Daniel Bates 2018-08-22 22:50:00 PDT
Comment on attachment 347553 [details]
Patch

I don’t like this feature and would prefer to remove it than advertise it. If you are going to land this patch then please at least re-read and spell and grammar check what you wrote because it contains obvious errors and does not read well.
Comment 10 Jonathan Bedard 2018-08-23 09:00:49 PDT
Created attachment 347925 [details]
Patch for landing
Comment 11 WebKit Commit Bot 2018-08-23 09:40:00 PDT
Comment on attachment 347925 [details]
Patch for landing

Clearing flags on attachment: 347925

Committed r235227: <https://trac.webkit.org/changeset/235227>
Comment 12 WebKit Commit Bot 2018-08-23 09:40:02 PDT
All reviewed patches have been landed.  Closing bug.