Bug 189903

Summary: Import WPT resource-timing tests
Product: WebKit Reporter: Yoav Weiss <yoav>
Component: New BugsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED FIXED    
Severity: Normal CC: achristensen, commit-queue, ews-watchlist, joepeck, realdawei, rniwa, ryanhaddad, tsavell, webkit-bug-importer, youennf
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch
none
Archive of layout-test-results from ews100 for mac-sierra
none
Archive of layout-test-results from ews104 for mac-sierra-wk2
none
Archive of layout-test-results from ews115 for mac-sierra
none
Patch
none
Archive of layout-test-results from ews103 for mac-sierra
none
Patch
none
Archive of layout-test-results from ews103 for mac-sierra
none
Patch
none
Patch none

Description Yoav Weiss 2018-09-23 18:32:23 PDT
Import WPT resource-timing tests
Comment 1 Yoav Weiss 2018-09-23 18:33:57 PDT
Created attachment 350596 [details]
Patch
Comment 2 EWS Watchlist 2018-09-23 19:30:19 PDT
Comment on attachment 350596 [details]
Patch

Attachment 350596 [details] did not pass mac-ews (mac):
Output: https://webkit-queues.webkit.org/results/9325284

Number of test failures exceeded the failure limit.
Comment 3 EWS Watchlist 2018-09-23 19:30:21 PDT
Created attachment 350597 [details]
Archive of layout-test-results from ews100 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews100  Port: mac-sierra  Platform: Mac OS X 10.12.6
Comment 4 EWS Watchlist 2018-09-23 19:42:02 PDT
Comment on attachment 350596 [details]
Patch

Attachment 350596 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/9325302

Number of test failures exceeded the failure limit.
Comment 5 EWS Watchlist 2018-09-23 19:42:04 PDT
Created attachment 350598 [details]
Archive of layout-test-results from ews104 for mac-sierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104  Port: mac-sierra-wk2  Platform: Mac OS X 10.12.6
Comment 6 EWS Watchlist 2018-09-23 20:01:23 PDT
Comment on attachment 350596 [details]
Patch

Attachment 350596 [details] did not pass mac-debug-ews (mac):
Output: https://webkit-queues.webkit.org/results/9325301

Number of test failures exceeded the failure limit.
Comment 7 EWS Watchlist 2018-09-23 20:01:25 PDT
Created attachment 350600 [details]
Archive of layout-test-results from ews115 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews115  Port: mac-sierra  Platform: Mac OS X 10.12.6
Comment 8 Yoav Weiss 2018-09-23 20:58:12 PDT
Created attachment 350602 [details]
Patch
Comment 9 EWS Watchlist 2018-09-23 22:03:39 PDT
Comment on attachment 350602 [details]
Patch

Attachment 350602 [details] did not pass mac-ews (mac):
Output: https://webkit-queues.webkit.org/results/9326203

New failing tests:
imported/w3c/web-platform-tests/resource-timing/resource_timing.worker.html
Comment 10 EWS Watchlist 2018-09-23 22:03:41 PDT
Created attachment 350604 [details]
Archive of layout-test-results from ews103 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103  Port: mac-sierra  Platform: Mac OS X 10.12.6
Comment 11 Yoav Weiss 2018-09-23 22:19:56 PDT
Created attachment 350606 [details]
Patch
Comment 12 EWS Watchlist 2018-09-23 23:24:29 PDT
Comment on attachment 350606 [details]
Patch

Attachment 350606 [details] did not pass mac-ews (mac):
Output: https://webkit-queues.webkit.org/results/9326932

New failing tests:
imported/w3c/web-platform-tests/resource-timing/resource_timing_store_and_clear_during_callback.html
Comment 13 EWS Watchlist 2018-09-23 23:24:31 PDT
Created attachment 350611 [details]
Archive of layout-test-results from ews103 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103  Port: mac-sierra  Platform: Mac OS X 10.12.6
Comment 14 Yoav Weiss 2018-09-23 23:44:58 PDT
Created attachment 350613 [details]
Patch
Comment 15 Yoav Weiss 2018-09-23 23:46:54 PDT
Created attachment 350614 [details]
Patch
Comment 16 Yoav Weiss 2018-09-24 08:50:47 PDT
Comment on attachment 350614 [details]
Patch

Thanks for reviewing! :)
Comment 17 WebKit Commit Bot 2018-09-24 08:58:51 PDT
Comment on attachment 350614 [details]
Patch

Clearing flags on attachment: 350614

Committed r236402: <https://trac.webkit.org/changeset/236402>
Comment 18 WebKit Commit Bot 2018-09-24 08:58:53 PDT
All reviewed patches have been landed.  Closing bug.
Comment 19 Radar WebKit Bug Importer 2018-09-24 09:00:20 PDT
<rdar://problem/44727969>
Comment 20 Dawei Fenton (:realdawei) 2018-09-24 10:57:14 PDT
(In reply to WebKit Commit Bot from comment #17)
> Comment on attachment 350614 [details]
> Patch
> 
> Clearing flags on attachment: 350614
> 
> Committed r236402: <https://trac.webkit.org/changeset/236402>

The following test has been failing on mac WK1 since it was added:

imported/w3c/web-platform-tests/resource-timing/resource_timing.worker.html

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fresource-timing%2Fresource_timing.worker.html
Comment 21 Dawei Fenton (:realdawei) 2018-09-24 11:27:40 PDT
(In reply to Dawei Fenton (:realdawei) from comment #20)
> (In reply to WebKit Commit Bot from comment #17)
> > Comment on attachment 350614 [details]
> > Patch
> > 
> > Clearing flags on attachment: 350614
> > 
> > Committed r236402: <https://trac.webkit.org/changeset/236402>
> 
> The following test has been failing on mac WK1 since it was added:
> 
> imported/w3c/web-platform-tests/resource-timing/resource_timing.worker.html
> 
> https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.
> html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fresource-
> timing%2Fresource_timing.worker.html

Sample diff:
--- /Volumes/Data/slave/highsierra-release-tests-wk1/build/layout-test-results/imported/w3c/web-platform-tests/resource-timing/resource_timing.worker-expected.txt
+++ /Volumes/Data/slave/highsierra-release-tests-wk1/build/layout-test-results/imported/w3c/web-platform-tests/resource-timing/resource_timing.worker-actual.txt
@@ -1,3 +1,5 @@
 
-PASS Performance Resouce Entries in workers 
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT Performance Resouce Entries in workers Test timed out
+
Comment 22 Yoav Weiss 2018-09-25 00:33:49 PDT
Yeah, I've added that test to LayoutTests/platform/mac/TestExpectations and opened webkit.org/b/189908 on it.

Is that not enough to exclude it from mac WK1 test bots?
Comment 23 Ryan Haddad 2018-09-25 09:54:47 PDT
(In reply to Yoav Weiss from comment #22)
> Yeah, I've added that test to LayoutTests/platform/mac/TestExpectations and
> opened webkit.org/b/189908 on it.
> 
> Is that not enough to exclude it from mac WK1 test bots?
It looks like the issue is that the test is hitting a timeout in the WPT harness, so it ends up being reported as a text failure rather than a timeout.

I'll update the expectations.