Bug 159678 - REGRESSION(r203110): LayoutTest http/tests/preload/single_download_preload_runner.html timing out
Summary: REGRESSION(r203110): LayoutTest http/tests/preload/single_download_preload_ru...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-07-12 10:14 PDT by Ryan Haddad
Modified: 2017-01-25 14:53 PST (History)
7 users (show)

See Also:


Attachments
Patch (8.41 KB, patch)
2016-07-14 21:51 PDT, Yoav Weiss
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews114 for mac-yosemite (1.49 MB, application/zip)
2016-07-14 22:51 PDT, Build Bot
no flags Details
Patch (10.32 KB, patch)
2016-09-01 03:45 PDT, Yoav Weiss
no flags Details | Formatted Diff | Diff
Patch (10.40 KB, patch)
2016-09-06 06:02 PDT, Yoav Weiss
no flags Details | Formatted Diff | Diff
Patch (37.99 KB, patch)
2017-01-18 05:24 PST, Yoav Weiss
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews102 for mac-elcapitan (881.64 KB, application/zip)
2017-01-18 06:24 PST, Build Bot
no flags Details
Archive of layout-test-results from ews104 for mac-elcapitan-wk2 (951.97 KB, application/zip)
2017-01-18 06:27 PST, Build Bot
no flags Details
Archive of layout-test-results from ews113 for mac-elcapitan (1.69 MB, application/zip)
2017-01-18 06:33 PST, Build Bot
no flags Details
Archive of layout-test-results from ews121 for ios-simulator-wk2 (817.09 KB, application/zip)
2017-01-18 06:38 PST, Build Bot
no flags Details
Patch (37.92 KB, patch)
2017-01-18 07:20 PST, Yoav Weiss
no flags Details | Formatted Diff | Diff
Patch (37.89 KB, patch)
2017-01-18 23:58 PST, Yoav Weiss
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews102 for mac-elcapitan (718.25 KB, application/zip)
2017-01-19 02:59 PST, Build Bot
no flags Details
Patch (38.02 KB, patch)
2017-01-25 05:01 PST, Yoav Weiss
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ryan Haddad 2016-07-12 10:14:32 PDT
LayoutTest http/tests/preload/single_download_preload_runner.html timing out

https://build.webkit.org/results/Apple%20El%20Capitan%20Release%20WK2%20(Tests)/r203115%20(7565)/results.html

--- /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/http/tests/preload/single_download_preload_runner-expected.txt
+++ /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/http/tests/preload/single_download_preload_runner-actual.txt
@@ -1,5 +1,4 @@
 CONSOLE MESSAGE: line 10: <link rel=preload> must have a valid `as` value
+CONSOLE MESSAGE: line 39: TypeError: performance.getEntriesByType is not a function. (In 'performance.getEntriesByType("resource")', 'performance.getEntriesByType' is undefined)
+FAIL: Timed out waiting for notifyDone to be called
 
-
-PASS Makes sure that preloaded resources are not downloaded again when used 
-
Comment 1 Ryan Haddad 2016-07-12 10:18:24 PDT
Marked as a timeout in http://trac.webkit.org/projects/webkit/changeset/203117
Comment 2 Alexey Proskuryakov 2016-07-12 16:50:49 PDT
Yoav, is this feature supposed to be disabled by default when running layout tests?
Comment 3 Yoav Weiss 2016-07-13 22:50:47 PDT
(In reply to comment #2)
> Yoav, is this feature supposed to be disabled by default when running layout
> tests?

Both preload and resource timing features (which this test relies on) are supposed to be disabled by default, but I'm not sure why the fix to stop leaking runtime features caused this test to stop working. I'm looking into it.
Comment 4 Yoav Weiss 2016-07-14 21:51:32 PDT
Created attachment 283735 [details]
Patch
Comment 5 Build Bot 2016-07-14 22:51:00 PDT
Comment on attachment 283735 [details]
Patch

Attachment 283735 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/1683903

New failing tests:
http/tests/preload/single_download_preload.html
Comment 6 Build Bot 2016-07-14 22:51:04 PDT
Created attachment 283737 [details]
Archive of layout-test-results from ews114 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews114  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 7 Yoav Weiss 2016-09-01 03:45:52 PDT
Created attachment 287613 [details]
Patch
Comment 8 Alex Christensen 2016-09-01 08:39:42 PDT
Comment on attachment 287613 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=287613&action=review

This new test fails for me on Sierra.

> LayoutTests/http/tests/preload/single_download_preload.html:9
> +<script src="../w3c/resources/testharness.js"></script>
> +<script src="../w3c/resources/testharnessreport.js"></script>

This is probably bad.  Either this should be a w3c test, or we should make our own resources, or we shouldn't need these resources.
Comment 9 Yoav Weiss 2016-09-01 08:50:33 PDT
(In reply to comment #8)
> Comment on attachment 287613 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=287613&action=review
> 
> This new test fails for me on Sierra.

What's the failure output? I don't have Sierra to test on...

> 
> > LayoutTests/http/tests/preload/single_download_preload.html:9
> > +<script src="../w3c/resources/testharness.js"></script>
> > +<script src="../w3c/resources/testharnessreport.js"></script>
> 
> This is probably bad.  Either this should be a w3c test, or we should make
> our own resources, or we shouldn't need these resources.

There are many other tests that are using testharness.js (for which there are multiple copies)
Would using "/js-test-resources/testharness.js" be better?
Comment 10 Alex Christensen 2016-09-01 09:55:48 PDT
(In reply to comment #9)
> (In reply to comment #8)
> > Comment on attachment 287613 [details]
> > Patch
> > 
> > View in context:
> > https://bugs.webkit.org/attachment.cgi?id=287613&action=review
> > 
> > This new test fails for me on Sierra.
> 
> What's the failure output? I don't have Sierra to test on...
--- /Users/alexchristensen/code/OpenSource/WebKitBuild/Debug/layout-test-results/http/tests/preload/single_download_preload-expected.txt
+++ /Users/alexchristensen/code/OpenSource/WebKitBuild/Debug/layout-test-results/http/tests/preload/single_download_preload-actual.txt
@@ -1,5 +1,9 @@
 CONSOLE MESSAGE: line 21: <link rel=preload> must have a valid `as` value
  
 
-PASS Makes sure that preloaded resources are not downloaded again when used 
+FAIL Makes sure that preloaded resources are not downloaded again when used assert_equals: The number of resource downloads expected 12 but got 14(stack: assert@http://127.0.0.1:8000/w3c/resources/testharness.js:2060:37
+assert_equals@http://127.0.0.1:8000/w3c/resources/testharness.js:699:15
+http://127.0.0.1:8000/preload/single_download_preload.html:52:22
+step@http://127.0.0.1:8000/w3c/resources/testharness.js:1160:30
+http://127.0.0.1:8000/w3c/resources/testharness.js:1189:33)
 
> 
> > 
> > > LayoutTests/http/tests/preload/single_download_preload.html:9
> > > +<script src="../w3c/resources/testharness.js"></script>
> > > +<script src="../w3c/resources/testharnessreport.js"></script>
> > 
> > This is probably bad.  Either this should be a w3c test, or we should make
> > our own resources, or we shouldn't need these resources.
> 
> There are many other tests that are using testharness.js (for which there
> are multiple copies)
> Would using "/js-test-resources/testharness.js" be better?
Probably.
Comment 11 Yoav Weiss 2016-09-01 13:23:57 PDT
(In reply to comment #10)
> (In reply to comment #9)
> > (In reply to comment #8)
> > > Comment on attachment 287613 [details]
> > > Patch
> > > 
> > > View in context:
> > > https://bugs.webkit.org/attachment.cgi?id=287613&action=review
> > > 
> > > This new test fails for me on Sierra.
> > 
> > What's the failure output? I don't have Sierra to test on...
> ---
> /Users/alexchristensen/code/OpenSource/WebKitBuild/Debug/layout-test-results/
> http/tests/preload/single_download_preload-expected.txt
> +++
> /Users/alexchristensen/code/OpenSource/WebKitBuild/Debug/layout-test-results/
> http/tests/preload/single_download_preload-actual.txt
> @@ -1,5 +1,9 @@
>  CONSOLE MESSAGE: line 21: <link rel=preload> must have a valid `as` value
>   
>  
> -PASS Makes sure that preloaded resources are not downloaded again when used 
> +FAIL Makes sure that preloaded resources are not downloaded again when used
> assert_equals: The number of resource downloads expected 12 but got
> 14(stack: assert@http://127.0.0.1:8000/w3c/resources/testharness.js:2060:37
> +assert_equals@http://127.0.0.1:8000/w3c/resources/testharness.js:699:15
> +http://127.0.0.1:8000/preload/single_download_preload.html:52:22
> +step@http://127.0.0.1:8000/w3c/resources/testharness.js:1160:30
> +http://127.0.0.1:8000/w3c/resources/testharness.js:1189:33)

Hmm, that seems like a real issue, but I can't reproduce locally :/ Any ideas on what can be different in Sierra? (I'm on El-Capitan)
Comment 12 Yoav Weiss 2016-09-02 02:11:29 PDT
(In reply to comment #11)
> (In reply to comment #10)
> > (In reply to comment #9)
> > > (In reply to comment #8)
> > > > Comment on attachment 287613 [details]
> > > > Patch
> > > > 
> > > > View in context:
> > > > https://bugs.webkit.org/attachment.cgi?id=287613&action=review
> > > > 
> > > > This new test fails for me on Sierra.
> > > 
> > > What's the failure output? I don't have Sierra to test on...
> > ---
> > /Users/alexchristensen/code/OpenSource/WebKitBuild/Debug/layout-test-results/
> > http/tests/preload/single_download_preload-expected.txt
> > +++
> > /Users/alexchristensen/code/OpenSource/WebKitBuild/Debug/layout-test-results/
> > http/tests/preload/single_download_preload-actual.txt
> > @@ -1,5 +1,9 @@
> >  CONSOLE MESSAGE: line 21: <link rel=preload> must have a valid `as` value
> >   
> >  
> > -PASS Makes sure that preloaded resources are not downloaded again when used 
> > +FAIL Makes sure that preloaded resources are not downloaded again when used
> > assert_equals: The number of resource downloads expected 12 but got
> > 14(stack: assert@http://127.0.0.1:8000/w3c/resources/testharness.js:2060:37
> > +assert_equals@http://127.0.0.1:8000/w3c/resources/testharness.js:699:15
> > +http://127.0.0.1:8000/preload/single_download_preload.html:52:22
> > +step@http://127.0.0.1:8000/w3c/resources/testharness.js:1160:30
> > +http://127.0.0.1:8000/w3c/resources/testharness.js:1189:33)
> 
> Hmm, that seems like a real issue, but I can't reproduce locally :/ Any
> ideas on what can be different in Sierra? (I'm on El-Capitan)

I was able to reproduce locally. Seems like image resources are (sometimes) double downloading, which is what this test is supposed to protect against. I'll investigate and fix.
Comment 13 Yoav Weiss 2016-09-06 06:02:45 PDT
Created attachment 288021 [details]
Patch
Comment 14 Yoav Weiss 2016-09-06 06:07:15 PDT
Eventually the problem I saw locally wasn't a bug, but a problem in my directory structure (which led to missing images, which got evicted from memorycache).

I'm now failing to reproduce issues both on El Capitan as well as Sierra. At the same time, I changed file references and replaced the .ttf with a .woff

Can you try again and see if you still see the test failing?
Comment 15 Yoav Weiss 2017-01-18 05:24:22 PST
Created attachment 299134 [details]
Patch
Comment 16 Yoav Weiss 2017-01-18 05:25:34 PST
Updated to test to avoid the timeouts as well as to make it more consistent and clear. PTAL
Comment 17 Build Bot 2017-01-18 06:24:37 PST
Comment on attachment 299134 [details]
Patch

Attachment 299134 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/2909523

New failing tests:
http/tests/preload/single_download_preload.html
Comment 18 Build Bot 2017-01-18 06:24:41 PST
Created attachment 299136 [details]
Archive of layout-test-results from ews102 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews102  Port: mac-elcapitan  Platform: Mac OS X 10.11.6
Comment 19 Build Bot 2017-01-18 06:27:19 PST
Comment on attachment 299134 [details]
Patch

Attachment 299134 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/2909524

New failing tests:
http/tests/preload/single_download_preload.html
Comment 20 Build Bot 2017-01-18 06:27:22 PST
Created attachment 299137 [details]
Archive of layout-test-results from ews104 for mac-elcapitan-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104  Port: mac-elcapitan-wk2  Platform: Mac OS X 10.11.6
Comment 21 Build Bot 2017-01-18 06:33:20 PST
Comment on attachment 299134 [details]
Patch

Attachment 299134 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/2909527

New failing tests:
http/tests/preload/single_download_preload.html
Comment 22 Build Bot 2017-01-18 06:33:24 PST
Created attachment 299138 [details]
Archive of layout-test-results from ews113 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews113  Port: mac-elcapitan  Platform: Mac OS X 10.11.6
Comment 23 Build Bot 2017-01-18 06:38:17 PST
Comment on attachment 299134 [details]
Patch

Attachment 299134 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/2909533

New failing tests:
http/tests/preload/single_download_preload.html
Comment 24 Build Bot 2017-01-18 06:38:20 PST
Created attachment 299139 [details]
Archive of layout-test-results from ews121 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews121  Port: ios-simulator-wk2  Platform: Mac OS X 10.11.6
Comment 25 Yoav Weiss 2017-01-18 07:20:09 PST
Created attachment 299141 [details]
Patch
Comment 26 Alex Christensen 2017-01-18 12:46:17 PST
Comment on attachment 299141 [details]
Patch

Sure, let's give this a shot and see if it fixes the timeouts.
Comment 27 WebKit Commit Bot 2017-01-18 13:27:19 PST
Comment on attachment 299141 [details]
Patch

Clearing flags on attachment: 299141

Committed r210881: <http://trac.webkit.org/changeset/210881>
Comment 28 WebKit Commit Bot 2017-01-18 13:27:25 PST
All reviewed patches have been landed.  Closing bug.
Comment 29 Ryan Haddad 2017-01-18 15:46:40 PST
It looks like the test fails now:

--- /Volumes/Data/slave/sierra-release-tests-wk1/build/layout-test-results/http/tests/preload/single_download_preload-expected.txt
+++ /Volumes/Data/slave/sierra-release-tests-wk1/build/layout-test-results/http/tests/preload/single_download_preload-actual.txt
@@ -1,5 +1,5 @@
 CONSOLE MESSAGE: line 21: <link rel=preload> must have a valid `as` value
  
 
-PASS Makes sure that preloaded resources are not downloaded again when used 
+FAIL Makes sure that preloaded resources are not downloaded again when used assert_equals: http://127.0.0.1:8000/resources/test.mp4 expected 1 but got 2
 
https://build.webkit.org/results/Apple%20Sierra%20Release%20WK1%20(Tests)/r210882%20(2912)/results.html
Comment 30 Ryan Haddad 2017-01-18 15:52:35 PST
Reverted r210881 for reason:

This test is failing on macOS.

Committed r210894: <http://trac.webkit.org/changeset/210894>
Comment 31 Yoav Weiss 2017-01-18 23:35:36 PST
OK, I see the failures locally as well. Looks like video and audio (flakily?) send multiple partial requests. I'll remove those verifications from the test and add a FIXME instead
Comment 32 Yoav Weiss 2017-01-18 23:58:02 PST
Created attachment 299236 [details]
Patch
Comment 33 Yoav Weiss 2017-01-18 23:59:14 PST
Removed the video/audio verification that was causing the flakiness. PTAL
Comment 34 Build Bot 2017-01-19 02:58:57 PST
Comment on attachment 299236 [details]
Patch

Attachment 299236 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/2914173

New failing tests:
imported/w3c/web-platform-tests/html/browsers/history/the-location-interface/location-protocol-setter-non-broken.html
Comment 35 Build Bot 2017-01-19 02:59:01 PST
Created attachment 299242 [details]
Archive of layout-test-results from ews102 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews102  Port: mac-elcapitan  Platform: Mac OS X 10.11.6
Comment 36 Yoav Weiss 2017-01-25 05:01:16 PST
Created attachment 299691 [details]
Patch
Comment 37 WebKit Commit Bot 2017-01-25 14:53:13 PST
Comment on attachment 299691 [details]
Patch

Clearing flags on attachment: 299691

Committed r211169: <http://trac.webkit.org/changeset/211169>
Comment 38 WebKit Commit Bot 2017-01-25 14:53:21 PST
All reviewed patches have been landed.  Closing bug.