Bug 159678 – REGRESSION(r203110): LayoutTest http/tests/preload/single_download_preload_runner.html timing out

• Component: New Bugs
• Status: RESOLVED FIXED

[Ryan Haddad]

LayoutTest http/tests/preload/single_download_preload_runner.html timing out

https://build.webkit.org/results/Apple%20El%20Capitan%20Release%20WK2%20(Tests)/r203115%20(7565)/results.html

--- /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/http/tests/preload/single_download_preload_runner-expected.txt
+++ /Volumes/Data/slave/elcapitan-release-tests-wk2/build/layout-test-results/http/tests/preload/single_download_preload_runner-actual.txt
@@ -1,5 +1,4 @@
 CONSOLE MESSAGE: line 10: <link rel=preload> must have a valid `as` value
+CONSOLE MESSAGE: line 39: TypeError: performance.getEntriesByType is not a function. (In 'performance.getEntriesByType("resource")', 'performance.getEntriesByType' is undefined)
+FAIL: Timed out waiting for notifyDone to be called
 
-
-PASS Makes sure that preloaded resources are not downloaded again when used 
-

[Ryan Haddad]

Marked as a timeout in http://trac.webkit.org/projects/webkit/changeset/203117

[Alexey Proskuryakov]

Yoav, is this feature supposed to be disabled by default when running layout tests?

[Yoav Weiss]

(In reply to comment #2)
> Yoav, is this feature supposed to be disabled by default when running layout
> tests?

Both preload and resource timing features (which this test relies on) are supposed to be disabled by default, but I'm not sure why the fix to stop leaking runtime features caused this test to stop working. I'm looking into it.

[Yoav Weiss]

Created attachment 283735 [details]
Patch

[Build Bot]

Comment on attachment 283735 [details]
Patch

Attachment 283735 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/1683903

New failing tests:
http/tests/preload/single_download_preload.html

[Build Bot]

Created attachment 283737 [details]
Archive of layout-test-results from ews114 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews114  Port: mac-yosemite  Platform: Mac OS X 10.10.5

[Yoav Weiss]

Created attachment 287613 [details]
Patch

[Alex Christensen]

Comment on attachment 287613 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=287613&action=review

This new test fails for me on Sierra.

> LayoutTests/http/tests/preload/single_download_preload.html:9
> +<script src="../w3c/resources/testharness.js"></script>
> +<script src="../w3c/resources/testharnessreport.js"></script>

This is probably bad.  Either this should be a w3c test, or we should make our own resources, or we shouldn't need these resources.

[Yoav Weiss]

(In reply to comment #8)
> Comment on attachment 287613 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=287613&action=review
> 
> This new test fails for me on Sierra.

What's the failure output? I don't have Sierra to test on...

> 
> > LayoutTests/http/tests/preload/single_download_preload.html:9
> > +<script src="../w3c/resources/testharness.js"></script>
> > +<script src="../w3c/resources/testharnessreport.js"></script>
> 
> This is probably bad.  Either this should be a w3c test, or we should make
> our own resources, or we shouldn't need these resources.

There are many other tests that are using testharness.js (for which there are multiple copies)
Would using "/js-test-resources/testharness.js" be better?

[Alex Christensen]

(In reply to comment #9)
> (In reply to comment #8)
> > Comment on attachment 287613 [details]
> > Patch
> > 
> > View in context:
> > https://bugs.webkit.org/attachment.cgi?id=287613&action=review
> > 
> > This new test fails for me on Sierra.
> 
> What's the failure output? I don't have Sierra to test on...
--- /Users/alexchristensen/code/OpenSource/WebKitBuild/Debug/layout-test-results/http/tests/preload/single_download_preload-expected.txt
+++ /Users/alexchristensen/code/OpenSource/WebKitBuild/Debug/layout-test-results/http/tests/preload/single_download_preload-actual.txt
@@ -1,5 +1,9 @@
 CONSOLE MESSAGE: line 21: <link rel=preload> must have a valid `as` value
  
 
-PASS Makes sure that preloaded resources are not downloaded again when used 
+FAIL Makes sure that preloaded resources are not downloaded again when used assert_equals: The number of resource downloads expected 12 but got 14(stack: assert@http://127.0.0.1:8000/w3c/resources/testharness.js:2060:37
+assert_equals@http://127.0.0.1:8000/w3c/resources/testharness.js:699:15
+http://127.0.0.1:8000/preload/single_download_preload.html:52:22
+step@http://127.0.0.1:8000/w3c/resources/testharness.js:1160:30
+http://127.0.0.1:8000/w3c/resources/testharness.js:1189:33)
 
> 
> > 
> > > LayoutTests/http/tests/preload/single_download_preload.html:9
> > > +<script src="../w3c/resources/testharness.js"></script>
> > > +<script src="../w3c/resources/testharnessreport.js"></script>
> > 
> > This is probably bad.  Either this should be a w3c test, or we should make
> > our own resources, or we shouldn't need these resources.
> 
> There are many other tests that are using testharness.js (for which there
> are multiple copies)
> Would using "/js-test-resources/testharness.js" be better?
Probably.

[Yoav Weiss]

(In reply to comment #10)
> (In reply to comment #9)
> > (In reply to comment #8)
> > > Comment on attachment 287613 [details]
> > > Patch
> > > 
> > > View in context:
> > > https://bugs.webkit.org/attachment.cgi?id=287613&action=review
> > > 
> > > This new test fails for me on Sierra.
> > 
> > What's the failure output? I don't have Sierra to test on...
> ---
> /Users/alexchristensen/code/OpenSource/WebKitBuild/Debug/layout-test-results/
> http/tests/preload/single_download_preload-expected.txt
> +++
> /Users/alexchristensen/code/OpenSource/WebKitBuild/Debug/layout-test-results/
> http/tests/preload/single_download_preload-actual.txt
> @@ -1,5 +1,9 @@
>  CONSOLE MESSAGE: line 21: <link rel=preload> must have a valid `as` value
>   
>  
> -PASS Makes sure that preloaded resources are not downloaded again when used 
> +FAIL Makes sure that preloaded resources are not downloaded again when used
> assert_equals: The number of resource downloads expected 12 but got
> 14(stack: assert@http://127.0.0.1:8000/w3c/resources/testharness.js:2060:37
> +assert_equals@http://127.0.0.1:8000/w3c/resources/testharness.js:699:15
> +http://127.0.0.1:8000/preload/single_download_preload.html:52:22
> +step@http://127.0.0.1:8000/w3c/resources/testharness.js:1160:30
> +http://127.0.0.1:8000/w3c/resources/testharness.js:1189:33)

Hmm, that seems like a real issue, but I can't reproduce locally :/ Any ideas on what can be different in Sierra? (I'm on El-Capitan)

[Yoav Weiss]

(In reply to comment #11)
> (In reply to comment #10)
> > (In reply to comment #9)
> > > (In reply to comment #8)
> > > > Comment on attachment 287613 [details]
> > > > Patch
> > > > 
> > > > View in context:
> > > > https://bugs.webkit.org/attachment.cgi?id=287613&action=review
> > > > 
> > > > This new test fails for me on Sierra.
> > > 
> > > What's the failure output? I don't have Sierra to test on...
> > ---
> > /Users/alexchristensen/code/OpenSource/WebKitBuild/Debug/layout-test-results/
> > http/tests/preload/single_download_preload-expected.txt
> > +++
> > /Users/alexchristensen/code/OpenSource/WebKitBuild/Debug/layout-test-results/
> > http/tests/preload/single_download_preload-actual.txt
> > @@ -1,5 +1,9 @@
> >  CONSOLE MESSAGE: line 21: <link rel=preload> must have a valid `as` value
> >   
> >  
> > -PASS Makes sure that preloaded resources are not downloaded again when used 
> > +FAIL Makes sure that preloaded resources are not downloaded again when used
> > assert_equals: The number of resource downloads expected 12 but got
> > 14(stack: assert@http://127.0.0.1:8000/w3c/resources/testharness.js:2060:37
> > +assert_equals@http://127.0.0.1:8000/w3c/resources/testharness.js:699:15
> > +http://127.0.0.1:8000/preload/single_download_preload.html:52:22
> > +step@http://127.0.0.1:8000/w3c/resources/testharness.js:1160:30
> > +http://127.0.0.1:8000/w3c/resources/testharness.js:1189:33)
> 
> Hmm, that seems like a real issue, but I can't reproduce locally :/ Any
> ideas on what can be different in Sierra? (I'm on El-Capitan)

I was able to reproduce locally. Seems like image resources are (sometimes) double downloading, which is what this test is supposed to protect against. I'll investigate and fix.

[Yoav Weiss]

Created attachment 288021 [details]
Patch

[Yoav Weiss]

Eventually the problem I saw locally wasn't a bug, but a problem in my directory structure (which led to missing images, which got evicted from memorycache).

I'm now failing to reproduce issues both on El Capitan as well as Sierra. At the same time, I changed file references and replaced the .ttf with a .woff

Can you try again and see if you still see the test failing?

[Yoav Weiss]

Created attachment 299134 [details]
Patch

[Yoav Weiss]

Updated to test to avoid the timeouts as well as to make it more consistent and clear. PTAL

[Build Bot]

Comment on attachment 299134 [details]
Patch

Attachment 299134 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/2909523

New failing tests:
http/tests/preload/single_download_preload.html

[Build Bot]

Created attachment 299136 [details]
Archive of layout-test-results from ews102 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews102  Port: mac-elcapitan  Platform: Mac OS X 10.11.6

[Build Bot]

Comment on attachment 299134 [details]
Patch

Attachment 299134 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/2909524

New failing tests:
http/tests/preload/single_download_preload.html

[Build Bot]

Created attachment 299137 [details]
Archive of layout-test-results from ews104 for mac-elcapitan-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104  Port: mac-elcapitan-wk2  Platform: Mac OS X 10.11.6

[Build Bot]

Comment on attachment 299134 [details]
Patch

Attachment 299134 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/2909527

New failing tests:
http/tests/preload/single_download_preload.html

[Build Bot]

Created attachment 299138 [details]
Archive of layout-test-results from ews113 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews113  Port: mac-elcapitan  Platform: Mac OS X 10.11.6

[Build Bot]

Comment on attachment 299134 [details]
Patch

Attachment 299134 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/2909533

New failing tests:
http/tests/preload/single_download_preload.html

[Build Bot]

Created attachment 299139 [details]
Archive of layout-test-results from ews121 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews121  Port: ios-simulator-wk2  Platform: Mac OS X 10.11.6

[Yoav Weiss]

Created attachment 299141 [details]
Patch

[Alex Christensen]

Comment on attachment 299141 [details]
Patch

Sure, let's give this a shot and see if it fixes the timeouts.

[WebKit Commit Bot]

Comment on attachment 299141 [details]
Patch

Clearing flags on attachment: 299141

Committed r210881: <http://trac.webkit.org/changeset/210881>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Ryan Haddad]

It looks like the test fails now:

--- /Volumes/Data/slave/sierra-release-tests-wk1/build/layout-test-results/http/tests/preload/single_download_preload-expected.txt
+++ /Volumes/Data/slave/sierra-release-tests-wk1/build/layout-test-results/http/tests/preload/single_download_preload-actual.txt
@@ -1,5 +1,5 @@
 CONSOLE MESSAGE: line 21: <link rel=preload> must have a valid `as` value
  
 
-PASS Makes sure that preloaded resources are not downloaded again when used 
+FAIL Makes sure that preloaded resources are not downloaded again when used assert_equals: http://127.0.0.1:8000/resources/test.mp4 expected 1 but got 2
 
https://build.webkit.org/results/Apple%20Sierra%20Release%20WK1%20(Tests)/r210882%20(2912)/results.html

[Ryan Haddad]

Reverted r210881 for reason:

This test is failing on macOS.

Committed r210894: <http://trac.webkit.org/changeset/210894>

[Yoav Weiss]

OK, I see the failures locally as well. Looks like video and audio (flakily?) send multiple partial requests. I'll remove those verifications from the test and add a FIXME instead

[Yoav Weiss]

Created attachment 299236 [details]
Patch

[Yoav Weiss]

Removed the video/audio verification that was causing the flakiness. PTAL

[Build Bot]

Comment on attachment 299236 [details]
Patch

Attachment 299236 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/2914173

New failing tests:
imported/w3c/web-platform-tests/html/browsers/history/the-location-interface/location-protocol-setter-non-broken.html

[Build Bot]

Created attachment 299242 [details]
Archive of layout-test-results from ews102 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews102  Port: mac-elcapitan  Platform: Mac OS X 10.11.6

[Yoav Weiss]

Created attachment 299691 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 299691 [details]
Patch

Clearing flags on attachment: 299691

Committed r211169: <http://trac.webkit.org/changeset/211169>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.