163362 2016-10-12 15:13:31 -0700 REGRESSION (r207162): [ios-simulator] LayoutTest platform/ios-simulator/ios/plugin/youtube-flash-plugin-iframe.html became more flaky 2017-12-13 12:55:07 -0800 1 1 1 Unclassified WebKit New Bugs WebKit Nightly Build Unspecified Unspecified NEW https://bugs.webkit.org/show_bug.cgi?id=163127 InRadar P2 Normal --- 1 ryanhaddad webkit-unassigned achristensen aestes ap cdumez dbates jlewis3 rniwa webkit-bug-importer oldest_to_newest 1239532 0 ryanhaddad 2016-10-12 15:13:31 -0700 LayoutTest platform/ios-simulator/ios/plugin/youtube-flash-plugin-iframe.html failing https://build.webkit.org/results/Apple%20iOS%2010%20Simulator%20Debug%20WK2%20(Tests)/r207162%20(707)/results.html http://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=platform%2Fios-simulator%2Fios%2Fplugin%2Fyoutube-flash-plugin-iframe.html --- /Volumes/Data/slave/ios-simulator-10-debug-tests-wk2/build/layout-test-results/platform/ios-simulator/ios/plugin/youtube-flash-plugin-iframe-expected.txt +++ /Volumes/Data/slave/ios-simulator-10-debug-tests-wk2/build/layout-test-results/platform/ios-simulator/ios/plugin/youtube-flash-plugin-iframe-actual.txt @@ -2,7 +2,6 @@ Blocked access to external URL http://www.youtube.com/embed/N0gb9v4LI4o Blocked access to external URL http://www.youtube.com/embed/N0gb9v4LI4o?version=3&hl=en_US Blocked access to external URL http://www.youtube.com/embed/N0gb9v4LI4o?version=3&hl=en_US -Blocked access to external URL http://www.youtube.com/embed/N0gb9v4LI4o Test transforming youtube flash plugin to youtube iFrame based player. < rdar://problem/11201356> On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE". @@ -19,8 +18,8 @@ PASS objectEmbedShadowRoot.firstChild.firstChild.tagName is "IFRAME" PASS internals.shadowPseudoId(objectNoEmbedShadowRoot.firstChild) is "-webkit-plugin-replacement" PASS objectNoEmbedShadowRoot.firstChild.firstChild.tagName is "IFRAME" -PASS internals.shadowPseudoId(normalEmbedInIframeShadowRoot.firstChild) is "-webkit-plugin-replacement" -PASS normalEmbedInIframeShadowRoot.firstChild.firstChild.tagName is "IFRAME" +FAIL internals.shadowPseudoId(normalEmbedInIframeShadowRoot.firstChild) should be -webkit-plugin-replacement. Threw exception TypeError: null is not an object (evaluating 'normalEmbedInIframeShadowRoot.firstChild') +FAIL normalEmbedInIframeShadowRoot.firstChild.firstChild.tagName should be IFRAME. Threw exception TypeError: null is not an object (evaluating 'normalEmbedInIframeShadowRoot.firstChild') PASS successfullyParsed is true TEST COMPLETE 1239533 1 ryanhaddad 2016-10-12 15:14:10 -0700 Flakiness dashboard seems to indicate that this may be fallout from http://trac.webkit.org/changeset/207162 1241226 2 ryanhaddad 2016-10-17 16:03:54 -0700 Marked as flaky in http://trac.webkit.org/projects/webkit/changeset/207440 1251531 3 ap 2016-11-15 11:49:08 -0800 Alex, is this something that you are looking into? 1251573 4 achristensen 2016-11-15 12:52:13 -0800 Not actively. 1251591 5 webkit-bug-importer 2016-11-15 13:32:30 -0800 <rdar://problem/29274536> 1258086 6 achristensen 2016-12-08 23:26:42 -0800 I verified that this test works fine for me locally. run-webkit-tests platform/ios-simulator/ios/plugin/youtube-flash-plugin-iframe.html --ios-simulator --iterations=10 --repeat-each=10 I think we should just unskip it. What do you think, Alexey? 1258087 7 296647 achristensen 2016-12-08 23:34:56 -0800 Created attachment 296647 Patch 1258088 8 296647 achristensen 2016-12-08 23:42:32 -0800 Comment on attachment 296647 Patch Nope, I can reproduce this about 10% of the time. URLParser enabled or not, doesn't matter. This means it's probably not related to the URLParser being enabled. 1258090 9 achristensen 2016-12-08 23:46:10 -0800 I'm not familiar with this test, or why these URLs should be blocked. Dan, what is going on here? 1258155 10 dbates 2016-12-09 09:51:10 -0800 (In reply to comment #9) > I'm not familiar with this test, or why these URLs should be blocked. Dan, > what is going on here? The "Blocked access to external URL" messages are because we do not allow external connections when running a test in DRT/WKTR. They can be ignored for the purposes of this test. > +FAIL internals.shadowPseudoId(normalEmbedInIframeShadowRoot.firstChild) should be -webkit-plugin-replacement. Threw exception TypeError: null is not an object (evaluating 'normalEmbedInIframeShadowRoot.firstChild') This means that that the plugin had not been not replaced by the time we checked. Plugin/plugin replacement loading is asynchronous with respect to the initial parsing of an HTML object/embed. We need to find a better way to know when a plugin/plugin-replacement has loaded. 1258157 11 dbates 2016-12-09 09:52:10 -0800 (In reply to comment #10) > We need to find a better way to know when a plugin/plugin-replacement has loaded. I meant to write: We need to find a better way to know when a plugin/plugin-replacement has loaded than using a zero timer. 1258168 12 ap 2016-12-09 10:08:01 -0800 Checking internal test history, I see this test being flaky on ASan and GuardMalloc bots even in September. Also, WebKit1 was a flaky failure. But debug WK2 bots did indeed see a very sharp increase in flakiness around r207162. 1259153 13 achristensen 2016-12-12 12:12:43 -0800 Having concluded that this was not related to the URLParser change because of the equal flakiness rate with or without the URLParser, I'm removing myself as the assignee and I do not intend to look into this further. 1259162 14 ap 2016-12-12 12:50:10 -0800 Alex and I looked at the flakiness rate, and it has very clearly increased dramatically around r207162. There isn't really another change close enough that can be blamed. There were no failures for ~200 revisions before r207162. Here is the history around it: 207158 oooo (this revision was tested 4 times, with 4 passes) 207159 o 207160 (this exact revision hasn't been tested) 207161 o 207162 x (tested once, failed) 207163 o (tested once, passed) 207164 xxxxoo (this revision was tested 6 times, with 4 failures and 2 passes) 207165 x 207166 xo 207167 207168 xx 207169 xx r207161 is a contributors.json change r207160 is purely a WebKit1 change r207159 is a change to iOS 9 built apps that could theoretically have a bug, but it just uses standard settings machinery, unlikely to be wrong. 1259168 15 ap 2016-12-12 12:56:54 -0800 It seems likely that it's not an obvious outright bug in the URL parser causing this, but some kind of subtle change (perhaps timing), but we still need to make the test work again. 1375883 16 ryanhaddad 2017-11-29 09:50:59 -0800 This test now fails 100% of the time on debug and it is flaky on release. Here is the diff from a release build: --- /Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/layout-test-results/platform/ios/ios/plugin/youtube-flash-plugin-iframe-expected.txt +++ /Volumes/Data/slave/ios-simulator-11-release-tests-wk2/build/layout-test-results/platform/ios/ios/plugin/youtube-flash-plugin-iframe-actual.txt @@ -15,8 +15,8 @@ PASS document.querySelectorAll("iframe").length is 1 PASS internals.shadowPseudoId(normalEmbedShadowRoot.firstChild) is "-webkit-plugin-replacement" PASS normalEmbedShadowRoot.firstChild.firstChild.tagName is "IFRAME" -PASS internals.shadowPseudoId(objectEmbedShadowRoot.firstChild) is "-webkit-plugin-replacement" -PASS objectEmbedShadowRoot.firstChild.firstChild.tagName is "IFRAME" +FAIL internals.shadowPseudoId(objectEmbedShadowRoot.firstChild) should be -webkit-plugin-replacement. Threw exception TypeError: null is not an object (evaluating 'objectEmbedShadowRoot.firstChild') +FAIL objectEmbedShadowRoot.firstChild.firstChild.tagName should be IFRAME. Threw exception TypeError: null is not an object (evaluating 'objectEmbedShadowRoot.firstChild') PASS internals.shadowPseudoId(objectNoEmbedShadowRoot.firstChild) is "-webkit-plugin-replacement" PASS objectNoEmbedShadowRoot.firstChild.firstChild.tagName is "IFRAME" PASS internals.shadowPseudoId(normalEmbedInIframeShadowRoot.firstChild) is "-webkit-plugin-replacement" 1381251 17 jlewis3 2017-12-13 12:55:07 -0800 This is still happening as of 12/13/2017 Updated Expectations: https://trac.webkit.org/changeset/225867/webkit 296647 2016-12-08 23:34:56 -0800 2016-12-08 23:42:32 -0800 Patch bug-163362-20161208233414.patch text/plain 2452 achristensen SW5kZXg6IExheW91dFRlc3RzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBMYXlvdXRUZXN0cy9D aGFuZ2VMb2cJKHJldmlzaW9uIDIwOTU5NykKKysrIExheW91dFRlc3RzL0NoYW5nZUxvZwkod29y a2luZyBjb3B5KQpAQCAtMSwzICsxLDE0IEBACisyMDE2LTEyLTA4ICBBbGV4IENocmlzdGVuc2Vu ICA8YWNocmlzdGVuc2VuQHdlYmtpdC5vcmc+CisKKyAgICAgICAgUkVHUkVTU0lPTiAocjIwNzE2 Mik6IFtpb3Mtc2ltdWxhdG9yIGRlYnVnXSBMYXlvdXRUZXN0IHBsYXRmb3JtL2lvcy1zaW11bGF0 b3IvaW9zL3BsdWdpbi95b3V0dWJlLWZsYXNoLXBsdWdpbi1pZnJhbWUuaHRtbCBmYWlsaW5nCisg ICAgICAgIGh0dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNnaT9pZD0xNjMzNjIKKwor ICAgICAgICBSZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAgICAgICAqIHBsYXRmb3Jt L2lvcy1zaW11bGF0b3Itd2sxL1Rlc3RFeHBlY3RhdGlvbnM6CisgICAgICAgICogcGxhdGZvcm0v aW9zLXNpbXVsYXRvci13azIvVGVzdEV4cGVjdGF0aW9uczoKKyAgICAgICAgVW5za2lwIHRoZSB0 ZXN0LiAgSXQgd29ya3MgZmluZS4KKwogMjAxNi0xMi0wOCAgQW50b2luZSBRdWludCAgPGdyYW91 dHNAYXBwbGUuY29tPgogCiAgICAgICAgIFtNb2Rlcm4gTWVkaWEgQ29udHJvbHNdIEV4aXRpbmcg ZnVsbHNjcmVlbiB3aXRoIHBhdXNlZCBtZWRpYSBzaG93cyB0aGUgc3RhcnQgYnV0dG9uCkluZGV4 OiBMYXlvdXRUZXN0cy9wbGF0Zm9ybS9pb3Mtc2ltdWxhdG9yLXdrMS9UZXN0RXhwZWN0YXRpb25z Cj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT0KLS0tIExheW91dFRlc3RzL3BsYXRmb3JtL2lvcy1zaW11bGF0b3Itd2sxL1Rl c3RFeHBlY3RhdGlvbnMJKHJldmlzaW9uIDIwOTU1OCkKKysrIExheW91dFRlc3RzL3BsYXRmb3Jt L2lvcy1zaW11bGF0b3Itd2sxL1Rlc3RFeHBlY3RhdGlvbnMJKHdvcmtpbmcgY29weSkKQEAgLTE3 NTAsNyArMTc1MCw2IEBAIGltcG9ydGVkL21vemlsbGEvc3ZnL3RzcGFuLXJvdGF0ZS0wNC5zdmcK IGltcG9ydGVkL21vemlsbGEvc3ZnL3RzcGFuLXJvdGF0ZS0wNS5zdmcgWyBQYXNzIEltYWdlT25s eUZhaWx1cmUgXQogaW1wb3J0ZWQvbW96aWxsYS9zdmcvdHNwYW4teHktMDEuc3ZnIFsgUGFzcyBJ bWFnZU9ubHlGYWlsdXJlIF0KIGltcG9ydGVkL3czYy93ZWItcGxhdGZvcm0tdGVzdHMvZmV0Y2gv YXBpL2NvcnMvY29ycy1vcmlnaW4uaHRtbCBbIFBhc3MgQ3Jhc2ggXQotcGxhdGZvcm0vaW9zLXNp bXVsYXRvci9pb3MvcGx1Z2luL3lvdXR1YmUtZmxhc2gtcGx1Z2luLWlmcmFtZS5odG1sIFsgUGFz cyBGYWlsdXJlIF0KIHN2Zy90ZXh0L2FkZC10c3Bhbi1wb3NpdGlvbi1idWcuaHRtbCBbIFBhc3Mg SW1hZ2VPbmx5RmFpbHVyZSBdCiBzZWN1cml0eS9jb250ZW50U2VjdXJpdHlQb2xpY3kvdmlkZW8t d2l0aC1maWxlLXVybC1hbGxvd2VkLWJ5LW1lZGlhLXNyYy1zdGFyLmh0bWwgWyBQYXNzIEltYWdl T25seUZhaWx1cmUgXQogc3ZnL3RleHQvaGlkcGktdGV4dC1zZWxlY3Rpb24tcmVjdC1wb3NpdGlv bi5odG1sIFsgUGFzcyBJbWFnZU9ubHlGYWlsdXJlIF0KSW5kZXg6IExheW91dFRlc3RzL3BsYXRm b3JtL2lvcy1zaW11bGF0b3Itd2syL1Rlc3RFeHBlY3RhdGlvbnMKPT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gTGF5 b3V0VGVzdHMvcGxhdGZvcm0vaW9zLXNpbXVsYXRvci13azIvVGVzdEV4cGVjdGF0aW9ucwkocmV2 aXNpb24gMjA5NTU4KQorKysgTGF5b3V0VGVzdHMvcGxhdGZvcm0vaW9zLXNpbXVsYXRvci13azIv VGVzdEV4cGVjdGF0aW9ucwkod29ya2luZyBjb3B5KQpAQCAtMTkwNCw4ICsxOTA0LDYgQEAgd2Vi a2l0Lm9yZy9iLzE2MjUyNCBbIFJlbGVhc2UgXSBodHRwL3RlcwogCiB3ZWJraXQub3JnL2IvMTI3 MTE2IFsgRGVidWcgXSBzdmcvY3VzdG9tL2FuaW1hdGUtaW5pdGlhbC1wYXVzZS11bnBhdXNlLmh0 bWwgWyBQYXNzIEZhaWx1cmUgXQogCi13ZWJraXQub3JnL2IvMTYzMzYyIFsgRGVidWcgXSBwbGF0 Zm9ybS9pb3Mtc2ltdWxhdG9yL2lvcy9wbHVnaW4veW91dHViZS1mbGFzaC1wbHVnaW4taWZyYW1l Lmh0bWwgWyBQYXNzIEZhaWx1cmUgXQotCiB3ZWJraXQub3JnL2IvMTY0OTYwIFsgUmVsZWFzZSBd IGh0dHAvdGVzdHMvc2VjdXJpdHkvbW9kdWxlLWNvcnJlY3QtbWltZS10eXBlcy5odG1sIFsgU2xv dyBdCiAKIHdlYmtpdC5vcmcvYi8xNjQ5NjEgWyBSZWxlYXNlIF0gaHR0cC90ZXN0cy9zdG9yYWdl L2NhbGxiYWNrcy1hcmUtY2FsbGVkLWluLWNvcnJlY3QtY29udGV4dC5odG1sIFsgVGltZW91dCBd Cg==