155760 2016-03-22 11:24:40 -0700 imported/w3c/web-platform-tests/streams/readable-streams/general.https.html is a flaky failure 2016-06-22 11:14:24 -0700 1 1 1 Unclassified WebKit Tools / Tests Other Unspecified Unspecified RESOLVED FIXED P2 Normal --- 1 ap youennf commit-queue jiewen_tan lforschler ryanhaddad youennf oldest_to_newest 1177055 0 ap 2016-03-22 11:24:40 -0700 imported/w3c/web-platform-tests/streams/readable-streams/general.https.html fails somewhat frequently: @@ -13,7 +13,7 @@ PASS ReadableStream should be able to enqueue different objects. PASS ReadableStream: if pull rejects, it should error the stream PASS ReadableStream: should only call pull once upon starting the stream -PASS ReadableStream: should call pull when trying to read from a started, empty stream +FAIL ReadableStream: should call pull when trying to read from a started, empty stream assert_equals: pull should be called when read is called expected 2 but got 1 PASS ReadableStream: should only call pull once on a non-empty stream read from before start fulfills PASS ReadableStream: should only call pull once on a non-empty stream read from after start fulfills PASS ReadableStream: should call pull in reaction to read()ing the last chunk, if not draining http://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fstreams%2Freadable-streams%2Fgeneral.https.html 1177834 1 ap 2016-03-24 18:16:38 -0700 It's also frequently reported as a crash on EWS, although I'm not sure if it's this test's fault. 1183473 2 jiewen_tan 2016-04-13 11:52:40 -0700 Committed r199507: <http://trac.webkit.org/changeset/199507> 1183474 3 jiewen_tan 2016-04-13 11:53:11 -0700 Reopen to track the fix. 1184731 4 youennf 2016-04-18 10:46:25 -0700 It seems that on slow bots, the order of the promise resolution does not play well with DOM timers. It seems that this can be solved by changing delay(1) to delay(10) in imported/w3c/web-platform-tests/streams/readable-streams/general.js. 1184846 5 ap 2016-04-18 16:00:08 -0700 Will that make any tests take 10 seconds as part of normal execution, or is that code just to handle failing tests nicely? Having tests take 10 seconds would be pretty horrible. 1184960 6 276705 youennf 2016-04-19 00:58:00 -0700 Created attachment 276705 Patch 1184961 7 youennf 2016-04-19 01:02:33 -0700 (In reply to comment #5) > Will that make any tests take 10 seconds as part of normal execution, or is > that code just to handle failing tests nicely? > > Having tests take 10 seconds would be pretty horrible. It is 10 milliseconds. The issue can probably be summarized to the following code: let pullCount1 = 0; let pullCount2 = 0; Promise.resolve().then(() => { ++pullCount1; Promise.resolve().then(() => { ++pullCount2; }); }); return delay(1).then(() => { // Not flaky assert_equals(pullCount1, 1); // May be flaky assert_equals(pullCount2, 1); }); By going to delay(10), the flakiness does not happen anymore on my machine. The test could probably be improved to further limit the flakiness (returning a resolved promise to the start method). This can be tackled directly when upstreaming that change to W3C WPT repo. 1185070 8 276705 ap 2016-04-19 10:34:05 -0700 Comment on attachment 276705 Patch Seems fine as a temporary workaround. Generally speaking, bots tend to still have flakiness when increasing timer delays from 1ms to 100ms, because that's just how much variability we get when running the tests. We can't really have any non-zero timeouts as part of normal test flow, they are only useful when catching failures (and then set them to 20 seconds or so). 1185083 9 youennf 2016-04-19 11:17:33 -0700 (In reply to comment #8) > Comment on attachment 276705 [details] > Patch > > Seems fine as a temporary workaround. > > Generally speaking, bots tend to still have flakiness when increasing timer > delays from 1ms to 100ms, because that's just how much variability we get > when running the tests. We can't really have any non-zero timeouts as part > of normal test flow, they are only useful when catching failures (and then > set them to 20 seconds or so). Thanks for the review. 10 ms is used in several streams tests without causing issue apparently. I'll land it and see whether this is a good enough fix. If so, I will upstream this change or a more deterministic version of the test to WPT or streams repo 1185106 10 276705 commit-queue 2016-04-19 12:06:30 -0700 Comment on attachment 276705 Patch Clearing flags on attachment: 276705 Committed r199736: <http://trac.webkit.org/changeset/199736> 1185107 11 commit-queue 2016-04-19 12:06:35 -0700 All reviewed patches have been landed. Closing bug. 1204079 12 youennf 2016-06-21 00:06:44 -0700 test is still flaky, although less than before. Let's fix it properly. 1204080 13 281722 youennf 2016-06-21 00:08:03 -0700 Created attachment 281722 Patch 1204147 14 youennf 2016-06-21 08:04:40 -0700 *** Bug 153095 has been marked as a duplicate of this bug. *** 1204586 15 281722 commit-queue 2016-06-22 11:14:21 -0700 Comment on attachment 281722 Patch Clearing flags on attachment: 281722 Committed r202338: <http://trac.webkit.org/changeset/202338> 1204587 16 commit-queue 2016-06-22 11:14:24 -0700 All reviewed patches have been landed. Closing bug. 276705 2016-04-19 00:58:00 -0700 2016-06-21 00:07:57 -0700 Patch bug-155760-20160419095818.patch text/plain 3390 youennf U3VidmVyc2lvbiBSZXZpc2lvbjogMTk5NjUwCmRpZmYgLS1naXQgYS9MYXlvdXRUZXN0cy9DaGFu Z2VMb2cgYi9MYXlvdXRUZXN0cy9DaGFuZ2VMb2cKaW5kZXggOTJmYzNiYWI4ZDNlOTMxZmU2MDI1 ZjVjMjBjNzU1YjRmNGQ1YjEyNC4uYzgzNWJmMjEwMTExNGJiZjBmODQxYjdlZTRkODU3M2UxMTBh ZGM1YSAxMDA2NDQKLS0tIGEvTGF5b3V0VGVzdHMvQ2hhbmdlTG9nCisrKyBiL0xheW91dFRlc3Rz L0NoYW5nZUxvZwpAQCAtMSwzICsxLDEyIEBACisyMDE2LTA0LTE5ICBZb3Vlbm4gRmFibGV0ICA8 eW91ZW5uLmZhYmxldEBjcmYuY2Fub24uZnI+CisKKyAgICAgICAgaW1wb3J0ZWQvdzNjL3dlYi1w bGF0Zm9ybS10ZXN0cy9zdHJlYW1zL3JlYWRhYmxlLXN0cmVhbXMvZ2VuZXJhbC5odHRwcy5odG1s IGlzIGEgZmxha3kgZmFpbHVyZQorICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9zaG93 X2J1Zy5jZ2k/aWQ9MTU1NzYwCisKKyAgICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChPT1BTISku CisKKyAgICAgICAgKiBUZXN0RXhwZWN0YXRpb25zOiByZW1vdmVkIGZsYWt5IGV4cGVjdGF0aW9u LgorCiAyMDE2LTA0LTE3ICBZb2F2IFdlaXNzICA8eW9hdkB5b2F2LndzPgogCiAgICAgICAgIElu aXRpYWwgTGluayBwcmVsb2FkIHN1cHBvcnQKZGlmZiAtLWdpdCBhL0xheW91dFRlc3RzL2ltcG9y dGVkL3czYy9DaGFuZ2VMb2cgYi9MYXlvdXRUZXN0cy9pbXBvcnRlZC93M2MvQ2hhbmdlTG9nCmlu ZGV4IDQ4MDgzZGNkNDk0MjkxOGY5MWI4MDUyMGQzNGNhOWRiM2RlYWQ3MmIuLjUxY2U1MzM3OGEy OWQ1NmQ3MjY2ZGYwYWI4NTI5YTVhMjMyYTdlOGIgMTAwNjQ0Ci0tLSBhL0xheW91dFRlc3RzL2lt cG9ydGVkL3czYy9DaGFuZ2VMb2cKKysrIGIvTGF5b3V0VGVzdHMvaW1wb3J0ZWQvdzNjL0NoYW5n ZUxvZwpAQCAtMSwzICsxLDE3IEBACisyMDE2LTA0LTE5ICBZb3Vlbm4gRmFibGV0ICA8eW91ZW5u LmZhYmxldEBjcmYuY2Fub24uZnI+CisKKyAgICAgICAgaW1wb3J0ZWQvdzNjL3dlYi1wbGF0Zm9y bS10ZXN0cy9zdHJlYW1zL3JlYWRhYmxlLXN0cmVhbXMvZ2VuZXJhbC5odHRwcy5odG1sIGlzIGEg Zmxha3kgZmFpbHVyZQorICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9zaG93X2J1Zy5j Z2k/aWQ9MTU1NzYwCisKKyAgICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChPT1BTISkuCisKKyAg ICAgICAgRmxha2luZXNzIHdhcyBjb21pbmcgZnJvbSBhIGNhbGxiYWNrIGJlaW5nIGNhbGxlZCBh ZnRlciBhIHRpbWVvdXQuCisgICAgICAgIFRpbWVvdXQgd2FzIHNldCBzbyB0aGF0IGEgZG91Ymxl IHByb21pc2UgcmVzb2x1dGlvbiAocHJvbWlzZSByZXNvbHZlZAorICAgICAgICBpbiBhbm90aGVy IHByb21pc2UgcmVzb2x1dGlvbiBjYWxsYmFjaykgaXMgZG9uZSBiZWZvcmUgY2FsbGluZyB0aGUg Y2FsbGJhY2suCisgICAgICAgIE9uIHNvbWUgc2xvdyBib3RzLCB0aGUgdGltZW91dCB3YXMgbm90 IGJpZyBlbm91Z2guCisKKyAgICAgICAgKiB3ZWItcGxhdGZvcm0tdGVzdHMvc3RyZWFtcy9yZWFk YWJsZS1zdHJlYW1zL2dlbmVyYWwuanM6IEluY3JlYXNlZCB0aGUgY2FsbGJhY2sgZGVsYXkuCisK IDIwMTYtMDQtMTcgIFlvdWVubiBGYWJsZXQgIDx5b3Vlbm4uZmFibGV0QGNyZi5jYW5vbi5mcj4K IAogICAgICAgICBbRmV0Y2ggQVBJXSBDb25zdW1lIEhUVFAgZGF0YSBhcyBhIFJlYWRhYmxlU3Ry ZWFtCmRpZmYgLS1naXQgYS9MYXlvdXRUZXN0cy9UZXN0RXhwZWN0YXRpb25zIGIvTGF5b3V0VGVz dHMvVGVzdEV4cGVjdGF0aW9ucwppbmRleCA5ZTg4ODYyYjI4Y2Q4YjkwZGE5NjUxZTZhMmZhZTcy ZDA1OTM2ODgwLi4yZmVjMWJmZTc2Y2JlY2UxNTQ0NjBhMmM2NWI2MzQwZDA1OTQ5M2M0IDEwMDY0 NAotLS0gYS9MYXlvdXRUZXN0cy9UZXN0RXhwZWN0YXRpb25zCisrKyBiL0xheW91dFRlc3RzL1Rl c3RFeHBlY3RhdGlvbnMKQEAgLTc0Miw3ICs3NDIsNiBAQCB3ZWJraXQub3JnL2IvMTQ4ODA1IGlt cG9ydGVkL3czYy9jc3MvY3NzLW11bHRpY29sLTEvbXVsdGljb2wtemVyby1oZWlnaHQtMDAxLnho dAogd2Via2l0Lm9yZy9iLzE0ODgwNSBpbXBvcnRlZC93M2MvY3NzL2Nzcy1tdWx0aWNvbC0xL211 bHRpY29sLXNwYW4tYWxsLWNoaWxkLTAwMS54aHQgWyBJbWFnZU9ubHlGYWlsdXJlIF0KIAogd2Vi a2l0Lm9yZy9iLzE0ODg4NCBpbXBvcnRlZC93M2Mvd2ViLXBsYXRmb3JtLXRlc3RzL2h0bWwvc2Vt YW50aWNzL2Zvcm1zL3RleHRmaWVsZHNlbGVjdGlvbi9zZWxlY3Rpb24uaHRtbCBbIFBhc3MgRmFp bHVyZSBdCi13ZWJraXQub3JnL2IvMTU1NzYwIGltcG9ydGVkL3czYy93ZWItcGxhdGZvcm0tdGVz dHMvc3RyZWFtcy9yZWFkYWJsZS1zdHJlYW1zL2dlbmVyYWwuaHR0cHMuaHRtbCBbIFBhc3MgRmFp bHVyZSBdCiAKIHdlYmtpdC5vcmcvYi8xNDg5MjIgc3ZnL2FzLW9iamVjdC9zdmctaW4tb2JqZWN0 LWR5bmFtaWMtYXR0cmlidXRlLWNoYW5nZS5odG1sIFsgUGFzcyBJbWFnZU9ubHlGYWlsdXJlIF0K IHdlYmtpdC5vcmcvYi8xNDg5MjUgc3ZnL2RvbS9zdmctcm9vdC1sZW5ndGhzLmh0bWwgWyBQYXNz IEZhaWx1cmUgXQpkaWZmIC0tZ2l0IGEvTGF5b3V0VGVzdHMvaW1wb3J0ZWQvdzNjL3dlYi1wbGF0 Zm9ybS10ZXN0cy9zdHJlYW1zL3JlYWRhYmxlLXN0cmVhbXMvZ2VuZXJhbC5qcyBiL0xheW91dFRl c3RzL2ltcG9ydGVkL3czYy93ZWItcGxhdGZvcm0tdGVzdHMvc3RyZWFtcy9yZWFkYWJsZS1zdHJl YW1zL2dlbmVyYWwuanMKaW5kZXggZmU4MWJkNTRkM2Y1MjBmOTU5NTQ3OGZiMjg2ZDZlMmM0OWEw OTllMi4uNTdhNzNmOTQ3NDQ4ZGRmN2U3MjQyMjAyMzg5MzgwMGI5NzFjMzI4OCAxMDA2NDQKLS0t IGEvTGF5b3V0VGVzdHMvaW1wb3J0ZWQvdzNjL3dlYi1wbGF0Zm9ybS10ZXN0cy9zdHJlYW1zL3Jl YWRhYmxlLXN0cmVhbXMvZ2VuZXJhbC5qcworKysgYi9MYXlvdXRUZXN0cy9pbXBvcnRlZC93M2Mv d2ViLXBsYXRmb3JtLXRlc3RzL3N0cmVhbXMvcmVhZGFibGUtc3RyZWFtcy9nZW5lcmFsLmpzCkBA IC0yOTIsNyArMjkyLDggQEAgcHJvbWlzZV90ZXN0KCgpID0+IHsKICAgICB9CiAgIH0pOwogCi0g IHJldHVybiBkZWxheSgxKS50aGVuKCgpID0+IHsKKyAgLy8gV0VCS0lUIG1vZGlmaWNhdGlvbjog ZGVsYXkoMSkgcmVwbGFjZWQgYnkgZGVsYXkoMTApIHRvIGVuc3VyZSB0aGF0IHB1bGwgcHJvbWlz ZSBpcyByZXNvbHZlZCBiZWZvcmUgY2FsbGluZyBiZWxvdyB0aGVuZWQgY29kZS4KKyAgcmV0dXJu IGRlbGF5KDEwKS50aGVuKCgpID0+IHsKICAgICBhc3NlcnRfZXF1YWxzKHB1bGxDb3VudCwgMSwg J3B1bGwgc2hvdWxkIGJlIGNhbGxlZCBvbmNlIHN0YXJ0IGZpbmlzaGVzJyk7CiAKICAgICBjb25z dCByZWFkZXIgPSBycy5nZXRSZWFkZXIoKTsK 281722 2016-06-21 00:08:03 -0700 2016-06-22 11:14:21 -0700 Patch bug-155760-20160621090756.patch text/plain 2253 youennf U3VidmVyc2lvbiBSZXZpc2lvbjogMjAyMTk4CmRpZmYgLS1naXQgYS9MYXlvdXRUZXN0cy9pbXBv cnRlZC93M2MvQ2hhbmdlTG9nIGIvTGF5b3V0VGVzdHMvaW1wb3J0ZWQvdzNjL0NoYW5nZUxvZwpp bmRleCBiYjIzZDExMzA0MWVlZTU3OTEzYzRlZDViY2IzYjg1ZTc2ZGQ3ZTFmLi4zOWIzMDlmMDU2 NTJhYmE5N2QyMjg2YmE2ZDlhOWRmMWI0ZWYxMGE2IDEwMDY0NAotLS0gYS9MYXlvdXRUZXN0cy9p bXBvcnRlZC93M2MvQ2hhbmdlTG9nCisrKyBiL0xheW91dFRlc3RzL2ltcG9ydGVkL3czYy9DaGFu Z2VMb2cKQEAgLTEsMyArMSwxNiBAQAorMjAxNi0wNi0yMSAgWW91ZW5uIEZhYmxldCAgPHlvdWVu bmZAZ21haWwuY29tPgorCisgICAgICAgIGltcG9ydGVkL3czYy93ZWItcGxhdGZvcm0tdGVzdHMv c3RyZWFtcy9yZWFkYWJsZS1zdHJlYW1zL2dlbmVyYWwuaHR0cHMuaHRtbCBpcyBhIGZsYWt5IGZh aWx1cmUKKyAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcvc2hvd19idWcuY2dpP2lkPTE1 NTc2MAorCisgICAgICAgIFJldmlld2VkIGJ5IE5PQk9EWSAoT09QUyEpLgorCisgICAgICAgIE1h a2luZyB0ZXN0IG1vcmUgcHJlZGljdGFibGUuCisgICAgICAgIFVwc3RyZWFtZWQgdG8gdzNjIHJl cG9zaXRvcnkgYXMgaHR0cHM6Ly9naXRodWIuY29tL3czYy93ZWItcGxhdGZvcm0tdGVzdHMvcHVs bC8zMjAzLgorCisgICAgICAgICogd2ViLXBsYXRmb3JtLXRlc3RzL3N0cmVhbXMvcmVhZGFibGUt c3RyZWFtcy9nZW5lcmFsLmpzOgorICAgICAgICAocHJvbWlzZV90ZXN0KToKKwogMjAxNi0wNi0x NyAgQmVuamFtaW4gUG91bGFpbiAgPGJlbmphbWluQHdlYmtpdC5vcmc+CiAKICAgICAgICAgOmlu ZGV0ZXJtaW5hdGUgcHNldWRvLWNsYXNzIHNob3VsZCBtYXRjaCByYWRpb3Mgd2hvc2UgZ3JvdXAg aGFzIG5vIGNoZWNrZWQgcmFkaW8KZGlmZiAtLWdpdCBhL0xheW91dFRlc3RzL2ltcG9ydGVkL3cz Yy93ZWItcGxhdGZvcm0tdGVzdHMvc3RyZWFtcy9yZWFkYWJsZS1zdHJlYW1zL2dlbmVyYWwuanMg Yi9MYXlvdXRUZXN0cy9pbXBvcnRlZC93M2Mvd2ViLXBsYXRmb3JtLXRlc3RzL3N0cmVhbXMvcmVh ZGFibGUtc3RyZWFtcy9nZW5lcmFsLmpzCmluZGV4IDRiNWYzZWI2ZmRkNGU5MmQ0NzY4MDQzYjRk YjQ2ZmVmYTJiNzkyMzQuLmE4OTI0YmUwNTMyZTNlNTQ5YmRjYTQwZmE2ZTlhMWE0NzZiNTM0MGMg MTAwNjQ0Ci0tLSBhL0xheW91dFRlc3RzL2ltcG9ydGVkL3czYy93ZWItcGxhdGZvcm0tdGVzdHMv c3RyZWFtcy9yZWFkYWJsZS1zdHJlYW1zL2dlbmVyYWwuanMKKysrIGIvTGF5b3V0VGVzdHMvaW1w b3J0ZWQvdzNjL3dlYi1wbGF0Zm9ybS10ZXN0cy9zdHJlYW1zL3JlYWRhYmxlLXN0cmVhbXMvZ2Vu ZXJhbC5qcwpAQCAtMjkzLDggKzI5MywxMiBAQCBwcm9taXNlX3Rlc3QoKCkgPT4gewogcHJvbWlz ZV90ZXN0KCgpID0+IHsKIAogICBsZXQgcHVsbENvdW50ID0gMDsKKyAgY29uc3Qgc3RhcnRQcm9t aXNlID0gUHJvbWlzZS5yZXNvbHZlKCk7CiAKICAgY29uc3QgcnMgPSBuZXcgUmVhZGFibGVTdHJl YW0oeworICAgIHN0YXJ0KCkgeworICAgICAgcmV0dXJuIHN0YXJ0UHJvbWlzZTsKKyAgICB9LAog ICAgIHB1bGwoYykgewogICAgICAgLy8gRG9uJ3QgZW5xdWV1ZSBpbW1lZGlhdGVseSBhZnRlciBz dGFydC4gV2Ugd2FudCB0aGUgc3RyZWFtIHRvIGJlIGVtcHR5IHdoZW4gd2UgY2FsbCAucmVhZCgp IG9uIGl0LgogICAgICAgaWYgKHB1bGxDb3VudCA+IDApIHsKQEAgLTMwNCwxMCArMzA4LDkgQEAg cHJvbWlzZV90ZXN0KCgpID0+IHsKICAgICB9CiAgIH0pOwogCi0gIC8vIFdFQktJVCBtb2RpZmlj YXRpb246IGRlbGF5KDEpIHJlcGxhY2VkIGJ5IGRlbGF5KDEwKSB0byBlbnN1cmUgdGhhdCBwdWxs IHByb21pc2UgaXMgcmVzb2x2ZWQgYmVmb3JlIGNhbGxpbmcgYmVsb3cgdGhlbmVkIGNvZGUuCi0g IHJldHVybiBkZWxheSgxMCkudGhlbigoKSA9PiB7CisgIHJldHVybiBzdGFydFByb21pc2UudGhl bigoKSA9PiB7CiAgICAgYXNzZXJ0X2VxdWFscyhwdWxsQ291bnQsIDEsICdwdWxsIHNob3VsZCBi ZSBjYWxsZWQgb25jZSBzdGFydCBmaW5pc2hlcycpOwotCisgIH0pLnRoZW4oKCkgPT4gewogICAg IGNvbnN0IHJlYWRlciA9IHJzLmdldFJlYWRlcigpOwogICAgIGNvbnN0IHJlYWQgPSByZWFkZXIu cmVhZCgpOwogICAgIGFzc2VydF9lcXVhbHMocHVsbENvdW50LCAyLCAncHVsbCBzaG91bGQgYmUg Y2FsbGVkIHdoZW4gcmVhZCBpcyBjYWxsZWQnKTsK