187389 2018-07-06 09:16:36 -0700 Layout Test editing/selection/navigation-clears-editor-state.html is flaky 2018-10-25 09:45:08 -0700 1 1 1 Unclassified WebKit Tools / Tests WebKit Nightly Build Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=187309 https://bugs.webkit.org/show_bug.cgi?id=190890 InRadar P2 Normal --- 1 tsavell saam commit-queue lforschler mark.lam realdawei rniwa ryanhaddad saam webkit-bug-importer wenson_hsieh oldest_to_newest 1439786 0 tsavell 2018-07-06 09:16:36 -0700 The following layout test is flaky on Sierra and High Sierra WK1 editing/selection/navigation-clears-editor-state.html Probable cause: I do not know, though it seems to have begun around revision r233184 Flakiness Dashboard: https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=editing%2Fselection%2Fnavigation-clears-editor-state.html looking into issue further 1439789 1 ryanhaddad 2018-07-06 09:27:30 -0700 Saam attempted to fix this in https://trac.webkit.org/changeset/233555, it it failing after that revision? 1439794 2 tsavell 2018-07-06 09:31:40 -0700 Yes its still flakey after that revision. But it seems like his revision might have fixed the test for windows. 1439799 3 344423 saam 2018-07-06 10:02:13 -0700 Created attachment 344423 patch 1439814 4 344423 commit-queue 2018-07-06 10:44:42 -0700 Comment on attachment 344423 patch Clearing flags on attachment: 344423 Committed r233581: <https://trac.webkit.org/changeset/233581> 1439815 5 commit-queue 2018-07-06 10:44:43 -0700 All reviewed patches have been landed. Closing bug. 1439816 6 webkit-bug-importer 2018-07-06 10:46:16 -0700 <rdar://problem/41898822> 1439922 7 tsavell 2018-07-06 14:41:56 -0700 So it looks like after that new revision the test is constantly failing on Sierra and High Sierra Debug WK1 and WK2. example of failure message: https://build.webkit.org/results/Apple%20High%20Sierra%20Debug%20WK1%20(Tests)/r233581%20(4602)/editing/selection/navigation-clears-editor-state-pretty-diff.html 1439923 8 ryanhaddad 2018-07-06 14:43:58 -0700 Reopening per Truitt's comment. 1440961 9 rniwa 2018-07-10 14:22:34 -0700 I think we just need to skip this test on debug bots. It's too slow there. 1440971 10 saam 2018-07-10 14:44:38 -0700 (In reply to Ryosuke Niwa from comment #9) > I think we just need to skip this test on debug bots. It's too slow there. Agreed. I'll do this. 1440974 11 saam 2018-07-10 14:53:23 -0700 (In reply to Saam Barati from comment #10) > (In reply to Ryosuke Niwa from comment #9) > > I think we just need to skip this test on debug bots. It's too slow there. > > Agreed. I'll do this. I spoke with Ryosuke offline. We're going to start by just removing the 18 second timeout. 1440975 12 saam 2018-07-10 14:53:41 -0700 (In reply to Saam Barati from comment #11) > (In reply to Saam Barati from comment #10) > > (In reply to Ryosuke Niwa from comment #9) > > > I think we just need to skip this test on debug bots. It's too slow there. > > > > Agreed. I'll do this. > > I spoke with Ryosuke offline. We're going to start by just removing the 18 > second timeout. We're also going to mark it slow on Debug 1440980 13 saam 2018-07-10 14:56:43 -0700 (In reply to Saam Barati from comment #12) > (In reply to Saam Barati from comment #11) > > (In reply to Saam Barati from comment #10) > > > (In reply to Ryosuke Niwa from comment #9) > > > > I think we just need to skip this test on debug bots. It's too slow there. > > > > > > Agreed. I'll do this. > > > > I spoke with Ryosuke offline. We're going to start by just removing the 18 > > second timeout. > > We're also going to mark it slow on Debug Actually, our new approach will be: - lower iframe count from 200 to 100 - remove 18 second timer We'll mark it as slow in the future if it times out. 1440983 14 saam 2018-07-10 15:01:00 -0700 (In reply to Saam Barati from comment #13) > (In reply to Saam Barati from comment #12) > > (In reply to Saam Barati from comment #11) > > > (In reply to Saam Barati from comment #10) > > > > (In reply to Ryosuke Niwa from comment #9) > > > > > I think we just need to skip this test on debug bots. It's too slow there. > > > > > > > > Agreed. I'll do this. > > > > > > I spoke with Ryosuke offline. We're going to start by just removing the 18 > > > second timeout. > > > > We're also going to mark it slow on Debug > > Actually, our new approach will be: > - lower iframe count from 200 to 100 > - remove 18 second timer > > We'll mark it as slow in the future if it times out. Landed in: https://trac.webkit.org/changeset/233701/webkit 1441184 15 tsavell 2018-07-11 10:45:24 -0700 After r233701 <https://trac.webkit.org/changeset/233701/webkit> Failure is back to being Flaky on WK1 Release Platforms. It has stopped failing on Debug. Example of test output on failure: --- /Volumes/Data/slave/highsierra-release-tests-wk1/build/layout-test-results/editing/selection/navigation-clears-editor-state-expected.txt +++ /Volumes/Data/slave/highsierra-release-tests-wk1/build/layout-test-results/editing/selection/navigation-clears-editor-state-actual.txt @@ -3,7 +3,7 @@ On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE". -PASS freed more than 15% of documents +FAIL freed less than 15% of documents PASS successfullyParsed is true TEST COMPLETE Test History: https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=editing%2Fselection%2Fnavigation-clears-editor-state.html 1441186 16 mark.lam 2018-07-11 10:51:08 -0700 (In reply to Truitt Savell from comment #15) > After r233701 <https://trac.webkit.org/changeset/233701/webkit> > > Failure is back to being Flaky on WK1 Release Platforms. It has stopped > failing on Debug. > > Example of test output on failure: > > --- > /Volumes/Data/slave/highsierra-release-tests-wk1/build/layout-test-results/ > editing/selection/navigation-clears-editor-state-expected.txt > +++ > /Volumes/Data/slave/highsierra-release-tests-wk1/build/layout-test-results/ > editing/selection/navigation-clears-editor-state-actual.txt > @@ -3,7 +3,7 @@ > On success, you will see a series of "PASS" messages, followed by "TEST > COMPLETE". > > > -PASS freed more than 15% of documents > +FAIL freed less than 15% of documents > PASS successfullyParsed is true > > TEST COMPLETE > > Test History: > https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard. > html#showAllRuns=true&tests=editing%2Fselection%2Fnavigation-clears-editor- > state.html How about adding <!-- webkit-test-runner [ jscOptions=--useConcurrentGC=false ] --> to the top of the test? That should make the GCing more consistent. 1441234 17 saam 2018-07-11 13:18:58 -0700 Ugh, we are putting more attention than needed into this test IMO. I say we just mark it as flaky and move on. 1441235 18 saam 2018-07-11 13:19:25 -0700 (In reply to Mark Lam from comment #16) > (In reply to Truitt Savell from comment #15) > > After r233701 <https://trac.webkit.org/changeset/233701/webkit> > > > > Failure is back to being Flaky on WK1 Release Platforms. It has stopped > > failing on Debug. > > > > Example of test output on failure: > > > > --- > > /Volumes/Data/slave/highsierra-release-tests-wk1/build/layout-test-results/ > > editing/selection/navigation-clears-editor-state-expected.txt > > +++ > > /Volumes/Data/slave/highsierra-release-tests-wk1/build/layout-test-results/ > > editing/selection/navigation-clears-editor-state-actual.txt > > @@ -3,7 +3,7 @@ > > On success, you will see a series of "PASS" messages, followed by "TEST > > COMPLETE". > > > > > > -PASS freed more than 15% of documents > > +FAIL freed less than 15% of documents > > PASS successfullyParsed is true > > > > TEST COMPLETE > > > > Test History: > > https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard. > > html#showAllRuns=true&tests=editing%2Fselection%2Fnavigation-clears-editor- > > state.html > > How about adding <!-- webkit-test-runner [ > jscOptions=--useConcurrentGC=false ] --> to the top of the test? That > should make the GCing more consistent. I believe the test is calling sync GCs manually. So I don't see how this will help 1441236 19 saam 2018-07-11 13:20:01 -0700 We could also just bump up the iteration count a bit again 1441240 20 mark.lam 2018-07-11 13:29:08 -0700 (In reply to Saam Barati from comment #18) > (In reply to Mark Lam from comment #16) > > How about adding <!-- webkit-test-runner [ > > jscOptions=--useConcurrentGC=false ] --> to the top of the test? That > > should make the GCing more consistent. > > I believe the test is calling sync GCs manually. So I don't see how this > will help You're right: GCController.collect() ultimately ends up in GCController::garbageCollectNow() in the web process. So, this wouldn't help. 1441369 21 rniwa 2018-07-11 20:31:29 -0700 I think this test is just slow. I think 200 was working fine. So let's go back there and just mark this test as SLOW. 1441371 22 344816 rniwa 2018-07-11 20:34:40 -0700 Created attachment 344816 Another fix attempt 1441375 23 rniwa 2018-07-11 20:36:20 -0700 Committed r233754: <https://trac.webkit.org/changeset/233754> 1442160 24 rniwa 2018-07-13 21:21:02 -0700 Looks like this latest change finally made it stable! 1442224 25 saam 2018-07-14 11:01:33 -0700 (In reply to Ryosuke Niwa from comment #24) > Looks like this latest change finally made it stable! Great! Thanks Ryosuke 344423 2018-07-06 10:02:13 -0700 2018-07-11 20:34:39 -0700 patch b-backup.diff text/plain 1414 saam SW5kZXg6IExheW91dFRlc3RzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBMYXlvdXRUZXN0cy9D aGFuZ2VMb2cJKHJldmlzaW9uIDIzMzU3OCkKKysrIExheW91dFRlc3RzL0NoYW5nZUxvZwkod29y a2luZyBjb3B5KQpAQCAtMSwzICsxLDE0IEBACisyMDE4LTA3LTA2ICBTYWFtIEJhcmF0aSAgPHNi YXJhdGlAYXBwbGUuY29tPgorCisgICAgICAgIExheW91dCBUZXN0IGVkaXRpbmcvc2VsZWN0aW9u L25hdmlnYXRpb24tY2xlYXJzLWVkaXRvci1zdGF0ZS5odG1sIGlzIGZsYWt5CisgICAgICAgIGh0 dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNnaT9pZD0xODczODkKKworICAgICAgICBS ZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAgICAgICAqIGVkaXRpbmcvc2VsZWN0aW9u L25hdmlnYXRpb24tY2xlYXJzLWVkaXRvci1zdGF0ZS5odG1sOgorICAgICAgICBNYWtlIHRoZSB0 ZXN0J3MgaW50ZXJuYWwgdGltZSBvdXQgbWVjaGFuaXNtIGxvbmdlciBzaW5jZQorICAgICAgICBp dCdzIGJlaW5nIGhpdCBvbiB0aGUgYm90cy4KKwogMjAxOC0wNy0wNiAgRGFuaWVsIEJhdGVzICA8 ZGFiYXRlc0BhcHBsZS5jb20+CiAKICAgICAgICAgUmVtb3ZlIFN0cm9uZyBDb25maXJtYXRpb24g UGFzc3dvcmQgYnV0dG9uCkluZGV4OiBMYXlvdXRUZXN0cy9lZGl0aW5nL3NlbGVjdGlvbi9uYXZp Z2F0aW9uLWNsZWFycy1lZGl0b3Itc3RhdGUuaHRtbAo9PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBMYXlvdXRUZXN0 cy9lZGl0aW5nL3NlbGVjdGlvbi9uYXZpZ2F0aW9uLWNsZWFycy1lZGl0b3Itc3RhdGUuaHRtbAko cmV2aXNpb24gMjMzNTc4KQorKysgTGF5b3V0VGVzdHMvZWRpdGluZy9zZWxlY3Rpb24vbmF2aWdh dGlvbi1jbGVhcnMtZWRpdG9yLXN0YXRlLmh0bWwJKHdvcmtpbmcgY29weSkKQEAgLTcyLDcgKzcy LDcgQEAgaWYgKCF3aW5kb3cuR0NDb250cm9sbGVyIHx8ICF3aW5kb3cuaW50ZQogICAgIGRlYnVn KCdUaGlzIHRlc3QgcmVxdWlyZXMgR0NDb250cm9sbGVyIGFuZCBpbnRlcm5hbHMnKTsKIH0gZWxz ZSB7CiAgICAgaWYgKHdpbmRvdy50ZXN0UnVubmVyKQotICAgICAgICBzZXRUaW1lb3V0KCgpID0+ IHRlc3RSdW5uZXIubm90aWZ5RG9uZSgpLCA1MDAwKTsKKyAgICAgICAgc2V0VGltZW91dCgoKSA9 PiB0ZXN0UnVubmVyLm5vdGlmeURvbmUoKSwgMTgwMDApOwogICAgIC8vIENsZWFyIG91dCBhbnkg bGluZ2VyaW5nIGRvY3VtZW50cyBmcm9tIHRoZSBwcmV2aW91cyB0ZXN0cy4KICAgICBHQ0NvbnRy b2xsZXIuY29sbGVjdCgpOwogICAgIEdDQ29udHJvbGxlci5jb2xsZWN0KCk7Cg== 344816 2018-07-11 20:34:40 -0700 2018-07-11 20:35:11 -0700 Another fix attempt bug-187389-20180711203440.patch text/plain 1896 rniwa SW5kZXg6IExheW91dFRlc3RzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBMYXlvdXRUZXN0cy9D aGFuZ2VMb2cJKHJldmlzaW9uIDIzMzc1MykKKysrIExheW91dFRlc3RzL0NoYW5nZUxvZwkod29y a2luZyBjb3B5KQpAQCAtMSwzICsxLDE1IEBACisyMDE4LTA3LTExICBSeW9zdWtlIE5pd2EgIDxy bml3YUB3ZWJraXQub3JnPgorCisgICAgICAgIExheW91dCBUZXN0IGVkaXRpbmcvc2VsZWN0aW9u L25hdmlnYXRpb24tY2xlYXJzLWVkaXRvci1zdGF0ZS5odG1sIGlzIGZsYWt5CisgICAgICAgIGh0 dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNnaT9pZD0xODczODkKKworICAgICAgICBS ZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAgICAgICBVc2UgMjAwIGl0ZXJhdGlvbnMg bGlrZSB3ZSBkaWQgYmVmb3JlIHIyMzM3MDEgYW5kIG1hcmsgaXQgYXMgU0xPVy4KKworICAgICAg ICAqIFRlc3RFeHBlY3RhdGlvbnM6CisgICAgICAgICogZWRpdGluZy9zZWxlY3Rpb24vbmF2aWdh dGlvbi1jbGVhcnMtZWRpdG9yLXN0YXRlLmh0bWw6CisKIDIwMTgtMDctMTEgIEFkaXR5YSBLZWVy dGhpICA8YWtlZXJ0aGlAYXBwbGUuY29tPgogCiAgICAgICAgIFJFR1JFU1NJT04gKDIzMTI3Nik6 IEF0dGVtcHRpbmcgdG8gY29weSBhbiBpbWFnZSBmYWlscwpJbmRleDogTGF5b3V0VGVzdHMvVGVz dEV4cGVjdGF0aW9ucwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBMYXlvdXRUZXN0cy9UZXN0RXhwZWN0YXRpb25z CShyZXZpc2lvbiAyMzM3NTIpCisrKyBMYXlvdXRUZXN0cy9UZXN0RXhwZWN0YXRpb25zCSh3b3Jr aW5nIGNvcHkpCkBAIC05MTgsNiArOTE4LDggQEAgd2Via2l0Lm9yZy9iLzYxOTMyIFsgRGVidWcg XSBqcXVlcnkvbWFuaQogWyBEZWJ1ZyBdIGpxdWVyeS9vZmZzZXQuaHRtbCBbIFNsb3cgXQogWyBE ZWJ1ZyBdIGpxdWVyeS90cmF2ZXJzaW5nLmh0bWwgWyBTbG93IF0KIAord2Via2l0Lm9yZy9iLzE4 NzM4OSBbIERlYnVnIF0gZWRpdGluZy9zZWxlY3Rpb24vbmF2aWdhdGlvbi1jbGVhcnMtZWRpdG9y LXN0YXRlLmh0bWwgWyBTbG93IF0KKwogd2Via2l0Lm9yZy9iLzEzOTYzOCBbIERlYnVnIF0gc3Zn L2ZpbHRlcnMvc3ZnLWRlZXBseS1uZXN0ZWQtY3Jhc2guaHRtbCBbIFNsb3cgXQogd2Via2l0Lm9y Zy9iLzEyNjQyMCBzdmcvY3VzdG9tL3Jlc291cmNlLWludmFsaWRhdGlvbi1jcmFzaC5zdmcgWyBG YWlsdXJlIFBhc3MgXQogCkluZGV4OiBMYXlvdXRUZXN0cy9lZGl0aW5nL3NlbGVjdGlvbi9uYXZp Z2F0aW9uLWNsZWFycy1lZGl0b3Itc3RhdGUuaHRtbAo9PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBMYXlvdXRUZXN0 cy9lZGl0aW5nL3NlbGVjdGlvbi9uYXZpZ2F0aW9uLWNsZWFycy1lZGl0b3Itc3RhdGUuaHRtbAko cmV2aXNpb24gMjMzNzUyKQorKysgTGF5b3V0VGVzdHMvZWRpdGluZy9zZWxlY3Rpb24vbmF2aWdh dGlvbi1jbGVhcnMtZWRpdG9yLXN0YXRlLmh0bWwJKHdvcmtpbmcgY29weSkKQEAgLTM2LDcgKzM2 LDcgQEAgYXN5bmMgZnVuY3Rpb24gcnVuVGVzdCgpCiAgICAgaW5pdGlhbERvY3VtZW50Q291bnQg PSBpbnRlcm5hbHMubnVtYmVyT2ZMaXZlRG9jdW1lbnRzKCk7CiAKICAgICBsZXQgZnJhbWVzID0g W107Ci0gICAgY29uc3QgY291bnQgPSAxMDA7CisgICAgY29uc3QgY291bnQgPSAyMDA7CiAgICAg Zm9yIChsZXQgaSA9IDA7IGkgPCBjb3VudDsgKytpKQogICAgICAgICBmcmFtZXMucHVzaChhcHBl bmRJZnJhbWUoKSk7CiAK