230684 2021-09-23 04:47:17 -0700 LayoutTestFinder doesn't return tests in order 2021-09-24 11:50:15 -0700 1 1 1 Unclassified WebKit Tools / Tests WebKit Local Build Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=230764 InRadar P2 Normal --- 1 gsnedders gsnedders ap ews-watchlist glenn jbedard webkit-bug-importer oldest_to_newest 1796452 0 gsnedders 2021-09-23 04:47:17 -0700 It is clearly intended that LayoutTestFinder returns the tests in a deterministic order. But it doesn't actually do that: top-level directories appear in an arbitrary order. Found while rewriting LayoutTestFinder and trying to compare my results against what we currently have. 1796574 1 439069 gsnedders 2021-09-23 11:04:44 -0700 Created attachment 439069 Patch 1796740 2 439069 ap 2021-09-23 17:51:44 -0700 Comment on attachment 439069 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=439069&action=review > Tools/ChangeLog:10 > + running run-webkit-tests with no arguments ends up running the top-level directories in a Is this a recent regression? In my experience, accessibility and animations always went first. 1796853 3 gsnedders 2021-09-24 05:45:30 -0700 (In reply to Alexey Proskuryakov from comment #2) > Comment on attachment 439069 [details] > Patch > > View in context: > https://bugs.webkit.org/attachment.cgi?id=439069&action=review > > > Tools/ChangeLog:10 > > + running run-webkit-tests with no arguments ends up running the top-level directories in a > > Is this a recent regression? In my experience, accessibility and animations > always went first. IIRC, it should in practice depend on the order the OS enumerates them (via libc). 1796855 4 ews-feeder 2021-09-24 05:58:48 -0700 Committed r283037 (242097@main): <https://commits.webkit.org/242097@main> All reviewed patches have been landed. Closing bug and clearing flags on attachment 439069. 1796856 5 webkit-bug-importer 2021-09-24 05:59:19 -0700 <rdar://problem/83493574> 439069 2021-09-23 11:04:44 -0700 2021-09-24 05:58:49 -0700 Patch bug-230684-20210923190442.patch text/plain 5352 gsnedders U3VidmVyc2lvbiBSZXZpc2lvbjogMjgyOTU0CmRpZmYgLS1naXQgYS9Ub29scy9DaGFuZ2VMb2cg Yi9Ub29scy9DaGFuZ2VMb2cKaW5kZXggNjIyMDMxYTdhMGNiNDliZDJmMTMyZWEzMDU4ZDJlMmYx OWZlYjEwNC4uY2U3MjJkZDg5ODUyOGExMjZmNTUyYWZlZmFkMDQzNjdjMjViMjAyYSAxMDA2NDQK LS0tIGEvVG9vbHMvQ2hhbmdlTG9nCisrKyBiL1Rvb2xzL0NoYW5nZUxvZwpAQCAtMSwzICsxLDI1 IEBACisyMDIxLTA5LTIzICBTYW0gU25lZGRvbiAgPGdzbmVkZGVyc0BhcHBsZS5jb20+CisKKyAg ICAgICAgTGF5b3V0VGVzdEZpbmRlciBzaG91bGQgcmV0dXJuIHRlc3RzIGluIG9yZGVyCisgICAg ICAgIGh0dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNnaT9pZD0yMzA2ODQKKworICAg ICAgICBSZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAgICAgICBJdCBoYXMgYWx3YXlz IGJlZW4gaW50ZW5kZWQgdGhhdCBMYXlvdXRUZXN0RmluZGVyIHJldHVybiB0ZXN0cyBpbiBhIGRl dGVybWluaXN0aWMgb3JkZXI7CisgICAgICAgIGhvd2V2ZXIsIGEgYnVnIGluIGZpbmRfZmlsZXMg bWVhbnMgdGhhdCBkaXJlY3RvcmllcyBmb3VuZCBieSBnbG9iIGFyZW4ndCBzb3J0ZWQsIGFuZCBo ZW5jZQorICAgICAgICBydW5uaW5nIHJ1bi13ZWJraXQtdGVzdHMgd2l0aCBubyBhcmd1bWVudHMg ZW5kcyB1cCBydW5uaW5nIHRoZSB0b3AtbGV2ZWwgZGlyZWN0b3JpZXMgaW4gYQorICAgICAgICBu b24tZGV0ZXJtaW5pc3RpYyBvcmRlci4KKworICAgICAgICAqIFNjcmlwdHMvd2Via2l0cHkvY29t bW9uL2ZpbmRfZmlsZXMucHk6CisgICAgICAgIChfbm9ybWFsaXplZF9maW5kKToKKyAgICAgICAg KF9ub3JtYWxpemVkX2ZpbmQuc29ydF9ieV9kaXJlY3Rvcnlfa2V5KToKKyAgICAgICAgKiBTY3Jp cHRzL3dlYmtpdHB5L2NvbW1vbi9maW5kX2ZpbGVzX3VuaXR0ZXN0LnB5OgorICAgICAgICAoVGVz dFdpbk5vcm1hbGl6ZS5hc3NlcnRfZmlsZXN5c3RlbV9ub3JtYWxpemVzKToKKyAgICAgICAgKFRl c3RXaW5Ob3JtYWxpemUudGVzdF93aW4pOgorICAgICAgICAoVGVzdEZpbmRGaWxlcyk6CisgICAg ICAgIChUZXN0RmluZEZpbGVzLnRlc3RfZGlyZWN0b3J5X3NvcnRfa2V5KToKKyAgICAgICAgKFRl c3RGaW5kRmlsZXMudGVzdF9kaXJlY3Rvcnlfc29ydF9rZXlfd2l0aF9wYXRocyk6CisKIDIwMjEt MDktMjMgIFNlcmdpbyBWaWxsYXIgU2VuaW4gIDxzdmlsbGFyQGlnYWxpYS5jb20+CiAKICAgICAg ICAgW0ZsYXRwYWtdIFVzZSBOVU1CRVJfT0ZfUFJPQ0VTU09SUyBpZiBhbHJlYWR5IGRlZmluZWQK ZGlmZiAtLWdpdCBhL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvY29tbW9uL2ZpbmRfZmlsZXMucHkg Yi9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L2NvbW1vbi9maW5kX2ZpbGVzLnB5CmluZGV4IDdhMTAx MjBlZmYzNzk4Y2NhOWIxNjNiYTFiMGI4ZjMwZWE4MTFkZDUuLjdiZmZkMWRhNWExMzVjZWQ0Zjdj ZjYyYmViOTU5MTQ5ZTBlZGEwNjAgMTAwNjQ0Ci0tLSBhL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkv Y29tbW9uL2ZpbmRfZmlsZXMucHkKKysrIGIvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9jb21tb24v ZmluZF9maWxlcy5weQpAQCAtNzMsMTIgKzczLDEzIEBAIGRlZiBfbm9ybWFsaXplZF9maW5kKGZp bGVzeXN0ZW0sIHBhdGhzLCBza2lwcGVkX2RpcmVjdG9yaWVzLCBmaWxlX2ZpbHRlciwgZGlyZWN0 CiAgICAgICAgICAgR2xvYiBwYXR0ZXJucyBhcmUgb2suCiAgICAgIiIiCiAKLSAgICBwYXRoc190 b193YWxrID0gaXRlcnRvb2xzLmNoYWluKCooZmlsZXN5c3RlbS5nbG9iKHBhdGgpIGZvciBwYXRo IGluIHBhdGhzKSkKLQogICAgIGRlZiBzb3J0X2J5X2RpcmVjdG9yeV9rZXkoZmlsZXNfbGlzdCk6 Ci0gICAgICAgIGlmIGRpcmVjdG9yeV9zb3J0X2tleToKLSAgICAgICAgICAgIGZpbGVzX2xpc3Qu c29ydChrZXk9ZGlyZWN0b3J5X3NvcnRfa2V5KQotICAgICAgICByZXR1cm4gZmlsZXNfbGlzdAor ICAgICAgICBpZiBub3QgZGlyZWN0b3J5X3NvcnRfa2V5OgorICAgICAgICAgICAgcmV0dXJuIGZp bGVzX2xpc3RbOl0KKworICAgICAgICByZXR1cm4gc29ydGVkKGZpbGVzX2xpc3QsIGtleT1kaXJl Y3Rvcnlfc29ydF9rZXkpCisKKyAgICBwYXRoc190b193YWxrID0gaXRlcnRvb2xzLmNoYWluKCoo c29ydF9ieV9kaXJlY3Rvcnlfa2V5KGZpbGVzeXN0ZW0uZ2xvYihwYXRoKSkgZm9yIHBhdGggaW4g cGF0aHMpKQogCiAgICAgYWxsX2ZpbGVzID0gaXRlcnRvb2xzLmNoYWluKCooc29ydF9ieV9kaXJl Y3Rvcnlfa2V5KGZpbGVzeXN0ZW0uZmlsZXNfdW5kZXIocGF0aCwgc2tpcHBlZF9kaXJlY3Rvcmll cywgZmlsZV9maWx0ZXIpKSBmb3IgcGF0aCBpbiBwYXRoc190b193YWxrKSkKICAgICByZXR1cm4g YWxsX2ZpbGVzCmRpZmYgLS1naXQgYS9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L2NvbW1vbi9maW5k X2ZpbGVzX3VuaXR0ZXN0LnB5IGIvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9jb21tb24vZmluZF9m aWxlc191bml0dGVzdC5weQppbmRleCA3ZmU2Y2E4NjIwMzU0NTAyMDYzNDQ0NjkwZjYwNjE4OTkx YTI4ZjEwLi5jYmE4YjkxNjdmNzRkOWQxNGUxYzNiNGQ2ZDAwNDEyODBhOWFiYjZmIDEwMDY0NAot LS0gYS9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L2NvbW1vbi9maW5kX2ZpbGVzX3VuaXR0ZXN0LnB5 CisrKyBiL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvY29tbW9uL2ZpbmRfZmlsZXNfdW5pdHRlc3Qu cHkKQEAgLTI2LDEyICsyNiwxNSBAQAogIyAoSU5DTFVESU5HIE5FR0xJR0VOQ0UgT1IgT1RIRVJX SVNFKSBBUklTSU5HIElOIEFOWSBXQVkgT1VUIE9GIFRIRSBVU0UKICMgT0YgVEhJUyBTT0ZUV0FS RSwgRVZFTiBJRiBBRFZJU0VEIE9GIFRIRSBQT1NTSUJJTElUWSBPRiBTVUNIIERBTUFHRS4KIAor aW1wb3J0IGl0ZXJ0b29scworaW1wb3J0IHJhbmRvbQogaW1wb3J0IHN5cwogaW1wb3J0IHVuaXR0 ZXN0CiAKIGltcG9ydCB3ZWJraXRweS5jb21tb24uZmluZF9maWxlcyBhcyBmaW5kX2ZpbGVzCiAK IGZyb20gd2Via2l0cHkuY29tbW9uLnN5c3RlbS5maWxlc3lzdGVtIGltcG9ydCBGaWxlU3lzdGVt Citmcm9tIHdlYmtpdHB5LmNvbW1vbi5zeXN0ZW0uZmlsZXN5c3RlbV9tb2NrIGltcG9ydCBNb2Nr RmlsZVN5c3RlbQogCiAKIGNsYXNzIE1vY2tXaW5GaWxlU3lzdGVtKG9iamVjdCk6CkBAIC00NCw5 ICs0NywxOCBAQCBjbGFzcyBNb2NrV2luRmlsZVN5c3RlbShvYmplY3QpOgogCiBjbGFzcyBUZXN0 V2luTm9ybWFsaXplKHVuaXR0ZXN0LlRlc3RDYXNlKToKICAgICBkZWYgYXNzZXJ0X2ZpbGVzeXN0 ZW1fbm9ybWFsaXplcyhzZWxmLCBmaWxlc3lzdGVtKToKLSAgICAgICAgc2VsZi5hc3NlcnRFcXVh bChmaW5kX2ZpbGVzLl9ub3JtYWxpemUoZmlsZXN5c3RlbSwgImM6XFxmb28iLAotICAgICAgICAg ICAgWydmYXN0L2h0bWwnLCAnZmFzdC9jYW52YXMvKicsICdjb21wb3NpdGluZy9mb28uaHRtbCdd KSwKLSAgICAgICAgICAgIFsnYzpcXGZvb1xcZmFzdFxcaHRtbCcsICdjOlxcZm9vXFxmYXN0XFxj YW52YXNcXConLCAnYzpcXGZvb1xcY29tcG9zaXRpbmdcXGZvby5odG1sJ10pCisgICAgICAgIHNl bGYuYXNzZXJ0RXF1YWwoCisgICAgICAgICAgICBmaW5kX2ZpbGVzLl9ub3JtYWxpemUoCisgICAg ICAgICAgICAgICAgZmlsZXN5c3RlbSwKKyAgICAgICAgICAgICAgICAiYzpcXGZvbyIsCisgICAg ICAgICAgICAgICAgWydmYXN0L2h0bWwnLCAnZmFzdC9jYW52YXMvKicsICdjb21wb3NpdGluZy9m b28uaHRtbCddLAorICAgICAgICAgICAgKSwKKyAgICAgICAgICAgIFsKKyAgICAgICAgICAgICAg ICAnYzpcXGZvb1xcZmFzdFxcaHRtbCcsCisgICAgICAgICAgICAgICAgJ2M6XFxmb29cXGZhc3Rc XGNhbnZhc1xcKicsCisgICAgICAgICAgICAgICAgJ2M6XFxmb29cXGNvbXBvc2l0aW5nXFxmb28u aHRtbCcsCisgICAgICAgICAgICBdLAorICAgICAgICApCiAKICAgICBkZWYgdGVzdF9tb2NrZWRf d2luKHNlbGYpOgogICAgICAgICAjIFRoaXMgdGVzdHMgdGVzdF9maWxlcy5ub3JtYWxpemUsIHVz aW5nIHBvcnRhYmxlIGJlaGF2aW9yIGVtdWxhdGluZwpAQCAtNjAsMyArNzIsMzQgQEAgY2xhc3Mg VGVzdFdpbk5vcm1hbGl6ZSh1bml0dGVzdC5UZXN0Q2FzZSk6CiAgICAgICAgIGlmIG5vdCBzeXMu cGxhdGZvcm0uc3RhcnRzd2l0aCgnd2luJyk6CiAgICAgICAgICAgICByZXR1cm4KICAgICAgICAg c2VsZi5hc3NlcnRfZmlsZXN5c3RlbV9ub3JtYWxpemVzKEZpbGVTeXN0ZW0oKSkKKworCitjbGFz cyBUZXN0RmluZEZpbGVzKHVuaXR0ZXN0LlRlc3RDYXNlKToKKyAgICBkZWYgdGVzdF9kaXJlY3Rv cnlfc29ydF9rZXkoc2VsZik6CisgICAgICAgIGZpbGVuYW1lcyA9IFtjaHIobykgZm9yIG8gaW4g cmFuZ2Uob3JkKCJhIiksIG9yZCgieiIpICsgMSldCisgICAgICAgIGZzID0gTW9ja0ZpbGVTeXN0 ZW0oCisgICAgICAgICAgICBmaWxlcz17YzogIiIgZm9yIGMgaW4gcmFuZG9tLnNhbXBsZShmaWxl bmFtZXMsIGxlbihmaWxlbmFtZXMpKX0KKyAgICAgICAgKQorICAgICAgICBzZWxmLmFzc2VydEVx dWFsKAorICAgICAgICAgICAgbGlzdChmaW5kX2ZpbGVzLmZpbmQoZnMsICIiLCBkaXJlY3Rvcnlf c29ydF9rZXk9bGFtYmRhIHg6IHgpKSwKKyAgICAgICAgICAgIHNvcnRlZChmaWxlbmFtZXMpLAor ICAgICAgICApCisKKyAgICBkZWYgdGVzdF9kaXJlY3Rvcnlfc29ydF9rZXlfd2l0aF9wYXRocyhz ZWxmKToKKyAgICAgICAgZmlsZW5hbWVzID0gWyIvIi5qb2luKGkpIGZvciBpIGluIGl0ZXJ0b29s cy5wcm9kdWN0KCJhYmNkZSIsICIxMjM0NSIpXQorICAgICAgICBmcyA9IE1vY2tGaWxlU3lzdGVt KAorICAgICAgICAgICAgZmlsZXM9e2M6ICIiIGZvciBjIGluIHJhbmRvbS5zYW1wbGUoZmlsZW5h bWVzLCBsZW4oZmlsZW5hbWVzKSl9CisgICAgICAgICkKKworICAgICAgICB0ZXN0X3N1YnNldCA9 IHJhbmRvbS5zYW1wbGUoImFiY2RlIiwgMykKKyAgICAgICAgc2VsZi5hc3NlcnRFcXVhbCgKKyAg ICAgICAgICAgIGxpc3QoCisgICAgICAgICAgICAgICAgZmluZF9maWxlcy5maW5kKAorICAgICAg ICAgICAgICAgICAgICBmcywKKyAgICAgICAgICAgICAgICAgICAgIiIsCisgICAgICAgICAgICAg ICAgICAgIHBhdGhzPVtpICsgIi8qIiBmb3IgaSBpbiB0ZXN0X3N1YnNldF0sCisgICAgICAgICAg ICAgICAgICAgIGRpcmVjdG9yeV9zb3J0X2tleT1sYW1iZGEgeDogeCwKKyAgICAgICAgICAgICAg ICApCisgICAgICAgICAgICApLAorICAgICAgICAgICAgWyIvIi5qb2luKGkpIGZvciBpIGluIGl0 ZXJ0b29scy5wcm9kdWN0KHRlc3Rfc3Vic2V0LCAiMTIzNDUiKV0sCisgICAgICAgICkK