95096 2012-08-27 09:39:20 -0700 Random test-webkitpy failures on the buildbot 2012-08-29 23:18:57 -0700 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) Unspecified Unspecified RESOLVED FIXED P2 Normal --- 1 zan zan dpranke pnormand tmpsantos oldest_to_newest 705329 0 zan 2012-08-27 09:39:20 -0700 When running webkitpy tests on the GTK 64-bit Release there are failures occurring every few runs: http://build.webkit.org/builders/GTK%20Linux%2064-bit%20Release/builds/28017/steps/webkitpy-test/logs/stdio http://build.webkit.org/builders/GTK%20Linux%2064-bit%20Release/builds/28008/steps/webkitpy-test/logs/stdio http://build.webkit.org/builders/GTK%20Linux%2064-bit%20Release/builds/28007/steps/webkitpy-test/logs/stdio http://build.webkit.org/builders/GTK%20Linux%2064-bit%20Release/builds/28006/steps/webkitpy-test/logs/stdio This is really annoying as it's causing unnecessary redness. Looking through other builders, I only found one similar failure occurrence on the Chromium Linux Tests builder: http://build.webkit.org/builders/Chromium%20Linux%20Release%20%28Tests%29/builds/37362/steps/webkitpy-test/logs/stdio This probably assures that the failures are not port- or builder-specific. Rather than that I'd blame the python tests being run in parallel. On the GTK 64-bit Release builder this means up to 24 tests run at the same time. I can't reproduce the failures locally in any way, the system being capable of running only 4 tests in parallel. OTOH, with all the tests being run serially (child_processes=1) the tests pass consistently and still take roughly 10 seconds to run (as compared to the 3-3.5 seconds when run in parallel (locally)). Because of that I'd propose to either run the python tests serially on all the builders or at least the GTK-specific ones. Dirk, your comments are most welcome. 706757 1 dpranke 2012-08-28 14:51:37 -0700 ideally we should figure out why the tests are failing when run in parallel. that said, redness is bad so I'm fine if we disable parallel testing on some of the builders, either as a temporary fix while we fix these tests, or indefinitely until you start worrying about cycle time. That said, I don't want to disable it on all of them, I'm happy to leave it on for Chromium, for example. On a side note, we should poke into why test-webkitpy thinks it's logging to a terminal rather than a logfile. Any chance usePty = True in your buildbot slave config? 707392 2 161209 zan 2012-08-29 06:20:43 -0700 Created attachment 161209 Run test-webkitpy in 1 child process on GTK builders 708338 3 161209 zan 2012-08-29 23:18:52 -0700 Comment on attachment 161209 Run test-webkitpy in 1 child process on GTK builders Clearing flags on attachment: 161209 Committed r127104: <http://trac.webkit.org/changeset/127104> 708339 4 zan 2012-08-29 23:18:57 -0700 All reviewed patches have been landed. Closing bug. 161209 2012-08-29 06:20:43 -0700 2012-08-29 23:18:52 -0700 Run test-webkitpy in 1 child process on GTK builders bug-95096-20120829062033.patch text/plain 2183 zan U3VidmVyc2lvbiBSZXZpc2lvbjogMTI2ODkwCmRpZmYgLS1naXQgYS9Ub29scy9DaGFuZ2VMb2cg Yi9Ub29scy9DaGFuZ2VMb2cKaW5kZXggYmU4ZmNlYzIzM2ZiMDIxM2E0ZjYzYzQ1Y2ZkOTY2NjU0 OWEyZTJkYy4uNzMxODI4M2YxM2JhZWZlMjM4NWFlMjg5MDMwZGQ4MzNiOGEzODBjMCAxMDA2NDQK LS0tIGEvVG9vbHMvQ2hhbmdlTG9nCisrKyBiL1Rvb2xzL0NoYW5nZUxvZwpAQCAtMSwzICsxLDIx IEBACisyMDEyLTA4LTI5ICBaYW4gRG9iZXJzZWsgIDx6YW5kb2JlcnNla0BnbWFpbC5jb20+CisK KyAgICAgICAgUmFuZG9tIHRlc3Qtd2Via2l0cHkgZmFpbHVyZXMgb24gdGhlIGJ1aWxkYm90Cisg ICAgICAgIGh0dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNnaT9pZD05NTA5NgorCisg ICAgICAgIFJldmlld2VkIGJ5IE5PQk9EWSAoT09QUyEpLgorCisgICAgICAgIFN1cHByZXNzIG9j Y2FzaW9uYWwgZXJyb3JzIHdoZW4gcnVubmluZyB0ZXN0LXdlYmtpdHB5IG9uIEdUSyBidWlsZGVy cworICAgICAgICBieSBydW5uaW5nIHRoZXNlIHRlc3RzIHNlcmlhbGx5LiBUaGUgNjQtYml0IFJl bGVhc2UgYnVpbGRlciBpcyBlc3BlY2lhbGx5CisgICAgICAgIHByb25lIHRvIHRoZXNlIGFzIGl0 IGNhbiBydW4gdXAgdG8gMjQgdGVzdHMgaW4gcGFyYWxsZWwuCisKKyAgICAgICAgRGVzcGl0ZSB0 aGUgdGVzdHMgbm90IGJlaW5nIHJ1biBpbiBwYXJhbGxlbCwgdGhlIHRlc3Rpbmcgb25seSBsYXN0 cyBhIGhhbmRmdWwKKyAgICAgICAgb2Ygc2Vjb25kcyBtb3JlLCBzbyB0aGlzIGlzIGEgd29ydGh5 IHRyYWRlLW9mZiB0byBhdm9pZCB1bm5lY2Vzc2FyeQorICAgICAgICBmYWxzZS1hbGFybSByZWRu ZXNzIG9uIHRoZSBHVEsgYnVpbGRlcnMuCisKKyAgICAgICAgKiBCdWlsZFNsYXZlU3VwcG9ydC9i dWlsZC53ZWJraXQub3JnLWNvbmZpZy9tYXN0ZXIuY2ZnOgorICAgICAgICAoUnVuUHl0aG9uVGVz dHMuc3RhcnQpOgorCiAyMDEyLTA4LTI4ICBHYWJvciBSYXBjc2FueWkgIDxyZ2Fib3JAd2Via2l0 Lm9yZz4KIAogICAgICAgICBbUXRdIEtlZXAgUVRfUVBBX1BMQVRGT1JNX1BMVUdJTl9QQVRIIGVu dmlyb25tZW50IHZhcmlhYmxlIGluIE5SV1QKZGlmZiAtLWdpdCBhL1Rvb2xzL0J1aWxkU2xhdmVT dXBwb3J0L2J1aWxkLndlYmtpdC5vcmctY29uZmlnL21hc3Rlci5jZmcgYi9Ub29scy9CdWlsZFNs YXZlU3VwcG9ydC9idWlsZC53ZWJraXQub3JnLWNvbmZpZy9tYXN0ZXIuY2ZnCmluZGV4IDM2M2U4 OTIxYWZmOTI4ZDFlOTc5NWY1MzQ0ZDk1MjI0ZjdmMmM3MzguLjgzNDE0YTBlZmMwMjRhYjgzNTgx ODc0ZTdlZDE5NGIxNmQ4MWJiNjUgMTAwNjQ0Ci0tLSBhL1Rvb2xzL0J1aWxkU2xhdmVTdXBwb3J0 L2J1aWxkLndlYmtpdC5vcmctY29uZmlnL21hc3Rlci5jZmcKKysrIGIvVG9vbHMvQnVpbGRTbGF2 ZVN1cHBvcnQvYnVpbGQud2Via2l0Lm9yZy1jb25maWcvbWFzdGVyLmNmZwpAQCAtNDU3LDYgKzQ1 NywxNCBAQCBjbGFzcyBSdW5QeXRob25UZXN0cyhUZXN0V2l0aEZhaWx1cmVDb3VudCk6CiAgICAg Y29tbWFuZCA9IFsicHl0aG9uIiwgIi4vVG9vbHMvU2NyaXB0cy90ZXN0LXdlYmtpdHB5Il0KICAg ICBmYWlsZWRUZXN0c0Zvcm1hdFN0cmluZyA9ICIlZCBweXRob24gdGVzdHMgZmFpbGVkIgogCisg ICAgZGVmIHN0YXJ0KHNlbGYpOgorICAgICAgICBwbGF0Zm9ybSA9IHNlbGYuZ2V0UHJvcGVydHko J3BsYXRmb3JtJykKKyAgICAgICAgIyBQeXRob24gdGVzdHMgYXJlIGZsYWt5IG9uIHRoZSBHVEsg YnVpbGRlcnMsIHJ1bm5pbmcgdGhlbSBzZXJpYWxseQorICAgICAgICAjIGhlbHBzIGFuZCBkb2Vz IG5vdCBzaWduaWZpY2FudGx5IHByb2xvbmcgdGhlIGN5Y2xlIHRpbWUuCisgICAgICAgIGlmIHBs YXRmb3JtID09ICdndGsnOgorICAgICAgICAgICAgc2VsZi5zZXRDb21tYW5kKHNlbGYuY29tbWFu ZCArIFsnLS1jaGlsZC1wcm9jZXNzZXMnLCAnMSddKQorICAgICAgICByZXR1cm4gc2hlbGwuVGVz dC5zdGFydChzZWxmKQorCiAgICAgZGVmIGNvdW50RmFpbHVyZXMoc2VsZiwgY21kKToKICAgICAg ICAgbG9nVGV4dCA9IGNtZC5sb2dzWydzdGRpbyddLmdldFRleHQoKQogICAgICAgICAjIFdlJ3Jl IGxvb2tpbmcgZm9yIHRoZSBsaW5lIHRoYXQgbG9va3MgbGlrZSB0aGlzOiBGQUlMRUQgKGZhaWx1 cmVzPTIsIGVycm9ycz0xKQo=