56729 nrwt-wk2 2011-03-20 21:15:00 -0700 Switch webkit2 bot to NRWT 2011-10-24 13:58:23 -0700 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) All All RESOLVED FIXED InRadar P2 Normal --- 63439 63501 63673 63683 63780 64056 67530 70371 70435 34984 66965 1 mjs eric aroben dpranke eric laszlo.gombos lforschler simon.fraser webkit-bug-importer webkit.review.bot oldest_to_newest 370277 0 mjs 2011-03-20 21:15:00 -0700 new-run-webkit-tests lacks old-run-webkit-tests' support for WebKit2. This consists of the --2 option which uses the WebKitTestRunner tool, and support for detecting WebProcess crashes. 427672 1 eric 2011-06-26 23:28:01 -0700 OK. I'm working on NRWT again. It's unclear how much work it will be to add this support. 427696 2 eric 2011-06-27 00:01:09 -0700 Adding support for using WebKitTestRunnner instead of DumpRenderTree seems trivial. 427703 3 eric 2011-06-27 00:04:24 -0700 Looks like this should be as simple as teaching the port how to return driver_name() of "WebKitTestRunner" when passed the --2 option, as well as teaching WebKitDriver._read_block how to understand #CRASHED and #CRASHED - WebProcess 427788 4 eric 2011-06-27 01:56:46 -0700 Did half of what is needed in bug 63439. The harder part of actually making us talk to WebKitTestRunner (and supporting the new #CRASHED stuff) is left. 427872 5 aroben 2011-06-27 05:40:13 -0700 (In reply to comment #3) > Looks like this should be as simple as teaching the port how to return driver_name() of "WebKitTestRunner" when passed the --2 option, as well as teaching WebKitDriver._read_block how to understand #CRASHED and #CRASHED - WebProcess FYI, Apple's Windows port of DRT also prints #CRASHED in some cases, so it would be useful for #CRASHED to be understood even when using DRT. 428588 6 eric 2011-06-27 20:41:28 -0700 new-run-webkit-tests -2 "just works" after bug 63501. I've not yet implemented the #CRASHED handling, but I'm not actually sure it's necessary. We're still correctly detecting crashes, etc. Does WebKitTestRunner continue running after a WebProcess crash, or does it itself exit? 428589 7 eric 2011-06-27 20:42:08 -0700 (My previous comment was not to imply that I'm in any way against the #CRASHED handling, just stating that new-run-webkit-tests -2 seems to work even though we haven't added it yet.) 428593 8 eric 2011-06-27 20:59:06 -0700 After the patch on bug 63501 lands, we still have some 20 unexpected failures with -2. We can use this bug to track such. 428683 9 mjs 2011-06-28 00:36:52 -0700 (In reply to comment #6) > new-run-webkit-tests -2 "just works" after bug 63501. I've not yet implemented the #CRASHED handling, but I'm not actually sure it's necessary. We're still correctly detecting crashes, etc. > > Does WebKitTestRunner continue running after a WebProcess crash, or does it itself exit? Yes, WebKitTestRunner continues after a WebProcess crash (unless the driver script tells it otherwise). That's kind of the point of the WebProcess, to not crash its client when it crashes. Also, if WKTR made itself crash when WebProcess crashes, then results would show a backtrace of the wrong process. 428685 10 mjs 2011-06-28 00:38:22 -0700 (In reply to comment #7) > (My previous comment was not to imply that I'm in any way against the #CRASHED handling, just stating that new-run-webkit-tests -2 seems to work even though we haven't added it yet.) It should work without it, you just need to detect WebProcess crashes and hangs if you don't want the test session to hang indefinitely when it happens. Making the WebProcess crash or hang deliberately (using kill -SEGV to crash or kill -STOP to hang) should be a reasonable way to test it. It could be a separate bug. 477097 11 webkit-bug-importer 2011-10-03 11:26:01 -0700 <rdar://problem/10224918> 487321 12 eric 2011-10-19 21:57:18 -0700 We're ready to pull the trigger. Support is in. 488155 13 111902 eric 2011-10-20 21:58:07 -0700 Created attachment 111902 Patch 488156 14 eric 2011-10-20 22:00:28 -0700 I've tested locally. My machine sees 81 failures with WebKit2, in both ORWT and NRWT. Which matches the 81 failures seen on the bot. 488213 15 111902 webkit.review.bot 2011-10-20 23:48:16 -0700 Comment on attachment 111902 Patch Clearing flags on attachment: 111902 Committed r98074: <http://trac.webkit.org/changeset/98074> 488214 16 webkit.review.bot 2011-10-20 23:48:23 -0700 All reviewed patches have been landed. Closing bug. 488509 17 simon.fraser 2011-10-21 11:12:54 -0700 This seems to have caused lots of plugin tests to fail: http://build.webkit.org/results/Lion%20Intel%20Release%20(Tests)/r98106%20(2090)/results.html 488511 18 eric 2011-10-21 11:15:21 -0700 (In reply to comment #17) > This seems to have caused lots of plugin tests to fail: > http://build.webkit.org/results/Lion%20Intel%20Release%20(Tests)/r98106%20(2090)/results.html That builder doesn't use WK2 does it? I"m not sure that's related to this specific change, but may be related to other changes which I made to support WK2? 488977 19 eric 2011-10-22 18:04:59 -0700 (In reply to comment #17) > This seems to have caused lots of plugin tests to fail: > http://build.webkit.org/results/Lion%20Intel%20Release%20(Tests)/r98106%20(2090)/results.html Do you still believe this to be the case? 489471 20 simon.fraser 2011-10-24 11:30:57 -0700 I think so. Maybe the test plugin isn't getting built? 489497 21 eric 2011-10-24 11:55:00 -0700 Behavior on the lion bot (non-wk2) should not have changed (at least not intentionally). It's unclear which of the tests you cite from: http://build.webkit.org/results/Lion%20Intel%20Release%20(Tests)/r98106%20(2090)/results.html are plugin tests. If you can give me an example test, which you see failing on the bots now with NRWT, or can demonstrate locally as failing with NRWT and passing with ORWT, i'm *very* happy to investigate and fix. Thank you again for your help and patience! 489559 22 simon.fraser 2011-10-24 13:21:31 -0700 Try this list: http://build.webkit.org/results/SnowLeopard%20Intel%20Release%20(WebKit2%20Tests)/r98243%20(15595)/results.html 489579 23 simon.fraser 2011-10-24 13:40:33 -0700 I looked on the bot, and TestNetscapePlugin.plugin is missing from the build results dir for some reason 489585 24 simon.fraser 2011-10-24 13:43:30 -0700 ORWT has: # FIXME: We build both DumpRenderTree and WebKitTestRunner for # WebKitTestRunner runs becuase DumpRenderTree still includes # the DumpRenderTreeSupport module and the TestNetscapePlugin. # These two projects should be factored out into their own # projects. buildDumpTool("DumpRenderTree"); buildDumpTool("WebKitTestRunner") if $useWebKitTestRunner; Do we need the same in NRWT? 489608 25 eric 2011-10-24 13:56:22 -0700 I suspect that's exactly the bug, yes. I'll move that hack into build-webkittestrunner instead of ORWT/NRWT. 489612 26 simon.fraser 2011-10-24 13:57:48 -0700 Yep, that's it. I filed bug 70759. 489614 27 eric 2011-10-24 13:58:23 -0700 (In reply to comment #25) > I suspect that's exactly the bug, yes. I'll move that hack into build-webkittestrunner instead of ORWT/NRWT. Filed https://bugs.webkit.org/show_bug.cgi?id=70760. Will post a patch momentarily. 111902 2011-10-20 21:58:07 -0700 2011-10-20 23:48:16 -0700 Patch bug-56729-20111020215806.patch text/plain 2072 eric U3VidmVyc2lvbiBSZXZpc2lvbjogOTgwNDAKZGlmZiAtLWdpdCBhL1Rvb2xzL0NoYW5nZUxvZyBi L1Rvb2xzL0NoYW5nZUxvZwppbmRleCBjZDhiZTZlNTcxYTI4NjQyODM5OTg3NmEwMmYwZjM4OGQ3 NDJhMDA0Li45M2FkMWVmMTY1ZDhkZGE4MWJlOWYxMzBlMDNiM2RjZmU3YzRjMjY2IDEwMDY0NAot LS0gYS9Ub29scy9DaGFuZ2VMb2cKKysrIGIvVG9vbHMvQ2hhbmdlTG9nCkBAIC0xLDUgKzEsMTkg QEAKIDIwMTEtMTAtMjAgIEVyaWMgU2VpZGVsICA8ZXJpY0B3ZWJraXQub3JnPgogCisgICAgICAg IFN3aXRjaCB3ZWJraXQyIGJvdCB0byBOUldUCisgICAgICAgIGh0dHBzOi8vYnVncy53ZWJraXQu b3JnL3Nob3dfYnVnLmNnaT9pZD01NjcyOQorCisgICAgICAgIFJldmlld2VkIGJ5IE5PQk9EWSAo T09QUyEpLgorCisgICAgICAgIE1vdmUgdGhlIFdLMiBib3QgdG8gdXNpbmcgTlJXVCBpbnN0ZWFk IG9mIE9SV1QsIGFzIGFubm91bmNlZCBvbiB3ZWJraXQtZGV2OgorICAgICAgICBodHRwczovL2xp c3RzLndlYmtpdC5vcmcvcGlwZXJtYWlsL3dlYmtpdC1kZXYvMjAxMS1PY3RvYmVyLzAxODMzNy5o dG1sCisKKyAgICAgICAgKiBTY3JpcHRzL3J1bi13ZWJraXQtdGVzdHM6CisgICAgICAgIChydW5u aW5nT25CdWlsZEJvdCk6CisgICAgICAgICh1c2VOZXdSdW5XZWJLaXRUZXN0cyk6CisKKzIwMTEt MTAtMjAgIEVyaWMgU2VpZGVsICA8ZXJpY0B3ZWJraXQub3JnPgorCiAgICAgICAgIFBvc3NpYmxl IFJFR1JFU1NJT04oOTc4NzkpOiBOUldUIGZhaWxzIHdoZW4gRHVtcFJlbmRlclRyZWUgY3Jhc2hl cwogICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9zaG93X2J1Zy5jZ2k/aWQ9NzA1MjQK IApkaWZmIC0tZ2l0IGEvVG9vbHMvU2NyaXB0cy9ydW4td2Via2l0LXRlc3RzIGIvVG9vbHMvU2Ny aXB0cy9ydW4td2Via2l0LXRlc3RzCmluZGV4IDRkMDFlNjcxNTBiNGQ2MGUzNzgyY2YyYWQ5NGJh ZWY3Y2Q1M2JkMWEuLjAyY2VlOTlmMGExMDY5ODgzNjJmNjc4NjEwYmE3ZmQzZGE0N2E5YTQgMTAw NzU1Ci0tLSBhL1Rvb2xzL1NjcmlwdHMvcnVuLXdlYmtpdC10ZXN0cworKysgYi9Ub29scy9TY3Jp cHRzL3J1bi13ZWJraXQtdGVzdHMKQEAgLTU1LDE2ICs1NSw2IEBAIHN1YiBydW5uaW5nT25CdWls ZEJvdCgpCiAgICAgcmV0dXJuICRpc0J1aWxkQm90VXNlcnskRU5WeyJVU0VSIn19OwogfQogCi0j IFdlIGNvdWxkIHVzZSBpc1dlYktpdDIoKSBpbnN0ZWFkIG9mIG91ciBvd24gY3VzdG9tIHVzaW5n V2ViS2l0MigpLCBidXQgdGhlbgotIyB3ZSdkIGhhdmUgdG8gYmUgY2FyZWZ1bCB0byBhZGQgLTIg b3IgLS13ZWJraXQtdGVzdC1ydW5uZXIgYmFjayBpbnRvIEBBUkdWIGJlbG93Lgotc3ViIHVzaW5n V2ViS2l0MigpCi17Ci0gICAgIyBJZiBlaXRoZXIgLTIgb3IgLS13ZWJraXQtdGVzdC1ydW5uZXIg aXMgcGFzc2VkLCB3ZSdyZSBzdXBwb3NlZCB0byB1c2UgV2ViS2l0Mi4KLSAgICAjIE5SV1QncyBX ZWJLaXQyIHN1cHBvcnQgaXMgc3RpbGwgYSB3b3JrIGluIHByb2dyZXNzOgotICAgICMgaHR0cHM6 Ly9idWdzLndlYmtpdC5vcmcvc2hvd19idWcuY2dpP2lkPTU2NzI5Ci0gICAgcmV0dXJuIGdyZXAo LygtMnwtLXdlYmtpdC10ZXN0LXJ1bm5lcikvLCBAQVJHVik7Ci19Ci0KIHN1YiB1c2luZ0xlYWtz KCkKIHsKICAgICAjIExlYWtzVmlld2VyIGdldHMgY29uZnVzZWQgYnkgTlJXVCdzIC0tbGVha3Mg b3V0cHV0LCBzZWUgYnVnczoKQEAgLTkwLDcgKzgwLDcgQEAgc3ViIHVzZU5ld1J1bldlYktpdFRl c3RzKCkKICAgICAgICAgcmV0dXJuICghaXNEYXJ3aW4oKSBhbmQgIWlzQVJNKCkpOwogICAgIH0K IAotICAgIHJldHVybiAoKGlzTGVvcGFyZCgpIG9yIGlzU25vd0xlb3BhcmQoKSBvciBpc0xpb24o KSBvciBpc0d0aygpKSBhbmQgIXVzaW5nV2ViS2l0MigpIGFuZCAhdXNpbmdMZWFrcygpKTsKKyAg ICByZXR1cm4gKChpc0xlb3BhcmQoKSBvciBpc1Nub3dMZW9wYXJkKCkgb3IgaXNMaW9uKCkgb3Ig aXNHdGsoKSkgYW5kICF1c2luZ0xlYWtzKCkpOwogfQogCiBteSAkaGFybmVzc05hbWUgPSAib2xk LXJ1bi13ZWJraXQtdGVzdHMiOwo=