95144 2012-08-27 15:41:56 -0700 Skipping tests on platform mac due to WebCore::JSEventListener::jsFunction(WebCore::ScriptExecutionContext*) crashes 2012-08-27 17:01:50 -0700 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) Unspecified Unspecified RESOLVED FIXED P2 Normal --- 1 mark.lam webkit-unassigned barraclough ggaren oliver webkit.review.bot oldest_to_newest 705789 0 mark.lam 2012-08-27 15:41:56 -0700 Skipping tests on platform mac due to WebCore::JSEventListener::jsFunction(WebCore::ScriptExecutionContext*) crashes as detailed in https://bugs.webkit.org/show_bug.cgi?id=93878. The tests that will be skipped are: - fast/events/keyevent-iframe-removed-crash.html - fullscreen/full-screen-iframe-zIndex.html - fullscreen/full-screen-restrictions.html - http/tests/security/MessagePort/event-listener-context.html - platform/mac/accessibility/search-with-frames.html - svg/custom/use-instanceRoot-as-event-target.xhtml 705804 1 160833 mark.lam 2012-08-27 15:47:46 -0700 Created attachment 160833 gardening. 705886 2 mark.lam 2012-08-27 16:58:17 -0700 How did I choose which tests to skip? ======================= I manually sampled hundreds of the bot test failure results including going back to very early tests whose results are still on the server but are not conveniently presented in the waterfall results. I did this by going to the following URL for Apple Lion WK1 debug: http://build.webkit.org/builders/Apple%20Lion%20Debug%20WK1%20%28Tests%29/builds/<build number> This is how I found build 1713 which was the earliest build I saw this crash manifest on. For each occurrence of the crash stack trace I encountered, I recorded the build number and the failing test that the crash stack trace was observed on. Based on these records, I was able to observe that there is a consistency in which tests fail due to this crash. Subsequently, I run each of those tests individually and repeatedly on my local machine like so: oss/webkit4$ perl ./Tools/Scripts/run-webkit-tests --no-launch-safari --no-new-test-results --no-sample-on-timeout --results-directory ./layout-test-results --use-remote-links-to-tests --debug --no-build --exit-after-n-crashes-or-timeouts 20 --exit-after-n-failures 500 -v --repeat-each 1000 <test name> If the test crashes as expected with the same stack trace, then I consider it a worthy candidate to be skipped. The fact that it crashes suggests that this test has the trigger / originating condition for manifesting the crash. There is no guarantee that once we exclude these tests that the crashes will go away altogether. Since the crashes are intermittent, there may be a few more straggles who just haven't manifested with this symptom yet. We would expect the set of the stragglers to be less in count than the set I am now skipping. Alternatively, there's a chance that the symptom will just start manifesting on other tests. But if that case happens, then we will see a whole new set of failures with this crash of about the same size as the original set. If that occurs, we will now have evidence that the crash can occur just anywhere, and we can and should revert this current change set for skipping these tests. Note: there were some tests that were seen to manifest the crash failure, but when run alone, do not manifest the crash. This suggests that there are tests that do not have the trigger / originating condition of the crash. As a result, if we successfully skip all the tests that do trigger the crash, there is a possibility that the crash will cease to manifest. Also, the test that do not manifest the crash when run alone, though previously seen in reported bot failures, are excluded from the skip list. 705889 3 160833 webkit.review.bot 2012-08-27 17:01:47 -0700 Comment on attachment 160833 gardening. Clearing flags on attachment: 160833 Committed r126821: <http://trac.webkit.org/changeset/126821> 705891 4 webkit.review.bot 2012-08-27 17:01:50 -0700 All reviewed patches have been landed. Closing bug. 160833 2012-08-27 15:47:46 -0700 2012-08-27 17:01:47 -0700 gardening. bug-95144.patch text/plain 2124 mark.lam SW5kZXg6IExheW91dFRlc3RzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBMYXlvdXRUZXN0cy9D aGFuZ2VMb2cJKHJldmlzaW9uIDEyNjgwOSkKKysrIExheW91dFRlc3RzL0NoYW5nZUxvZwkod29y a2luZyBjb3B5KQpAQCAtMSwzICsxLDE5IEBACisyMDEyLTA4LTI3ICBNYXJrIExhbSAgPG1hcmsu bGFtQGFwcGxlLmNvbT4KKworICAgICAgICBHYXJkZW5pbmc6IHNraXBwaW5nIHRlc3RzIGR1ZSB0 byBXZWJDb3JlOjpKU0V2ZW50TGlzdGVuZXI6OmpzRnVuY3Rpb24oV2ViQ29yZTo6U2NyaXB0RXhl Y3V0aW9uQ29udGV4dCopIGNyYXNoZXMuCisgICAgICAgIGh0dHBzOi8vYnVncy53ZWJraXQub3Jn L3Nob3dfYnVnLmNnaT9pZD05NTE0NC4KKworICAgICAgICBSZXZpZXdlZCBieSBOT0JPRFkgKE9P UFMhKS4KKworICAgICAgICBmYXN0L2V2ZW50cy9rZXlldmVudC1pZnJhbWUtcmVtb3ZlZC1jcmFz aC5odG1sCisgICAgICAgIGZ1bGxzY3JlZW4vZnVsbC1zY3JlZW4taWZyYW1lLXpJbmRleC5odG1s CisgICAgICAgIGZ1bGxzY3JlZW4vZnVsbC1zY3JlZW4tcmVzdHJpY3Rpb25zLmh0bWwKKyAgICAg ICAgaHR0cC90ZXN0cy9zZWN1cml0eS9NZXNzYWdlUG9ydC9ldmVudC1saXN0ZW5lci1jb250ZXh0 Lmh0bWwKKyAgICAgICAgcGxhdGZvcm0vbWFjL2FjY2Vzc2liaWxpdHkvc2VhcmNoLXdpdGgtZnJh bWVzLmh0bWwKKyAgICAgICAgc3ZnL2N1c3RvbS91c2UtaW5zdGFuY2VSb290LWFzLWV2ZW50LXRh cmdldC54aHRtbAorCisgICAgICAgICogcGxhdGZvcm0vbWFjL1NraXBwZWQ6CisKIDIwMTItMDgt MjcgIExlbyBZYW5nICA8bGVveWFuZ0ByaW0uY29tPgogCiAgICAgICAgIFtCbGFja0JlcnJ5XSBU ZXN0IGV4cGVjdGF0aW9uIGZvciBmYXN0L2pzL2NvbnN0cnVjdG9yLWxlbmd0aC5odG1sCkluZGV4 OiBMYXlvdXRUZXN0cy9wbGF0Zm9ybS9tYWMvU2tpcHBlZAo9PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBMYXlvdXRU ZXN0cy9wbGF0Zm9ybS9tYWMvU2tpcHBlZAkocmV2aXNpb24gMTI2ODA2KQorKysgTGF5b3V0VGVz dHMvcGxhdGZvcm0vbWFjL1NraXBwZWQJKHdvcmtpbmcgY29weSkKQEAgLTEwMTYsMTAgKzEwMTYs MTkgQEAgY29tcG9zaXRpbmcvb3ZlcmZsb3cvdGV4dGFyZWEtc2Nyb2xsLXRvdQogZmFzdC9ibG9j ay9mbG9hdC8wMTYuaHRtbAogCiAjIGh0dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNn aT9pZD05Mzg3OAotIyBEdWUgdG8gSlNDOjpCaW5kaW5nczo6SW5zdGFuY2U6OmNyZWF0ZVJ1bnRp bWVPYmplY3QoSlNDOjpFeGVjU3RhdGUqKSBjcmFzaDoKKyMgRHVlIHRvIEpTQzo6QmluZGluZ3M6 Okluc3RhbmNlOjpjcmVhdGVSdW50aW1lT2JqZWN0KEpTQzo6RXhlY1N0YXRlKikgY3Jhc2hlczoK IGVkaXRpbmcvcGFzdGVib2FyZC9wYXN0ZS1USUZGLmh0bWwKIHBsYXRmb3JtL21hYy9lZGl0aW5n L3Bhc3RlYm9hcmQvZGF0YVRyYW5zZmVyLXNldC1kYXRhLWZpbGUtdXJsLmh0bWwKIAorIyBodHRw czovL2J1Z3Mud2Via2l0Lm9yZy9zaG93X2J1Zy5jZ2k/aWQ9OTM4NzggKHllcywgc2FtZSBidWcg YXMgdGhlIHByZXZpb3VzKQorIyBEdWUgdG8gV2ViQ29yZTo6SlNFdmVudExpc3RlbmVyOjpqc0Z1 bmN0aW9uKFdlYkNvcmU6OlNjcmlwdEV4ZWN1dGlvbkNvbnRleHQqKSBjcmFzaGVzOgorZmFzdC9l dmVudHMva2V5ZXZlbnQtaWZyYW1lLXJlbW92ZWQtY3Jhc2guaHRtbAorZnVsbHNjcmVlbi9mdWxs LXNjcmVlbi1pZnJhbWUtekluZGV4Lmh0bWwKK2Z1bGxzY3JlZW4vZnVsbC1zY3JlZW4tcmVzdHJp Y3Rpb25zLmh0bWwKK2h0dHAvdGVzdHMvc2VjdXJpdHkvTWVzc2FnZVBvcnQvZXZlbnQtbGlzdGVu ZXItY29udGV4dC5odG1sCitwbGF0Zm9ybS9tYWMvYWNjZXNzaWJpbGl0eS9zZWFyY2gtd2l0aC1m cmFtZXMuaHRtbAorc3ZnL2N1c3RvbS91c2UtaW5zdGFuY2VSb290LWFzLWV2ZW50LXRhcmdldC54 aHRtbAorCiAjIEFzc29ydGVkIGZhaWx1cmVzIHRoYXQgbmVlZCBpbnZlc3RpZ2F0aW9uCiAKICMg LS0tIENhbnZhcyAtLS0K