148501 2015-08-26 17:03:34 -0700 iOS Simulator API tests fails as Simulator is not running 2017-03-15 17:17:46 -0700 1 1 1 Unclassified WebKit Tools / Tests WebKit Local Build Unspecified Unspecified RESOLVED FIXED InRadar P1 Critical --- 1 aakash_jain aakash_jain aakash_jain commit-queue dbates ddkilzer jake.nielsen.webkit oldest_to_newest 1121103 0 aakash_jain 2015-08-26 17:03:34 -0700 see rdar://problem/22447525 1121129 1 260008 aakash_jain 2015-08-26 17:34:14 -0700 Created attachment 260008 Proposed patch Launching the iOS Simulator before starting the iOS Simulator API tests and quitting it after finishing the tests. 1121183 2 260008 ddkilzer 2015-08-26 20:40:08 -0700 Comment on attachment 260008 Proposed patch View in context: https://bugs.webkit.org/attachment.cgi?id=260008&action=review r- to address issue with simulator not quitting when running dumpTestsBySuite(). If dumpTestsBySuite() needs the simulator, then we need another call to quitIOSSimulator(). > Tools/Scripts/run-api-tests:100 > +relaunchIOSSimulator(findOrCreateSimulatorForIOSDevice(SIMULATOR_DEVICE_SUFFIX_FOR_WEBKIT_DEVELOPMENT)) if willUseIOSSimulatorSDKWhenBuilding(); The simulator is not stopped if dumpTestsBySuite() is called below. I think this line should move down to just before the 'exit' line, or into runTestsBySuite() if dumpTestsBySuite() does not need the simulator running. 1121189 3 260008 dbates 2015-08-26 21:08:19 -0700 Comment on attachment 260008 Proposed patch View in context: https://bugs.webkit.org/attachment.cgi?id=260008&action=review > Tools/ChangeLog:10 > + * Scripts/run-api-tests: Launch the iOS Simulator before starting the iOS Simulator API tests and quit it after finishing the tests. > + * Scripts/webkitdirs.pm: Store string "For WebKit Development" to constant SIMULATOR_DEVICE_SUFFIX_FOR_WEBKIT_DEVELOPMENT to make it consistently reusable. These lines are long. We tend to wrap lines longer than 100 characters in a ChangeLog entry. >> Tools/Scripts/run-api-tests:100 >> +relaunchIOSSimulator(findOrCreateSimulatorForIOSDevice(SIMULATOR_DEVICE_SUFFIX_FOR_WEBKIT_DEVELOPMENT)) if willUseIOSSimulatorSDKWhenBuilding(); > > The simulator is not stopped if dumpTestsBySuite() is called below. > > I think this line should move down to just before the 'exit' line, or into runTestsBySuite() if dumpTestsBySuite() does not need the simulator running. We may want to consider adding a FIXME comment above this that explains that we are launch/quit the iOS Simulator as a workaround for <rdar://problem/22388812>. > Tools/Scripts/run-api-tests:162 > + quitIOSSimulator() if willUseIOSSimulatorSDKWhenBuilding(); We should put this logic inside an END block to ensure that we always run this logic on program termination regardless of whether termination is caused by calling die() (which may be invoked in runTest()). We may also want to consider adding a FIXME comment above this that explains that we launch/quit the iOS Simulator as a workaround for <rdar://problem/22388812>. You can read more about the END block at <http://perldoc.perl.org/perlmod.html#BEGIN%2c-UNITCHECK%2c-CHECK%2c-INIT-and-END>. > Tools/Scripts/webkitdirs.pm:84 > USE_OPEN_COMMAND > + SIMULATOR_DEVICE_SUFFIX_FOR_WEBKIT_DEVELOPMENT Please swap the order of these lines so that this list is in sorted order according to the Unix sort command. 1121429 4 260090 aakash_jain 2015-08-27 14:26:22 -0700 Created attachment 260090 Proposed patch @David, moved quitIOSSimulator() to END block so that it is always executed. Also moved the code little bit down, can't move it below runTestsBySuite() as it uses simulator. @Daniel, modified the ChangeLog, added FIXME comments, added the END block and swapped the order of lines. 1121495 5 dbates 2015-08-27 16:58:54 -0700 > Created attachment 260090 [details] > Proposed patch > > @David, moved quitIOSSimulator() to END block so that it is always executed. > Also moved the code little bit down, can't move it below runTestsBySuite() > as it uses simulator. From talking with Aakash today in person, listAllTests() also invokes TestWebKitAPI to get a listing of tests. According to Aakash, invoking TestWebKitAPI to get the list of tests errors out with an error message (@Aakash: can you post the error message?). Aakash noticed this does not seem to happen when the iOS Simulator is not running (why?). So, we launch/relaunch the iOS Simulator before calling listAllTests(). We should further investigate this issue. 1121505 6 260090 commit-queue 2015-08-27 17:36:42 -0700 Comment on attachment 260090 Proposed patch Clearing flags on attachment: 260090 Committed r189078: <http://trac.webkit.org/changeset/189078> 1121506 7 commit-queue 2015-08-27 17:36:46 -0700 All reviewed patches have been landed. Closing bug. 1121523 8 aakash_jain 2015-08-27 18:31:06 -0700 It was a warning message and appeared only when I was running in verbose mode. The message was: /Applications/Xcode.app/Contents/Developer/usr/bin/sim: No simulator devices appear to be running. Setting data directories to /var/empty It seems to be coming from "xcrun sim" command inside listAllTests(), when the Simulator is not running. That's the reason we are starting simulator before calling listAllTests(). On investigating further it seems that if the simulator is not running, (and executing in Verbose mode) every test gives this warning. We should continue debugging it as part of <rdar://problem/22388812>. 260008 2015-08-26 17:34:14 -0700 2015-08-27 14:26:22 -0700 Proposed patch patch-QuitSimulator-APITests text/plain 2855 aakash_jain SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDE4OTAwNSkKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE0IEBACisyMDE1LTA4LTI2ICBBYWthc2ggSmFpbiAgPGFha2FzaF9qYWluQGFwcGxlLmNv bT4KKworICAgICAgICBpT1MgU2ltdWxhdG9yIEFQSSB0ZXN0cyBmYWlscyBhcyBTaW11bGF0b3Ig aXMgbm90IHJ1bm5pbmcKKyAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcvc2hvd19idWcu Y2dpP2lkPTE0ODUwMQorICAgICAgICByZGFyOi8vcHJvYmxlbS8yMjQ0NzUyNQorCisgICAgICAg IFJldmlld2VkIGJ5IE5PQk9EWSAoT09QUyEpLgorCisgICAgICAgICogU2NyaXB0cy9ydW4tYXBp LXRlc3RzOiBMYXVuY2ggdGhlIGlPUyBTaW11bGF0b3IgYmVmb3JlIHN0YXJ0aW5nIHRoZSBpT1Mg U2ltdWxhdG9yIEFQSSB0ZXN0cyBhbmQgcXVpdCBpdCBhZnRlciBmaW5pc2hpbmcgdGhlIHRlc3Rz LgorICAgICAgICAqIFNjcmlwdHMvd2Via2l0ZGlycy5wbTogU3RvcmUgc3RyaW5nICJGb3IgV2Vi S2l0IERldmVsb3BtZW50IiB0byBjb25zdGFudCBTSU1VTEFUT1JfREVWSUNFX1NVRkZJWF9GT1Jf V0VCS0lUX0RFVkVMT1BNRU5UIHRvIG1ha2UgaXQgY29uc2lzdGVudGx5IHJldXNhYmxlLgorCiAy MDE1LTA4LTI2ICBBbGV4IENocmlzdGVuc2VuICA8YWNocmlzdGVuc2VuQHdlYmtpdC5vcmc+CiAK ICAgICAgICAgQnVpbGQgZml4IGFmdGVyIHIxODg5ODIKSW5kZXg6IFRvb2xzL1NjcmlwdHMvcnVu LWFwaS10ZXN0cwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9TY3JpcHRzL3J1bi1hcGktdGVzdHMJKHJl dmlzaW9uIDE4OTAwMCkKKysrIFRvb2xzL1NjcmlwdHMvcnVuLWFwaS10ZXN0cwkod29ya2luZyBj b3B5KQpAQCAtOTcsNiArOTcsOCBAQCBpZiAoISRnZXRPcHRpb25zUmVzdWx0IHx8ICRzaG93SGVs cCkgewogICAgZXhpdCAxOwogfQogCityZWxhdW5jaElPU1NpbXVsYXRvcihmaW5kT3JDcmVhdGVT aW11bGF0b3JGb3JJT1NEZXZpY2UoU0lNVUxBVE9SX0RFVklDRV9TVUZGSVhfRk9SX1dFQktJVF9E RVZFTE9QTUVOVCkpIGlmIHdpbGxVc2VJT1NTaW11bGF0b3JTREtXaGVuQnVpbGRpbmcoKTsKKwog c2V0Q29uZmlndXJhdGlvbigpOwogCiBzZXRDb25maWd1cmF0aW9uUHJvZHVjdERpcihDd2Q6OmFi c19wYXRoKCRyb290KSkgaWYgKGRlZmluZWQoJHJvb3QpKTsKQEAgLTE1Nyw2ICsxNTksOCBAQCBz dWIgcnVuVGVzdHNCeVN1aXRlKFxAKQogICAgICAgICB9CiAgICAgfQogCisgICAgcXVpdElPU1Np bXVsYXRvcigpIGlmIHdpbGxVc2VJT1NTaW11bGF0b3JTREtXaGVuQnVpbGRpbmcoKTsKKwogICAg IHJldHVybiBAdGVzdHNGYWlsZWQgPiAwIHx8IEB0ZXN0c1RpbWVkT3V0ID4gMDsKIH0KIApJbmRl eDogVG9vbHMvU2NyaXB0cy93ZWJraXRkaXJzLnBtCj09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIFRvb2xzL1Njcmlw dHMvd2Via2l0ZGlycy5wbQkocmV2aXNpb24gMTg5MDAwKQorKysgVG9vbHMvU2NyaXB0cy93ZWJr aXRkaXJzLnBtCSh3b3JraW5nIGNvcHkpCkBAIC04MSw2ICs4MSw3IEBAIEJFR0lOIHsKICAgICAg ICAmc2hhcmVkQ29tbWFuZExpbmVPcHRpb25zCiAgICAgICAgJnNoYXJlZENvbW1hbmRMaW5lT3B0 aW9uc1VzYWdlCiAgICAgICAgVVNFX09QRU5fQ09NTUFORAorICAgICAgIFNJTVVMQVRPUl9ERVZJ Q0VfU1VGRklYX0ZPUl9XRUJLSVRfREVWRUxPUE1FTlQKICAgICk7CiAgICAlRVhQT1JUX1RBR1Mg PSAoICk7CiAgICBARVhQT1JUX09LICAgPSAoKTsKQEAgLTkwLDYgKzkxLDcgQEAgdXNlIGNvbnN0 YW50IFVTRV9PUEVOX0NPTU1BTkQgPT4gMTsgIyBVcwogdXNlIGNvbnN0YW50IElOQ0xVREVfT1BU SU9OU19GT1JfREVCVUdHSU5HID0+IDE7CiB1c2UgY29uc3RhbnQgU0lNVUxBVE9SX0RFVklDRV9T VEFURV9TSFVURE9XTiA9PiAiMSI7CiB1c2UgY29uc3RhbnQgU0lNVUxBVE9SX0RFVklDRV9TVEFU RV9CT09URUQgPT4gIjMiOwordXNlIGNvbnN0YW50IFNJTVVMQVRPUl9ERVZJQ0VfU1VGRklYX0ZP Ul9XRUJLSVRfREVWRUxPUE1FTlQgID0+ICJGb3IgV2ViS2l0IERldmVsb3BtZW50IjsKIAogb3Vy IEBFWFBPUlRfT0s7CiAKQEAgLTIyMzcsNyArMjIzOSw3IEBAIHN1YiBydW5JT1NXZWJLaXRBcHBJ blNpbXVsYXRvcigkOyQpCiAgICAgbXkgJHByb2R1Y3REaXIgPSBwcm9kdWN0RGlyKCk7CiAgICAg bXkgJGFwcERpc3BsYXlOYW1lID0gYXBwRGlzcGxheU5hbWVGcm9tQnVuZGxlKCRhcHBCdW5kbGUp OwogICAgIG15ICRhcHBJZGVudGlmaWVyID0gYXBwSWRlbnRpZmllckZyb21CdW5kbGUoJGFwcEJ1 bmRsZSk7Ci0gICAgbXkgJHNpbXVsYXRlZERldmljZSA9IGZpbmRPckNyZWF0ZVNpbXVsYXRvckZv cklPU0RldmljZSgiRm9yIFdlYktpdCBEZXZlbG9wbWVudCIpOworICAgIG15ICRzaW11bGF0ZWRE ZXZpY2UgPSBmaW5kT3JDcmVhdGVTaW11bGF0b3JGb3JJT1NEZXZpY2UoU0lNVUxBVE9SX0RFVklD RV9TVUZGSVhfRk9SX1dFQktJVF9ERVZFTE9QTUVOVCk7CiAgICAgbXkgJHNpbXVsYXRlZERldmlj ZVVESUQgPSAkc2ltdWxhdGVkRGV2aWNlLT57VURJRH07CiAKICAgICBteSAkd2lsbFVzZVN5c3Rl bUluc3RhbGxlZEFwcCA9IGlzSU9TU2ltdWxhdG9yU3lzdGVtSW5zdGFsbGVkQXBwKCRhcHBCdW5k bGUpOwo= 260090 2015-08-27 14:26:22 -0700 2015-08-27 17:36:42 -0700 Proposed patch patch-QuitSimulator-APITests2 text/plain 3293 aakash_jain SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDE4OTA1MCkKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE4IEBACisyMDE1LTA4LTI3ICBBYWthc2ggSmFpbiAgPGFha2FzaF9qYWluQGFwcGxlLmNv bT4KKworICAgICAgICBpT1MgU2ltdWxhdG9yIEFQSSB0ZXN0cyBmYWlscyBhcyBTaW11bGF0b3Ig aXMgbm90IHJ1bm5pbmcKKyAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcvc2hvd19idWcu Y2dpP2lkPTE0ODUwMQorICAgICAgICByZGFyOi8vcHJvYmxlbS8yMjQ0NzUyNQorCisgICAgICAg IFJldmlld2VkIGJ5IE5PQk9EWSAoT09QUyEpLgorCisgICAgICAgIFRoaXMgY2hhbmdlIGlzIGEg d29ya2Fyb3VuZCBmb3IgPHJkYXI6Ly9wcm9ibGVtLzIyMzg4ODEyPi4KKworICAgICAgICAgKiBT Y3JpcHRzL3J1bi1hcGktdGVzdHM6IExhdW5jaCB0aGUgaU9TIFNpbXVsYXRvciBiZWZvcmUgc3Rh cnRpbmcgdGhlIEFQSSBUZXN0cworICAgICAgICAgYW5kIHF1aXQgaXQgb24gcHJvZ3JhbSB0ZXJt aW5hdGlvbiB3aGVuIHJ1bm5pbmcgdGVzdHMgb24gdGhlIGlPUyBwb3J0LgorICAgICAgICAgKiBT Y3JpcHRzL3dlYmtpdGRpcnMucG06IFN0b3JlIHN0cmluZyAiRm9yIFdlYktpdCBEZXZlbG9wbWVu dCIgaW4gY29uc3RhbnQKKyAgICAgICAgIFNJTVVMQVRPUl9ERVZJQ0VfU1VGRklYX0ZPUl9XRUJL SVRfREVWRUxPUE1FTlQgdG8gbWFrZSBpdCBjb25zaXN0ZW50bHkgcmV1c2FibGUuCisKIDIwMTUt MDgtMjcgIE5hbiBXYW5nICA8bl93YW5nQGFwcGxlLmNvbT4KIAogICAgICAgICBBWDogRml4IGFj Y2Vzc2liaWxpdHkvc2VsZWN0LWVsZW1lbnQtYXQtaW5kZXguaHRtbCB0ZXN0CkluZGV4OiBUb29s cy9TY3JpcHRzL3J1bi1hcGktdGVzdHMKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMvU2NyaXB0cy9ydW4t YXBpLXRlc3RzCShyZXZpc2lvbiAxODkwNTApCisrKyBUb29scy9TY3JpcHRzL3J1bi1hcGktdGVz dHMJKHdvcmtpbmcgY29weSkKQEAgLTEwMyw2ICsxMDMsOSBAQCBzZXRDb25maWd1cmF0aW9uUHJv ZHVjdERpcihDd2Q6OmFic19wYXRoCiAKIGJ1aWxkVGVzdFRvb2woKSBpZiAkYnVpbGQgJiYgIWRl ZmluZWQoJHJvb3QpOwogc2V0UGF0aEZvclJ1bm5pbmdXZWJLaXRBcHAoXCVFTlYpOworCisjIEZJ WE1FOiBXZSBsYXVuY2ggKGFuZCBxdWl0KSBpT1MgU2ltdWxhdG9yIGFzIGEgd29ya2Fyb3VuZCBm b3IgPHJkYXI6Ly9wcm9ibGVtLzIyMzg4ODEyPi4KK3JlbGF1bmNoSU9TU2ltdWxhdG9yKGZpbmRP ckNyZWF0ZVNpbXVsYXRvckZvcklPU0RldmljZShTSU1VTEFUT1JfREVWSUNFX1NVRkZJWF9GT1Jf V0VCS0lUX0RFVkVMT1BNRU5UKSkgaWYgd2lsbFVzZUlPU1NpbXVsYXRvclNES1doZW5CdWlsZGlu ZygpOwogbXkgQHRlc3RzVG9SdW4gPSBsaXN0QWxsVGVzdHMoKTsKIAogQHRlc3RzVG9SdW4gPSBn cmVwIHsgbXkgJHRlc3QgPSAkXzsgZ3JlcCB7ICR0ZXN0ID1+IG0vXlxRJF9cRS8gfSBAQVJHVjsg fSBAdGVzdHNUb1J1biBpZiBAQVJHVjsKQEAgLTExMiw2ICsxMTUsOSBAQCBpZiAoJGR1bXBUZXN0 cykgewogICAgIGV4aXQgMDsKIH0KIAorIyBGSVhNRTogV2UgbGF1bmNoIChhbmQgcXVpdCkgaU9T IFNpbXVsYXRvciBhcyBhIHdvcmthcm91bmQgZm9yIDxyZGFyOi8vcHJvYmxlbS8yMjM4ODgxMj4u CitFTkQgeyBxdWl0SU9TU2ltdWxhdG9yKCkgaWYgd2lsbFVzZUlPU1NpbXVsYXRvclNES1doZW5C dWlsZGluZygpOyB9CisKIGV4aXQgcnVuVGVzdHNCeVN1aXRlKEB0ZXN0c1RvUnVuKTsKIAogc3Vi IGlzU3VwcG9ydGVkUGxhdGZvcm0oKQpJbmRleDogVG9vbHMvU2NyaXB0cy93ZWJraXRkaXJzLnBt Cj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT0KLS0tIFRvb2xzL1NjcmlwdHMvd2Via2l0ZGlycy5wbQkocmV2aXNpb24gMTg5 MDUwKQorKysgVG9vbHMvU2NyaXB0cy93ZWJraXRkaXJzLnBtCSh3b3JraW5nIGNvcHkpCkBAIC04 MCw2ICs4MCw3IEBAIEJFR0lOIHsKICAgICAgICAmc2V0dXBNYWNXZWJLaXRFbnZpcm9ubWVudAog ICAgICAgICZzaGFyZWRDb21tYW5kTGluZU9wdGlvbnMKICAgICAgICAmc2hhcmVkQ29tbWFuZExp bmVPcHRpb25zVXNhZ2UKKyAgICAgICBTSU1VTEFUT1JfREVWSUNFX1NVRkZJWF9GT1JfV0VCS0lU X0RFVkVMT1BNRU5UCiAgICAgICAgVVNFX09QRU5fQ09NTUFORAogICAgKTsKICAgICVFWFBPUlRf VEFHUyA9ICggKTsKQEAgLTkwLDYgKzkxLDcgQEAgdXNlIGNvbnN0YW50IFVTRV9PUEVOX0NPTU1B TkQgPT4gMTsgIyBVcwogdXNlIGNvbnN0YW50IElOQ0xVREVfT1BUSU9OU19GT1JfREVCVUdHSU5H ID0+IDE7CiB1c2UgY29uc3RhbnQgU0lNVUxBVE9SX0RFVklDRV9TVEFURV9TSFVURE9XTiA9PiAi MSI7CiB1c2UgY29uc3RhbnQgU0lNVUxBVE9SX0RFVklDRV9TVEFURV9CT09URUQgPT4gIjMiOwor dXNlIGNvbnN0YW50IFNJTVVMQVRPUl9ERVZJQ0VfU1VGRklYX0ZPUl9XRUJLSVRfREVWRUxPUE1F TlQgID0+ICJGb3IgV2ViS2l0IERldmVsb3BtZW50IjsKIAogb3VyIEBFWFBPUlRfT0s7CiAKQEAg LTIyMzcsNyArMjIzOSw3IEBAIHN1YiBydW5JT1NXZWJLaXRBcHBJblNpbXVsYXRvcigkOyQpCiAg ICAgbXkgJHByb2R1Y3REaXIgPSBwcm9kdWN0RGlyKCk7CiAgICAgbXkgJGFwcERpc3BsYXlOYW1l ID0gYXBwRGlzcGxheU5hbWVGcm9tQnVuZGxlKCRhcHBCdW5kbGUpOwogICAgIG15ICRhcHBJZGVu dGlmaWVyID0gYXBwSWRlbnRpZmllckZyb21CdW5kbGUoJGFwcEJ1bmRsZSk7Ci0gICAgbXkgJHNp bXVsYXRlZERldmljZSA9IGZpbmRPckNyZWF0ZVNpbXVsYXRvckZvcklPU0RldmljZSgiRm9yIFdl YktpdCBEZXZlbG9wbWVudCIpOworICAgIG15ICRzaW11bGF0ZWREZXZpY2UgPSBmaW5kT3JDcmVh dGVTaW11bGF0b3JGb3JJT1NEZXZpY2UoU0lNVUxBVE9SX0RFVklDRV9TVUZGSVhfRk9SX1dFQktJ VF9ERVZFTE9QTUVOVCk7CiAgICAgbXkgJHNpbXVsYXRlZERldmljZVVESUQgPSAkc2ltdWxhdGVk RGV2aWNlLT57VURJRH07CiAKICAgICBteSAkd2lsbFVzZVN5c3RlbUluc3RhbGxlZEFwcCA9IGlz SU9TU2ltdWxhdG9yU3lzdGVtSW5zdGFsbGVkQXBwKCRhcHBCdW5kbGUpOwo=