Bug 206932 – run-safari / run-webkit-tests --ios-simulator not working with XCode 13.3.1 (11C504)

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Frédéric Wang (:fredw)]

run-safari hangs without launching simulator, I'm not sure why.

For run-webkit-tests, traceback is shown below. In get_runtime_for_device_type() the device_type is software_variant="iOS" and software_version="13.2" but SimulatedDeviceManager.AVAILABLE_RUNTIMES only has "13.3" versions. A ugly workaround is to set device_type.software_version = None, which allows to run the tests for me.

Not sure that really helps, but I remember that in previous XCode version run-safari was actually opening two versions of the simulator which seemed a bug.

AssertionError raised: 
Traceback (most recent call last):
  File "/Users/fred/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 91, in main
    run_details = run(port, options, args, stderr)
  File "/Users/fred/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 485, in run
    run_details = manager.run(args)
  File "/Users/fred/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 270, in run
    if not self._set_up_run(tests_to_run_by_device[device_type], device_type=device_type):
  File "/Users/fred/WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 176, in _set_up_run
    self._port.setup_test_run(device_type)
  File "/Users/fred/WebKit/Tools/Scripts/webkitpy/port/device_port.py", line 179, in setup_test_run
    reboot=self.get_option('reboot', False),
  File "/Users/fred/WebKit/Tools/Scripts/webkitpy/xcode/simulated_device.py", line 404, in initialize_devices
    device = SimulatedDeviceManager._create_or_find_device_for_request(request, host, name_base)
  File "/Users/fred/WebKit/Tools/Scripts/webkitpy/xcode/simulated_device.py", line 264, in _create_or_find_device_for_request
    device_type = SimulatedDeviceManager._disambiguate_device_type(request.device_type)
  File "/Users/fred/WebKit/Tools/Scripts/webkitpy/xcode/simulated_device.py", line 226, in _disambiguate_device_type
    assert runtime is not None
AssertionError

[Jonathan Bedard]

What's the output of 'xcrun simctl list'?

[Frédéric Wang (:fredw)]

Created attachment 389142 [details]
Output of xcrun simctl list

Here is what I get.

Maybe it's also worth noting that I don't see "iPhone SE for WebKit development" (or whathever how it is called) anymore when using the simulator UI.

[Jonathan Bedard]

(In reply to Frédéric Wang (:fredw) from comment #2)
> Created attachment 389142 [details]
> Output of xcrun simctl list
> 
> Here is what I get.
> 
> Maybe it's also worth noting that I don't see "iPhone SE for WebKit
> development" (or whathever how it is called) anymore when using the
> simulator UI.

We have a bot exhibiting this too. Try running 'xcrun simctl delete unavailable', you have a bunch of simulators for old SDK versions. I'm going to try the same thing on our bot.

[Jonathan Bedard]

(In reply to Jonathan Bedard from comment #3)
> (In reply to Frédéric Wang (:fredw) from comment #2)
> > Created attachment 389142 [details]
> > Output of xcrun simctl list
> > 
> > Here is what I get.
> > 
> > Maybe it's also worth noting that I don't see "iPhone SE for WebKit
> > development" (or whathever how it is called) anymore when using the
> > simulator UI.
> 
> We have a bot exhibiting this too. Try running 'xcrun simctl delete
> unavailable', you have a bunch of simulators for old SDK versions. I'm going
> to try the same thing on our bot.

Not going to be that simple, this will need a code change, hopefully something minor.

[Jonathan Bedard]

This is probably an issue with Xcode, since what causes this is having the iOS 13.2 SDK installed but the iOS 13.3 simulator runtime. For a work-around, Xcode -> Preferences -> Components, install iOS 13.2 simulator runtime. Things should work.

[Jonathan Bedard]

Created attachment 389193 [details]
Patch

[Jonathan Bedard]

(In reply to Jonathan Bedard from comment #6)
> Created attachment 389193 [details]
> Patch

Had a discussion with Alexey where he pointed out that runtime versions and SDK versions don't need to remain in lockstep. This is actually just a more generalized case of what we were already doing with allowing a request for just a major version to match the minor versions too.

[Alexey Proskuryakov]

Comment on attachment 389193 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=389193&action=review

> Tools/Scripts/webkitpy/xcode/simulated_device.py:206
> +        # Return the available runtime closest to specified runtime

I don't think that "closest" explains it well. We should either call it "best match", or actually explain what the code is trying to achieve. Also, please add a period at the end of the sentence.

[Jonathan Bedard]

Created attachment 389195 [details]
Patch

[Jonathan Bedard]

Committed r255404: <https://trac.webkit.org/changeset/255404>

[Radar WebKit Bug Importer]

<rdar://problem/59013211>