Bug 195472 – Running a single test will always use the default device

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Dean Jackson]

e.g.

Put a test in
fast/events/ios/ipad/foo.html

In platform/ios/TestExpectations
fast/events/ios/ipad [ Skip ]

In platform/ipad/TestExpectations
fast/events/ios/ipad [ Pass ]

> run-webkit-tests --ios-simulator fast/events/ios/ipad

launches an iPhone SE, not an iPad

[Jonathan Bedard]

Edited the bug title since talking with Dean (and Shawn a few days ago). Running a single test should override skipped expectations (this has been the case for a long time), but with the addition of device specific expectations and the fact that the ios-simulator port supports 3 devices, we expect fast/events/ios/ipad to be run on iPad when using the ios-simulator port, not iPhone SE.

[Radar WebKit Bug Importer]

<rdar://problem/48724825>

[Jonathan Bedard]

Created attachment 368322 [details]
Patch

[Lucas Forschler]

Comment on attachment 368322 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=368322&action=review

> Tools/ChangeLog:6
> +

the title of this doesn't seem to match the description of what the change is about... I see nothing about devices in the code below...

[Jonathan Bedard]

Comment on attachment 368322 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=368322&action=review

>> Tools/ChangeLog:6
>> +
> 
> the title of this doesn't seem to match the description of what the change is about... I see nothing about devices in the code below...

It does, although webkitpy has abstracted enough code that it is not at all obvious why. There are two changes that are device specific. The first is removing this code:

...
elif self._options.skipped != 'always':
    tests_to_skip -= set(paths)
...

Each device has it's own layout test finder (because each device is a separate configuration). Before this change, we would run tests which should be skipped on the current configuration but were explicitly requested by the user. Instead of doing this in the LayoutTestFinder (which, it must be added, only has this behavior when actually running layout tests, not when, for example, rebaselining tests), I moved it to the manager, which is the second part of this change:

...
tests_to_run_by_device[device_type_list[0]].append(arg)
...

This part of the change DOES contain a device type (or rather, a list of device types) but this is actually the new implementation of the above removed code. Instead of adding explicitly requests tests to all configurations, the code will now only add a test which was SKIPPED in ALL configurations to default configuration if that test was specified by the user.  It's a bit confusing, but the usability conclusion is that if I did:

run-webkit-tests --ios-simulator platform/ipad/fast/viewport/width-is-device-width.html

The test would be run on iPad, but if I did:

run-webkit-tests --iphone-simulator platform/ipad/fast/viewport/width-is-device-width.html

The test would be run on iPhone.

[WebKit Commit Bot]

Comment on attachment 368322 [details]
Patch

Clearing flags on attachment: 368322

Committed r244701: <https://trac.webkit.org/changeset/244701>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.