Bug 179426 - webkitpy: Unify version parsing code
Summary: webkitpy: Unify version parsing code
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Jonathan Bedard
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2017-11-08 08:54 PST by Jonathan Bedard
Modified: 2017-11-28 00:32 PST (History)
9 users (show)

See Also:


Attachments
Patch (30.17 KB, patch)
2017-11-08 09:32 PST, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Patch (31.44 KB, patch)
2017-11-08 11:45 PST, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Patch (31.43 KB, patch)
2017-11-08 14:27 PST, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Patch (31.93 KB, patch)
2017-11-08 17:21 PST, Jonathan Bedard
no flags Details | Formatted Diff | Diff
Patch (33.79 KB, patch)
2017-11-09 15:40 PST, Jonathan Bedard
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Bedard 2017-11-08 08:54:00 PST
In webkitpy, we parse version strings (of the format x.x.x) in a number of different places.  We should unify these checks into a single Version class which we can easily compare different versions.
Comment 1 Radar WebKit Bug Importer 2017-11-08 08:57:33 PST
<rdar://problem/35415191>
Comment 2 Jonathan Bedard 2017-11-08 09:32:22 PST
Created attachment 326334 [details]
Patch
Comment 3 Jonathan Bedard 2017-11-08 09:55:35 PST
Comment on attachment 326334 [details]
Patch

Looks like it broke some iOS Sim stuff.
Comment 4 Jonathan Bedard 2017-11-08 11:45:51 PST
Created attachment 326341 [details]
Patch
Comment 5 Jonathan Bedard 2017-11-08 14:27:18 PST
Created attachment 326377 [details]
Patch
Comment 6 Jonathan Bedard 2017-11-08 17:21:41 PST
Created attachment 326411 [details]
Patch
Comment 7 David Kilzer (:ddkilzer) 2017-11-09 13:42:25 PST
Comment on attachment 326411 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=326411&action=review

r=me

You don't have to make any of the suggested changes before landing, but at least consider:
- Tightening up __len__ and parsing of versions with more than 3 integers (maybe throw an exception due to "loss" of data)?
- Add tests for __len__ (whether or not the above is done) and __setitem__ to make sure they work.
- Consider using 'tiny' instead of 'build'.  (The word 'build' is very over-loaded, and 'build version' means something different than the 3rd integer in a dotted version string at Apple.)

> Tools/ChangeLog:20
> +        (PlatformInfo.__init__): Convert mac version string to version object and
> +        use _win_version instead of _win_version_tuple.
> +        (PlatformInfo.xcode_sdk_version): Convert SDK version string to Version object
> +        before returning it.
> +        (PlatformInfo.xcode_version): Return version object instead of version string.
> +        (PlatformInfo._determine_mac_version): Accept version object instead of string,
> +        eliminate parsing.
> +        (PlatformInfo._determine_win_version): Accept version object instead of tuple.

Nit:  Not using capital "Version object" consistently.  Doesn't have to be fixed for landing.

> Tools/Scripts/webkitpy/common/version.py:29
> +        self.major = 0
> +        self.minor = 0
> +        self.build = 0

In Source/JavaScriptCore/Configurations/Version.xcconfig,the following names are used for each position:

MAJOR_VERSION = 605;
MINOR_VERSION = 1;
TINY_VERSION = 13;
MICRO_VERSION = 0;
NANO_VERSION = 0;

Not sure this class needs to support 5 dotted versions, but you might consider renaming 'build' to 'tiny' for consistency.

(I actually don't know what the history of those names is, though.)

Not required to land the patch.

> Tools/Scripts/webkitpy/common/version.py:43
> +        raise ValueError('Version expected to be int, str, tuple or list of ints')

Nit: Spell out "ints" as "integers".

> Tools/Scripts/webkitpy/common/version.py:46
> +    def __len__(self):
> +        return 3

This doesn't return a correct value if one of the for loops in the __init__ method is used as it will set keys for '3' and '4' (or more).

May not matter currently, but it should do the right thing if we support passing in "1.2.3.4" or "1.2.3.4.5".

Hmm...then again, we raise an exception in __getitem__ and __setitem__ if the key is larger than '2'.  Not sure what you want to do here.

> Tools/Scripts/webkitpy/common/version.py:98
> +    def __str__(self):
> +        result = '{}'.format(self.major)
> +        if self.minor or self.build:
> +            result += '.{}'.format(self.minor)
> +        if self.build:
> +            result += '.{}'.format(self.build)
> +        return result

If we support more than 3 dotted numbers (see above comments), may want to make this more generic?

> Tools/Scripts/webkitpy/common/version_unittest.py:28
> +class VersionTestCase(unittest.TestCase):

One of these tests should verify __len__.

You should also have a test for using __setitem__ with both numbers (0, 1, 2) and strings (major, minor, build/tiny).

> Tools/Scripts/webkitpy/common/system/platforminfo.py:61
>          if self.os_name.startswith('mac'):
> -            self.os_version = self._determine_mac_version(platform_module.mac_ver()[0])
> +            self.os_version = self._determine_mac_version(Version(platform_module.mac_ver()[0]))
>          if self.os_name.startswith('win'):
> -            self.os_version = self._determine_win_version(self._win_version_tuple(sys_module))
> +            self.os_version = self._determine_win_version(self._win_version(sys_module))

Kind of weird how platform_module.mac_ver() doesn't return a Version object here, but this doesn't block the patch from landing.

Maybe too much to change for this patch (which is already pretty large)?

> Tools/Scripts/webkitpy/port/ios_simulator.py:191
>      def use_multiple_simulator_apps(self):
> -        return int(self.host.platform.xcode_version().split('.')[0]) < 9
> +        return int(self.host.platform.xcode_version()[0]) < 9

After reading the patch up to this point, I find that using major/minor/build is probably more readable that 0/1/2.

Not necessary to fix before landing, though.
Comment 8 Jonathan Bedard 2017-11-09 14:54:08 PST
Comment on attachment 326411 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=326411&action=review

>> Tools/Scripts/webkitpy/common/version.py:29
>> +        self.build = 0
> 
> In Source/JavaScriptCore/Configurations/Version.xcconfig,the following names are used for each position:
> 
> MAJOR_VERSION = 605;
> MINOR_VERSION = 1;
> TINY_VERSION = 13;
> MICRO_VERSION = 0;
> NANO_VERSION = 0;
> 
> Not sure this class needs to support 5 dotted versions, but you might consider renaming 'build' to 'tiny' for consistency.
> 
> (I actually don't know what the history of those names is, though.)
> 
> Not required to land the patch.

I like those names better, I will use them.

>> Tools/Scripts/webkitpy/common/version.py:46
>> +        return 3
> 
> This doesn't return a correct value if one of the for loops in the __init__ method is used as it will set keys for '3' and '4' (or more).
> 
> May not matter currently, but it should do the right thing if we support passing in "1.2.3.4" or "1.2.3.4.5".
> 
> Hmm...then again, we raise an exception in __getitem__ and __setitem__ if the key is larger than '2'.  Not sure what you want to do here.

I haven't seen us passing any 4 or 5 indexed versions around.  The current code will throw an exception if we tried to construct with that size (because of the exception in __getitem__ and __setitem__)

Since we have JavaScriptCore code which reference micro and nano versions as well, I will allow for those too.

>> Tools/Scripts/webkitpy/common/system/platforminfo.py:61
>> +            self.os_version = self._determine_win_version(self._win_version(sys_module))
> 
> Kind of weird how platform_module.mac_ver() doesn't return a Version object here, but this doesn't block the patch from landing.
> 
> Maybe too much to change for this patch (which is already pretty large)?

This will be addressed in a future patch.

Ultimately, we need an os_version -> os_name mapping to solve this.
Comment 9 Jonathan Bedard 2017-11-09 15:40:18 PST
Created attachment 326498 [details]
Patch
Comment 10 WebKit Commit Bot 2017-11-09 16:58:36 PST
Comment on attachment 326498 [details]
Patch

Clearing flags on attachment: 326498

Committed r224657: <https://trac.webkit.org/changeset/224657>
Comment 11 WebKit Commit Bot 2017-11-09 16:58:38 PST
All reviewed patches have been landed.  Closing bug.
Comment 12 Jonathan Bedard 2017-11-09 17:48:42 PST
Committed r224658: <https://trac.webkit.org/changeset/224658>
Comment 13 Jonathan Bedard 2017-11-09 17:49:32 PST
Windows seems to return a namedtuple instead of a standard tuple.
Comment 14 Fujii Hironori 2017-11-09 19:32:45 PST
5th element of getwindowsversion() is ''. This causes another error.
Should be truncated like sys.getwindowsversion()[0:3].

> ActivePython 2.7.10.12 (ActiveState Software Inc.) based on
> Python 2.7.10 (default, Aug 21 2015, 12:07:58) [MSC v.1500 64 bit (AMD64)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import sys
> >>> sys.getwindowsversion()
> sys.getwindowsversion(major=6, minor=2, build=9200, platform=2, service_pack='')
> >>> sys.getwindowsversion()[0:3]
> (6, 2, 9200)
> >>>
Comment 15 Fujii Hironori 2017-11-09 20:20:01 PST
Filed Bug 179520 for ActivePython issue.
Comment 16 Fujii Hironori 2017-11-09 20:37:37 PST
r224657 breaks Cygwin Python, too.

> Traceback (most recent call last):
>   File "./Tools/Scripts/webkit-patch", line 84, in <module>
>     main()
>   File "./Tools/Scripts/webkit-patch", line 79, in main
>     WebKitPatch(os.path.abspath(__file__)).main()
>   File "/cygdrive/c/webkit/ga/Tools/Scripts/webkitpy/tool/main.py", line 57, in __init__
>     Host.__init__(self)
>   File "/cygdrive/c/webkit/ga/Tools/Scripts/webkitpy/common/host.py", line 48, in __init__
>     SystemHost.__init__(self)
>   File "/cygdrive/c/webkit/ga/Tools/Scripts/webkitpy/common/system/systemhost.py", line 40, in __init__
>     self.user = user.User()
>   File "/cygdrive/c/webkit/ga/Tools/Scripts/webkitpy/common/system/user.py", line 61, in __init__
>     self._platforminfo = platforminfo or PlatformInfo(sys, platform, Executive())
>   File "/cygdrive/c/webkit/ga/Tools/Scripts/webkitpy/common/system/platforminfo.py", line 61, in __init__
>     self.os_version = self._determine_win_version(self._win_version(sys_module))
>   File "/cygdrive/c/webkit/ga/Tools/Scripts/webkitpy/common/system/platforminfo.py", line 197, in _win_version
>     return Version(self._executive.run_command(['cmd', '/c', 'ver'], decode_output=False))
>   File "/cygdrive/c/webkit/ga/Tools/Scripts/webkitpy/common/version.py", line 38, in __init__
>     self[i] = ver.split('.')[i]
>   File "/cygdrive/c/webkit/ga/Tools/Scripts/webkitpy/common/version.py", line 73, in __setitem__
>     self.major = int(value)
> ValueError: invalid literal for int() with base 10: 'Microsoft Windows [Version 10'

This is the output of ver command.

> C:\>ver
> 
> Microsoft Windows [Version 10.0.14393]
Comment 17 Fujii Hironori 2017-11-09 22:28:34 PST
One more issue.

Bug 179522 – check-webkit-style: AttributeError: 'NoneType' object has no attribute 'major'
Comment 18 Daniel Bates 2017-11-13 22:21:55 PST
The overloaded Version constructors with suboptimal parsing of version numbers given as a string and inconsistent usage of Version constructors throughout this patch just make me sad.
Comment 19 David Kilzer (:ddkilzer) 2017-11-13 23:50:50 PST
(In reply to Daniel Bates from comment #18)
> The overloaded Version constructors with suboptimal parsing of version
> numbers given as a string and inconsistent usage of Version constructors
> throughout this patch just make me sad.

Can you be more specific about what exactly you'd change?

For example:
- What is suboptimal about the parsing of the version numbers given as a string?
- How would you fix the overloading of the constructors?
- How could Version constructor usage be made more consistent (or does this go hand-in-hand with overloading of the constructor)?
Comment 20 Jonathan Bedard 2017-11-14 11:09:50 PST
I talked with Dan today (11/14).

His complaint is that the Version class is too permissive and that we should pick which method of Version parsing we would like to standardize.

I think we should standardize the string method.  With that being said, I still think that there should be sane defaults when passing None, String and Version.  Tracking this work here: <https://bugs.webkit.org/show_bug.cgi?id=179677>.
Comment 21 Fujii Hironori 2017-11-28 00:32:47 PST
Filed for Cygwin.
  Bug 180069 – webkitpy: PlatformInfo raises AssertionError "assert self.os_version is not None" in Cygwin since Bug 179426