Bug 149549 – Add iOS builders and testers

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Daniel Bates]

Add definitions for iOS builders and testers.

[Daniel Bates]

Created attachment 261913 [details]
Patch

[Daniel Bates]

See <rdar://problem/22291016> and <rdar://problem/22291018> for more details.

[Alexey Proskuryakov]

Comment on attachment 261913 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=261913&action=review

> Tools/BuildSlaveSupport/build.webkit.org-config/config.json:40
> +                    { "name": "bot407", "platform": "ios-9" },

I'm curious what "platform" is used for. Is it supposed to be host OS, target OS, or is it completely unused?

> Tools/BuildSlaveSupport/build.webkit.org-config/config.json:191
> +                      "slavenames": ["bot215"]

My understanding was that we planned to have two Mac minis per test queue, same as on OS X. Does testing 32-bit simulator provide any benefit at all?

[Alexey Proskuryakov]

Comment on attachment 261913 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=261913&action=review

> Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:399
> +    'Apple iOS Simulator Release WK1 (Tests)' : ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'layout-test', 'run-api-tests', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'archive-test-results', 'upload', 'MasterShellCommand'],

Do any of the test steps beyond layout-test actually work?

[Daniel Bates]

(In reply to comment #3)
> Comment on attachment 261913 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=261913&action=review
> 
> > Tools/BuildSlaveSupport/build.webkit.org-config/config.json:40
> > +                    { "name": "bot407", "platform": "ios-9" },
> 
> I'm curious what "platform" is used for. Is it supposed to be host OS,
> target OS, or is it completely unused?

It is an identifier whose purpose is to identify a group of builders/testers by a particular WebKit port so that we can append the appropriate command line flag (e.g. --gtk) to script invocations among other tasks. A port may be specific to a host OS/target OS (e.g. OS X) or a toolkit (e.g. GTK).

> 
> > Tools/BuildSlaveSupport/build.webkit.org-config/config.json:191
> > +                      "slavenames": ["bot215"]
> 
> My understanding was that we planned to have two Mac minis per test queue,
> same as on OS X. Does testing 32-bit simulator provide any benefit at all?

I'm unclear of the benefit of testing 32-bit simulator builds. Maybe David Kilzer can elaborate on his remark in <rdar://problem/22291018> with regards to the benefit of testing 32-bit simulator builds.

[Daniel Bates]

(In reply to comment #4)
> Comment on attachment 261913 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=261913&action=review
> 
> > Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:399
> > +    'Apple iOS Simulator Release WK1 (Tests)' : ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'layout-test', 'run-api-tests', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'archive-test-results', 'upload', 'MasterShellCommand'],
> 
> Do any of the test steps beyond layout-test actually work?

Yes, they all work.

[David Kilzer (:ddkilzer)]

(In reply to comment #5)
> (In reply to comment #3)
> > Comment on attachment 261913 [details]
> > Patch
> > 
> > View in context:
> > https://bugs.webkit.org/attachment.cgi?id=261913&action=review
> > 
> > > Tools/BuildSlaveSupport/build.webkit.org-config/config.json:191
> > > +                      "slavenames": ["bot215"]
> > 
> > My understanding was that we planned to have two Mac minis per test queue,
> > same as on OS X. Does testing 32-bit simulator provide any benefit at all?
> 
> I'm unclear of the benefit of testing 32-bit simulator builds. Maybe David
> Kilzer can elaborate on his remark in <rdar://problem/22291018> with regards
> to the benefit of testing 32-bit simulator builds.

I listed both architectures for completeness.  If we have 32-bit hardware builds, we probably don't need 32-bit simulator builds.  It's fine to omit them until we find that we're missing test coverage (e.g., we would have found a bug sooner).

[David Kilzer (:ddkilzer)]

Comment on attachment 261913 [details]
Patch

r=me

[Daniel Bates]

Comment on attachment 261913 [details]
Patch

I am on vacation today and will be back Monday (09/28). I do not have access to a computer to update the patch. If we want to update the patch to remove the 32-bit iOS Simulator testers then can someone take over this bug? Otherwise, if the patch is sufficient to land as-is then mark it cq+.

[Daniel Bates]

Created attachment 262184 [details]
Patch

After talking with Alexey Proskuryakov in-person this morning (09/30), we feel it would be beneficial to repurpose the 32-bit simulator builders and testers as 64-bit simulator builders and testers because the 64-bit variant represents a typical build as opposed to the 32-bit variant. This allocation will also allow us to service two build/test requests per queue concurently, decreasing the time from commit to the time when results for the build/test are available, and helping developers catch failures sooner. We can always follow up and adjust the build bot allocation, including adding 32-bit simulator builders and/or testers in a subsequent patch.

[Daniel Bates]

Comment on attachment 262184 [details]
Patch

Clearing flags on attachment: 262184

Committed r190359: <http://trac.webkit.org/changeset/190359>

[Daniel Bates]

All reviewed patches have been landed.  Closing bug.