Bug 202249 – TestWebKitAPI.SOAuthorizationSubFrame.AuthorizationOptions is failing consistently on iOS API tests EWS

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Aakash Jain]

TestWebKitAPI.SOAuthorizationSubFrame.AuthorizationOptions is failing consistently on EWS. e.g.: https://ews-build.webkit.org/#/builders/9/builds/9043, https://ews-build.webkit.org/#/builders/9/builds/9000

It seems to have started failing since we upgraded Xcode from 10.2 to 11.0 few days back. Upgrading further to 11.1 didn't fix it.

First instance of this failure (with Xcode 11.0): https://ews-build.webkit.org/#/builders/9/builds/8733
Previously with XCode 10.2 this failure wasn't present, e.g.: https://ews-build.webkit.org/#/builders/9/builds/8721

[Aakash Jain]

This is making iOS API Tests EWS very slow (as EWS has to re-run the tests to rule out flakiness and then re-run on clean tree to verify if the failure is pre-existing).

[Jiewen Tan]

(In reply to Aakash Jain from comment #1)
> This is making iOS API Tests EWS very slow (as EWS has to re-run the tests
> to rule out flakiness and then re-run on clean tree to verify if the failure
> is pre-existing).

I couldn't locate the iOS version via the links. Anyway to confirm that?

[Ryan Haddad]

(In reply to Jiewen Tan from comment #2)
> (In reply to Aakash Jain from comment #1)
> > This is making iOS API Tests EWS very slow (as EWS has to re-run the tests
> > to rule out flakiness and then re-run on clean tree to verify if the failure
> > is pre-existing).
> 
> I couldn't locate the iOS version via the links. Anyway to confirm that?
We are seeing it with bot 13.0 and 13.1.

[Ryan Haddad]

with both* 13.0 and 13.1.

[Aakash Jain]

(In reply to Jiewen Tan from comment #2)
> I couldn't locate the iOS version via the links. Anyway to confirm that?
On the buildbot page, expand the 'configuration' step and click on 'view all * lines'. It would show the output of 'xcodebuild -sdk -version' command. 

Direct links:
iOS 13.1: https://ews-build.webkit.org/#/builders/9/builds/9043/steps/2/logs/stdio
iOS 13.0: https://ews-build.webkit.org/#/builders/9/builds/8733/steps/2/logs/stdio
iOS 12.2: https://ews-build.webkit.org/#/builders/9/builds/8721/steps/2/logs/stdio

[Jiewen Tan]

(In reply to Aakash Jain from comment #5)
> (In reply to Jiewen Tan from comment #2)
> > I couldn't locate the iOS version via the links. Anyway to confirm that?
> On the buildbot page, expand the 'configuration' step and click on 'view all
> * lines'. It would show the output of 'xcodebuild -sdk -version' command. 
> 
> Direct links:
> iOS 13.1:
> https://ews-build.webkit.org/#/builders/9/builds/9043/steps/2/logs/stdio
> iOS 13.0:
> https://ews-build.webkit.org/#/builders/9/builds/8733/steps/2/logs/stdio
> iOS 12.2:
> https://ews-build.webkit.org/#/builders/9/builds/8721/steps/2/logs/stdio

Which build bot can I monitor it? I didn't see any failures on either OpenSource or Internal. Is this EWS only? EWS use different configurations than build.webkit.org?

[Jiewen Tan]

Created attachment 379618 [details]
Patch

[Jiewen Tan]

Comment on attachment 379618 [details]
Patch

Thanks Alex for r+ this patch.

[WebKit Commit Bot]

Comment on attachment 379618 [details]
Patch

Clearing flags on attachment: 379618

Committed r250382: <https://trac.webkit.org/changeset/250382>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/55737456>

[Aakash Jain]

Thanks for the fix. EWS is not failing this test anymore (https://ews-build.webkit.org/#/builders/9).

EWS is failing another test WebKit.ReloadWithDifferingInitialScale, but that's being fixed in https://bugs.webkit.org/show_bug.cgi?id=202273

[Alexey Proskuryakov]

> Which build bot can I monitor it? I didn't see any failures on either
> OpenSource or Internal. Is this EWS only? EWS use different configurations
> than build.webkit.org?

Did we ever find out why this appeared to only affect EWS?