Bug 172030 – REGRESSION (r216711): API test WKWebView.ClearAppCache is failing

• Component: New Bugs
• Status: RESOLVED FIXED

[Ryan Haddad]

API test WKWebView.ClearAppCache is failing

FAIL WKWebView.ClearAppCache

/Volumes/Data/slave/sierra-release/build/Tools/TestWebKitAPI/Tests/WebKit2Cocoa/LocalStorageClear.mm:175
Value of: originalWebsiteDataRecordCount - 1
  Actual: 1
Expected: websiteDataRecords.count
Which is: 0

https://build.webkit.org/builders/Apple%20El%20Capitan%20Release%20WK2%20%28Tests%29/builds/1452

[Ryan Haddad]

It looks like this may have started with https://trac.webkit.org/changeset/216711/webkit

[Brady Eidson]

(In reply to Ryan Haddad from comment #1)
> It looks like this may have started with
> https://trac.webkit.org/changeset/216711/webkit

If 216711 caused this, then I think this test was utterly broken in the first place.

[Brady Eidson]

(In reply to Ryan Haddad from comment #1)
> It looks like this may have started with
> https://trac.webkit.org/changeset/216711/webkit

You say *MAY* have started with this...  how sure are you?

[Ryan Haddad]

(In reply to Brady Eidson from comment #3)
> (In reply to Ryan Haddad from comment #1)
> > It looks like this may have started with
> > https://trac.webkit.org/changeset/216711/webkit
> 
> You say *MAY* have started with this...  how sure are you?

I said may because I did not reproduce it locally, but regression range is between https://trac.webkit.org/changeset/216709 and https://trac.webkit.org/changeset/216712. None of the other commits look related.

[Brady Eidson]

Cannot reproduce this locally no matter what I try.

Looking at changes to the test to make it more resilient.

[Brady Eidson]

Created attachment 309927 [details]
Patch for suppression/exploration

[WebKit Commit Bot]

Comment on attachment 309927 [details]
Patch for suppression/exploration

Rejecting attachment 309927 [details] from commit-queue.

Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-02', 'validate-changelog', '--check-oops', '--non-interactive', 309927, '--port=mac']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

Nobody - Test gardening, in effect found in /Volumes/Data/EWS/WebKit/Tools/ChangeLog does not appear to be a valid reviewer according to contributors.json.
/Volumes/Data/EWS/WebKit/Tools/ChangeLog neither lists a valid reviewer nor contains the string "Unreviewed" or "Rubber stamp" (case insensitive).

Full output: http://webkit-queues.webkit.org/results/3727676

[Brady Eidson]

Created attachment 309928 [details]
Patch for suppression/exploration

[WebKit Commit Bot]

Comment on attachment 309928 [details]
Patch for suppression/exploration

Clearing flags on attachment: 309928

Committed r216777: <http://trac.webkit.org/changeset/216777>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Brady Eidson]

New failure on bots:

/Volumes/Data/slave/elcapitan-release/build/Tools/TestWebKitAPI/Tests/WebKit2Cocoa/LocalStorageClear.mm:170
Value of: 1ul
  Actual: 1
Expected: websiteDataRecords.count
Which is: 2


/Volumes/Data/slave/elcapitan-release/build/Tools/TestWebKitAPI/Tests/WebKit2Cocoa/LocalStorageClear.mm:186
Value of: originalWebsiteDataRecordCount - 1
  Actual: 1
Expected: websiteDataRecords.count
Which is: 0

[Brady Eidson]

(In reply to Brady Eidson from comment #11)
> New failure on bots:
> 
> /Volumes/Data/slave/elcapitan-release/build/Tools/TestWebKitAPI/Tests/
> WebKit2Cocoa/LocalStorageClear.mm:170
> Value of: 1ul
>   Actual: 1
> Expected: websiteDataRecords.count
> Which is: 2

Since we wiped the entire data store directory *and* installed only a single record of app cache ness, we expect exactly *one* record here.

Getting 2 is entirely unexpected.

> /Volumes/Data/slave/elcapitan-release/build/Tools/TestWebKitAPI/Tests/
> WebKit2Cocoa/LocalStorageClear.mm:186
> Value of: originalWebsiteDataRecordCount - 1
>   Actual: 1
> Expected: websiteDataRecords.count
> Which is: 0

This is actually right - At the end of the test, there should absolutely be 0 results left for getting all data of all types.

[Brady Eidson]

Created attachment 309945 [details]
Get more info from the bots

[WebKit Commit Bot]

The commit-queue encountered the following flaky tests while processing attachment 309945 [details]:

The commit-queue is continuing to process your patch.

[WebKit Commit Bot]

The commit-queue encountered the following flaky tests while processing attachment 309945 [details]:

http/tests/misc/acid3.html bug 165734 (authors: [e-mail], [e-mail], [e-mail], [e-mail], and [e-mail])
The commit-queue is continuing to process your patch.

[WebKit Commit Bot]

Comment on attachment 309945 [details]
Get more info from the bots

Clearing flags on attachment: 309945

Committed r216800: <http://trac.webkit.org/changeset/216800>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Brady Eidson]

Reopening since the landed patch will not fix this, but rather hopefully just tell us what's broken.

[Brady Eidson]

Finally got the unexpected results:

/Volumes/Data/slave/sierra-release/build/Tools/TestWebKitAPI/Tests/WebKit2Cocoa/LocalStorageClear.mm:172
Value of: [record.displayName UTF8String]
  Actual: "Local documents on your computer"
Expected: "127.0.0.1"


/Volumes/Data/slave/sierra-release/build/Tools/TestWebKitAPI/Tests/WebKit2Cocoa/LocalStorageClear.mm:174
Value of: [type UTF8String]
  Actual: "WKWebsiteDataTypeIndexedDBDatabases"
Expected: [WKWebsiteDataTypeOfflineWebApplicationCache UTF8String]
Which is: "WKWebsiteDataTypeOfflineWebApplicationCache"

So, bizarrely, there are IndexedDB remnants around.

No clue why, and why nobody can reproduce locally instead of on the bots.

[Brady Eidson]

Also quite bizarre that deleting the IDB directory at the start doesn't resolve this.

[Brady Eidson]

I see, app cache doesn't go into that directory.

Okay, fine.

Let's resolve this once and for all.

[Brady Eidson]

Created attachment 309969 [details]
Patch to fix

[Build Bot]

Attachment 309969 [details] did not pass style-queue:


ERROR: Tools/TestWebKitAPI/Tests/WebKit2Cocoa/LocalStorageClear.mm:139:  This { should be at the end of the previous line  [whitespace/braces] [4]
Total errors found: 1 in 2 files


If any of these errors are false positives, please file a bug against check-webkit-style.

[WebKit Commit Bot]

Comment on attachment 309969 [details]
Patch to fix

Clearing flags on attachment: 309969

Committed r216805: <http://trac.webkit.org/changeset/216805>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.