Bug 172030 - REGRESSION (r216711): API test WKWebView.ClearAppCache is failing
Summary: REGRESSION (r216711): API test WKWebView.ClearAppCache is failing
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Brady Eidson
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-05-12 09:25 PDT by Ryan Haddad
Modified: 2017-05-15 09:03 PDT (History)
4 users (show)

See Also:


Attachments
Patch for suppression/exploration (4.75 KB, patch)
2017-05-12 12:19 PDT, Brady Eidson
commit-queue: commit-queue-
Details | Formatted Diff | Diff
Patch for suppression/exploration (4.75 KB, patch)
2017-05-12 12:41 PDT, Brady Eidson
no flags Details | Formatted Diff | Diff
Get more info from the bots (1.80 KB, patch)
2017-05-12 14:20 PDT, Brady Eidson
no flags Details | Formatted Diff | Diff
Patch to fix (3.85 KB, patch)
2017-05-12 16:51 PDT, Brady Eidson
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ryan Haddad 2017-05-12 09:25:51 PDT
API test WKWebView.ClearAppCache is failing

FAIL WKWebView.ClearAppCache

/Volumes/Data/slave/sierra-release/build/Tools/TestWebKitAPI/Tests/WebKit2Cocoa/LocalStorageClear.mm:175
Value of: originalWebsiteDataRecordCount - 1
  Actual: 1
Expected: websiteDataRecords.count
Which is: 0

https://build.webkit.org/builders/Apple%20El%20Capitan%20Release%20WK2%20%28Tests%29/builds/1452
Comment 1 Ryan Haddad 2017-05-12 09:27:17 PDT
It looks like this may have started with https://trac.webkit.org/changeset/216711/webkit
Comment 2 Brady Eidson 2017-05-12 10:56:03 PDT
(In reply to Ryan Haddad from comment #1)
> It looks like this may have started with
> https://trac.webkit.org/changeset/216711/webkit

If 216711 caused this, then I think this test was utterly broken in the first place.
Comment 3 Brady Eidson 2017-05-12 10:56:24 PDT
(In reply to Ryan Haddad from comment #1)
> It looks like this may have started with
> https://trac.webkit.org/changeset/216711/webkit

You say *MAY* have started with this...  how sure are you?
Comment 4 Ryan Haddad 2017-05-12 10:58:59 PDT
(In reply to Brady Eidson from comment #3)
> (In reply to Ryan Haddad from comment #1)
> > It looks like this may have started with
> > https://trac.webkit.org/changeset/216711/webkit
> 
> You say *MAY* have started with this...  how sure are you?

I said may because I did not reproduce it locally, but regression range is between https://trac.webkit.org/changeset/216709 and https://trac.webkit.org/changeset/216712. None of the other commits look related.
Comment 5 Brady Eidson 2017-05-12 11:43:26 PDT
Cannot reproduce this locally no matter what I try.

Looking at changes to the test to make it more resilient.
Comment 6 Brady Eidson 2017-05-12 12:19:30 PDT
Created attachment 309927 [details]
Patch for suppression/exploration
Comment 7 WebKit Commit Bot 2017-05-12 12:20:30 PDT
Comment on attachment 309927 [details]
Patch for suppression/exploration

Rejecting attachment 309927 [details] from commit-queue.

Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-02', 'validate-changelog', '--check-oops', '--non-interactive', 309927, '--port=mac']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

Nobody - Test gardening, in effect found in /Volumes/Data/EWS/WebKit/Tools/ChangeLog does not appear to be a valid reviewer according to contributors.json.
/Volumes/Data/EWS/WebKit/Tools/ChangeLog neither lists a valid reviewer nor contains the string "Unreviewed" or "Rubber stamp" (case insensitive).

Full output: http://webkit-queues.webkit.org/results/3727676
Comment 8 Brady Eidson 2017-05-12 12:41:05 PDT
Created attachment 309928 [details]
Patch for suppression/exploration
Comment 9 WebKit Commit Bot 2017-05-12 13:09:28 PDT
Comment on attachment 309928 [details]
Patch for suppression/exploration

Clearing flags on attachment: 309928

Committed r216777: <http://trac.webkit.org/changeset/216777>
Comment 10 WebKit Commit Bot 2017-05-12 13:09:30 PDT
All reviewed patches have been landed.  Closing bug.
Comment 11 Brady Eidson 2017-05-12 14:10:04 PDT
New failure on bots:

/Volumes/Data/slave/elcapitan-release/build/Tools/TestWebKitAPI/Tests/WebKit2Cocoa/LocalStorageClear.mm:170
Value of: 1ul
  Actual: 1
Expected: websiteDataRecords.count
Which is: 2


/Volumes/Data/slave/elcapitan-release/build/Tools/TestWebKitAPI/Tests/WebKit2Cocoa/LocalStorageClear.mm:186
Value of: originalWebsiteDataRecordCount - 1
  Actual: 1
Expected: websiteDataRecords.count
Which is: 0
Comment 12 Brady Eidson 2017-05-12 14:17:55 PDT
(In reply to Brady Eidson from comment #11)
> New failure on bots:
> 
> /Volumes/Data/slave/elcapitan-release/build/Tools/TestWebKitAPI/Tests/
> WebKit2Cocoa/LocalStorageClear.mm:170
> Value of: 1ul
>   Actual: 1
> Expected: websiteDataRecords.count
> Which is: 2

Since we wiped the entire data store directory *and* installed only a single record of app cache ness, we expect exactly *one* record here.

Getting 2 is entirely unexpected.

> /Volumes/Data/slave/elcapitan-release/build/Tools/TestWebKitAPI/Tests/
> WebKit2Cocoa/LocalStorageClear.mm:186
> Value of: originalWebsiteDataRecordCount - 1
>   Actual: 1
> Expected: websiteDataRecords.count
> Which is: 0

This is actually right - At the end of the test, there should absolutely be 0 results left for getting all data of all types.
Comment 13 Brady Eidson 2017-05-12 14:20:58 PDT
Created attachment 309945 [details]
Get more info from the bots
Comment 14 WebKit Commit Bot 2017-05-12 15:13:59 PDT
The commit-queue encountered the following flaky tests while processing attachment 309945 [details]:

The commit-queue is continuing to process your patch.
Comment 15 WebKit Commit Bot 2017-05-12 15:14:26 PDT
The commit-queue encountered the following flaky tests while processing attachment 309945 [details]:

http/tests/misc/acid3.html bug 165734 (authors: abecsi@webkit.org, arv@chromium.org, cdumez@apple.com, rniwa@webkit.org, and tonyg@chromium.org)
The commit-queue is continuing to process your patch.
Comment 16 WebKit Commit Bot 2017-05-12 15:51:12 PDT
Comment on attachment 309945 [details]
Get more info from the bots

Clearing flags on attachment: 309945

Committed r216800: <http://trac.webkit.org/changeset/216800>
Comment 17 WebKit Commit Bot 2017-05-12 15:51:13 PDT
All reviewed patches have been landed.  Closing bug.
Comment 18 Brady Eidson 2017-05-12 16:15:37 PDT
Reopening since the landed patch will not fix this, but rather hopefully just tell us what's broken.
Comment 19 Brady Eidson 2017-05-12 16:40:07 PDT
Finally got the unexpected results:

/Volumes/Data/slave/sierra-release/build/Tools/TestWebKitAPI/Tests/WebKit2Cocoa/LocalStorageClear.mm:172
Value of: [record.displayName UTF8String]
  Actual: "Local documents on your computer"
Expected: "127.0.0.1"


/Volumes/Data/slave/sierra-release/build/Tools/TestWebKitAPI/Tests/WebKit2Cocoa/LocalStorageClear.mm:174
Value of: [type UTF8String]
  Actual: "WKWebsiteDataTypeIndexedDBDatabases"
Expected: [WKWebsiteDataTypeOfflineWebApplicationCache UTF8String]
Which is: "WKWebsiteDataTypeOfflineWebApplicationCache"

So, bizarrely, there are IndexedDB remnants around.

No clue why, and why nobody can reproduce locally instead of on the bots.
Comment 20 Brady Eidson 2017-05-12 16:41:06 PDT
Also quite bizarre that deleting the IDB directory at the start doesn't resolve this.
Comment 21 Brady Eidson 2017-05-12 16:42:57 PDT
I see, app cache doesn't go into that directory.

Okay, fine.

Let's resolve this once and for all.
Comment 22 Brady Eidson 2017-05-12 16:51:11 PDT
Created attachment 309969 [details]
Patch to fix
Comment 23 Build Bot 2017-05-12 16:52:59 PDT
Attachment 309969 [details] did not pass style-queue:


ERROR: Tools/TestWebKitAPI/Tests/WebKit2Cocoa/LocalStorageClear.mm:139:  This { should be at the end of the previous line  [whitespace/braces] [4]
Total errors found: 1 in 2 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 24 WebKit Commit Bot 2017-05-12 17:31:50 PDT
Comment on attachment 309969 [details]
Patch to fix

Clearing flags on attachment: 309969

Committed r216805: <http://trac.webkit.org/changeset/216805>
Comment 25 WebKit Commit Bot 2017-05-12 17:31:52 PDT
All reviewed patches have been landed.  Closing bug.