Bug 175139 - Network cache should be usable as non-singleton
Summary: Network cache should be usable as non-singleton
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Page Loading (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2017-08-03 10:28 PDT by Antti Koivisto
Modified: 2017-08-04 10:02 PDT (History)
8 users (show)

See Also:


Attachments
patch (44.59 KB, patch)
2017-08-03 14:40 PDT, Antti Koivisto
no flags Details | Formatted Diff | Diff
patch (44.60 KB, patch)
2017-08-03 15:27 PDT, Antti Koivisto
no flags Details | Formatted Diff | Diff
patch (46.27 KB, patch)
2017-08-03 15:34 PDT, Antti Koivisto
sam: review+
Details | Formatted Diff | Diff
patch (57.42 KB, patch)
2017-08-04 04:34 PDT, Antti Koivisto
no flags Details | Formatted Diff | Diff
patch (57.60 KB, patch)
2017-08-04 05:21 PDT, Antti Koivisto
commit-queue: commit-queue-
Details | Formatted Diff | Diff
patch (57.60 KB, patch)
2017-08-04 05:45 PDT, Antti Koivisto
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Antti Koivisto 2017-08-03 10:28:12 PDT
Currently we always use it as a singleton. We might want to use it as backend for cache web API and should make it work as a non-singleton instance too.
Comment 1 Antti Koivisto 2017-08-03 14:40:47 PDT
Created attachment 317158 [details]
patch
Comment 2 Antti Koivisto 2017-08-03 15:27:40 PDT
Created attachment 317167 [details]
patch
Comment 3 Antti Koivisto 2017-08-03 15:34:05 PDT
Created attachment 317168 [details]
patch
Comment 4 Sam Weinig 2017-08-03 16:03:52 PDT
Comment on attachment 317168 [details]
patch

Nice! This also has the nice side effect that if we want to do targeted unit testing of it, it's much more straight forward.

One thing that might make things a little nicer, though could be left for later, is to make the NetworkProcess object, which is a singleton, own the NetworkCache instance, any make accessing the singleton be via NetworkProcess::singleton()->diskCache() (or whatever).  It would also make the calls to NetworkCache::Cache::open() that don't use their return values much less mysterious.
Comment 5 Sam Weinig 2017-08-03 16:07:47 PDT
Comment on attachment 317168 [details]
patch

View in context: https://bugs.webkit.org/attachment.cgi?id=317168&action=review

r=me when it all builds

> Source/WebKit/NetworkProcess/cocoa/NetworkProcessCocoa.mm:121
> +            NetworkCache::Cache::open(m_diskCacheDirectory, cacheOptions);

This feels too mysterious.  While I think you should store the Cache ref here, maybe for now just a comment explaining what's happening.

> Source/WebKit/NetworkProcess/soup/NetworkProcessSoup.cpp:124
> +    NetworkCache::Cache::open(m_diskCacheDirectory, cacheOptions);

This feels too mysterious.  While I think you should store the Cache ref here, maybe for now just a comment explaining what's happening.
Comment 6 Antti Koivisto 2017-08-04 01:26:09 PDT
> One thing that might make things a little nicer, though could be left for
> later, is to make the NetworkProcess object, which is a singleton, own the
> NetworkCache instance, any make accessing the singleton be via
> NetworkProcess::singleton()->diskCache() (or whatever).  It would also make
> the calls to NetworkCache::Cache::open() that don't use their return values
> much less mysterious.

Good point. I'll update the patch.
Comment 7 Antti Koivisto 2017-08-04 04:34:13 PDT
Created attachment 317233 [details]
patch
Comment 8 Antti Koivisto 2017-08-04 05:21:36 PDT
Created attachment 317236 [details]
patch
Comment 9 WebKit Commit Bot 2017-08-04 05:23:01 PDT
Comment on attachment 317236 [details]
patch

Rejecting attachment 317236 [details] from commit-queue.

Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-01', 'validate-changelog', '--check-oops', '--non-interactive', 317236, '--port=mac']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

Last 500 characters of output:
a/EWS/WebKit/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 73, in run_and_handle_errors
    self._run(tool, options, state)
  File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 67, in _run
    step(tool, options).run(state)
  File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/tool/steps/validatereviewer.py", line 54, in run
    if changelog_entry.has_valid_reviewer():
AttributeError: 'NoneType' object has no attribute 'has_valid_reviewer'

Full output: http://webkit-queues.webkit.org/results/4252602
Comment 10 Antti Koivisto 2017-08-04 05:44:04 PDT
> AttributeError: 'NoneType' object has no attribute 'has_valid_reviewer'

Huh?
Comment 11 Antti Koivisto 2017-08-04 05:45:06 PDT
Created attachment 317238 [details]
patch
Comment 12 WebKit Commit Bot 2017-08-04 06:25:06 PDT
Comment on attachment 317238 [details]
patch

Clearing flags on attachment: 317238

Committed r220267: <http://trac.webkit.org/changeset/220267>
Comment 13 WebKit Commit Bot 2017-08-04 06:25:08 PDT
All reviewed patches have been landed.  Closing bug.
Comment 14 Radar WebKit Bug Importer 2017-08-04 06:25:26 PDT
<rdar://problem/33723415>
Comment 15 Brady Eidson 2017-08-04 09:40:12 PDT
(In reply to Sam Weinig from comment #4)
> 
> One thing that might make things a little nicer, though could be left for
> later, is to make the NetworkProcess object, which is a singleton, own the
> NetworkCache instance, any make accessing the singleton be via
> NetworkProcess::singleton()->diskCache() (or whatever).  

Actually we need to do better than that (and now we can) because it's faulty to only have a singleton network cache.

This is because the WebKit API now supports multiple persistent data stores, yet all NetworkResourceLoaders with persistent data stores share the same NetworkCache.

We need to have a map of sessionIDs to NetworkCaches (each pointing to a unique location on disk)
Comment 16 Antti Koivisto 2017-08-04 09:57:49 PDT
That should be pretty easy to implement now though it is not clear to me why someone would want split caches within a single client. Some sort of weaker private browsing mode? Things like cache shrinking policy in presence of multiple caches would need to be resolved.

Another approach for that would be to keep a single cache instance but include a data store identifier with the cache key. This would allow data deduplication and existing shrinking policy to function to work without any extra effort.

Or maybe we want to have multiple Cache instances that share the same Storage backend. We are not far architecturally from supporting that either.
Comment 17 Brady Eidson 2017-08-04 10:02:53 PDT
(In reply to Antti Koivisto from comment #16)
> That should be pretty easy to implement now though it is not clear to me why
> someone would want split caches within a single client. Some sort of weaker
> private browsing mode? Things like cache shrinking policy in presence of
> multiple caches would need to be resolved.
> 
> Another approach for that would be to keep a single cache instance but
> include a data store identifier with the cache key. This would allow data
> deduplication and existing shrinking policy to function to work without any
> extra effort.
> 
> Or maybe we want to have multiple Cache instances that share the same
> Storage backend. We are not far architecturally from supporting that either.

I'm not sure which one of these approaches is better, but we should analyze and pick one to move forward with.

Upcoming OSes have clients with multiple sessions in the same Networking Process.
Ping me on IRC for details.