Bug 234103 - Add FileSystem function to read a file at a path
Summary: Add FileSystem function to read a file at a path
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Web Template Framework (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Don Olmstead
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-12-09 14:41 PST by Don Olmstead
Modified: 2021-12-10 16:37 PST (History)
21 users (show)

See Also:


Attachments
WIP Patch (10.21 KB, patch)
2021-12-09 14:41 PST, Don Olmstead
no flags Details | Formatted Diff | Diff
Patch (4.28 KB, patch)
2021-12-09 17:07 PST, Don Olmstead
no flags Details | Formatted Diff | Diff
Patch (14.64 KB, patch)
2021-12-09 17:11 PST, Don Olmstead
no flags Details | Formatted Diff | Diff
Patch (14.55 KB, patch)
2021-12-09 17:14 PST, Don Olmstead
achristensen: commit-queue-
Details | Formatted Diff | Diff
Patch (14.99 KB, patch)
2021-12-09 22:02 PST, Don Olmstead
no flags Details | Formatted Diff | Diff
Patch (15.38 KB, patch)
2021-12-10 11:17 PST, Don Olmstead
no flags Details | Formatted Diff | Diff
Patch (15.33 KB, patch)
2021-12-10 11:23 PST, Don Olmstead
achristensen: review-
Details | Formatted Diff | Diff
Patch (16.09 KB, patch)
2021-12-10 14:01 PST, Don Olmstead
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch (16.08 KB, patch)
2021-12-10 14:09 PST, Don Olmstead
achristensen: review+
Details | Formatted Diff | Diff
Patch (15.98 KB, patch)
2021-12-10 15:08 PST, Don Olmstead
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Don Olmstead 2021-12-09 14:41:27 PST
There's lots of code that wants to load a file into a buffer.
Comment 1 Don Olmstead 2021-12-09 14:41:54 PST Comment hidden (obsolete)
Comment 2 Don Olmstead 2021-12-09 17:07:17 PST Comment hidden (obsolete)
Comment 3 Don Olmstead 2021-12-09 17:11:36 PST Comment hidden (obsolete)
Comment 4 Don Olmstead 2021-12-09 17:14:35 PST
Created attachment 446639 [details]
Patch
Comment 5 Alex Christensen 2021-12-09 17:17:33 PST
Comment on attachment 446639 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=446639&action=review

> Source/WTF/wtf/FileSystem.cpp:543
> +    auto handle = FileSystem::openFile(path, FileSystem::FileOpenMode::Read);

if (FileSystem::isHandleValid(handle)) return std::nullopt;
Comment 6 Don Olmstead 2021-12-09 17:19:53 PST
Comment on attachment 446639 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=446639&action=review

>> Source/WTF/wtf/FileSystem.cpp:543
>> +    auto handle = FileSystem::openFile(path, FileSystem::FileOpenMode::Read);
> 
> if (FileSystem::isHandleValid(handle)) return std::nullopt;

The first line of `readEntireFile` is that check so I didn't want to do that check twice.
Comment 7 Alex Christensen 2021-12-09 17:39:34 PST
ok
Comment 8 Don Olmstead 2021-12-09 22:02:09 PST
Created attachment 446660 [details]
Patch

Adding another usage
Comment 9 Sihui Liu 2021-12-10 11:01:05 PST
Comment on attachment 446660 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=446660&action=review

> Source/JavaScriptCore/inspector/remote/socket/RemoteInspectorSocket.cpp:268
> +    return contents.has_value() ? String::adopt(WTFMove(contents.value())) : emptyString();

You may remove has_value(), std::optional supports operator bool; and you already use "content ? ... : ... " below.

> Source/WTF/wtf/FileSystem.cpp:538
> +std::optional<Vector<uint8_t>> readEntireFileAt(const String& path)

We don't use "At" for functions that takes path string parameter in FileSystem.h/.cpp, so it seems better to just use "readEntireFile".

> Source/WebCore/platform/network/curl/CurlCacheEntry.cpp:118
> +    if (!buffer.has_value()) {

has_value() not needed.

> Source/WebCore/platform/network/curl/CurlCacheEntry.cpp:158
> +    if (!buffer.has_value()) {

Ditto.

> Source/WebCore/platform/network/curl/CurlCacheEntry.cpp:-227
> -bool CurlCacheEntry::loadFileToBuffer(const String& filepath, Vector<uint8_t>& buffer)

Do you need to remove its definition in header file?

> Source/WebCore/platform/network/curl/CurlCacheManager.cpp:106
> +    if (!buffer.has_value()) {

has_value() not needed.

> Tools/TestWebKitAPI/Tests/WTF/FileSystem.cpp:830
> +    EXPECT_FALSE(FileSystem::readEntireFile(FileSystem::invalidPlatformFileHandle).has_value());
> +    EXPECT_FALSE(FileSystem::readEntireFileAt(emptyString()).has_value());
> +    EXPECT_FALSE(FileSystem::readEntireFileAt(FileSystem::pathByAppendingComponent(tempEmptyFolderPath(), "does-not-exist")).has_value());
> +    EXPECT_FALSE(FileSystem::readEntireFileAt(tempEmptyFilePath()).has_value());
> +
> +    auto buffer = FileSystem::readEntireFileAt(tempFilePath());
> +    EXPECT_TRUE(buffer.has_value());

Ditto
Comment 10 Alex Christensen 2021-12-10 11:17:02 PST
Comment on attachment 446660 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=446660&action=review

> Source/WTF/wtf/FileSystem.cpp:540
> +    if (path.isEmpty())

Also, I'm not sure if this does anything.  Won't openFile return an invalid file handle if given an empty path?
Comment 11 Don Olmstead 2021-12-10 11:17:32 PST Comment hidden (obsolete)
Comment 12 Don Olmstead 2021-12-10 11:23:16 PST
Created attachment 446768 [details]
Patch

Addressing both reviewer feedback
Comment 13 Don Olmstead 2021-12-10 11:24:19 PST
(In reply to Alex Christensen from comment #10)
> Comment on attachment 446660 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=446660&action=review
> 
> > Source/WTF/wtf/FileSystem.cpp:540
> > +    if (path.isEmpty())
> 
> Also, I'm not sure if this does anything.  Won't openFile return an invalid
> file handle if given an empty path?

The tests say yes so removed.
Comment 14 Alex Christensen 2021-12-10 12:04:57 PST
Comment on attachment 446768 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=446768&action=review

> Source/WTF/wtf/FileSystem.cpp:541
> +    auto contents = readEntireFile(handle);

readEntireFile calls readFromFile once.  readFromFile can return a positive number of bytes indicating that the file was partially read successfully and you need to call it again to read the rest.  I think it would be worth fixing that while making more functionality rely on it just working.  Otherwise we will increase the number of times a file read is reported to fail, especially with larger files.
It would probably also be worth returning failure if the large allocation fails instead of crashing.
Comment 15 Don Olmstead 2021-12-10 14:01:16 PST
Created attachment 446807 [details]
Patch

Read in a loop until all the contents are read.
Comment 16 Don Olmstead 2021-12-10 14:09:00 PST
Created attachment 446808 [details]
Patch
Comment 17 Alex Christensen 2021-12-10 14:54:20 PST
Comment on attachment 446808 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=446808&action=review

> Source/WTF/wtf/FileSystem.cpp:531
> +    unsigned totalBytesRead = 0;

nit: this might limit the capability to 4GB.  size_t would increase that to the size of the virtual address space.

> Source/WebCore/platform/network/curl/CurlCacheEntry.cpp:123
> +    if (auto bufferSize = buffer.value().size())

nit: buffer->size() would also work

> Source/WebCore/platform/network/curl/CurlCacheEntry.cpp:124
> +        job->getInternal()->client()->didReceiveBuffer(job, SharedBuffer::create(WTFMove(buffer.value())), bufferSize);

nit: WTFMove(*buffer) would also work.  I don't actually care about this though.

> Source/WebCore/rendering/RenderThemeWin.cpp:1028
> -        return String();
> +        return emptyString();

Why was this change made?  I guess it makes it more consistent with the !USE(CF) path, but why not change that to be a null string?
Comment 18 Don Olmstead 2021-12-10 15:08:19 PST
Created attachment 446825 [details]
Patch
Comment 19 EWS 2021-12-10 16:36:37 PST
Committed r286883 (245113@main): <https://commits.webkit.org/245113@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 446825 [details].
Comment 20 Radar WebKit Bug Importer 2021-12-10 16:37:21 PST
<rdar://problem/86345889>