Bug 231706 – Implement File System standard

• Component: New Bugs
• Status: REOPENED

[Sihui Liu]

...

[Radar WebKit Bug Importer]

<rdar://problem/84484202>

[Jen Simmons]

*** Bug 213775 has been marked as a duplicate of this bug. ***

[Thomas Steiner]

Now that we have the Origin Private File System, any word about the picker methods?
showOpenFilePicker()
showSaveFilePicker()
showDirectoryPicker()

[Anne van Kesteren]

I think this can be considered done. This work essentially morphed into supporting the File System standard. As per https://github.com/WebKit/standards-positions/issues/28 we don't think the methods mentioned in comment 3 are a good idea.

[Sihui Liu]

Current spec link: https://fs.spec.whatwg.org/

[Sihui Liu]

Reopen this as

[Sihui Liu]

Reopened as we have one interface `FileSystemWritableFileStream` unimplemented as in current spec:
https://fs.spec.whatwg.org/#api-filesystemwritablefilestream

[Adam Zielinski]

I'm a WordPress core committer working at Automattic and WordPress could really use this feature in the WordPress Playground [1] project which runs an entire WordPress in the browser via WebAssembly.

The in-browser storage [2] already works great in Safari, but loading and saving changes to the local directory only works in Google Chrome [3]. Implementing it in WebKit would unlock using Safari as a WordPress development environment or even as a runtime for the Blocknotes app [4] where all notes are stored on the disk and synchronized across all the devices through iCloud. Ditto for other WordPress-based portable WASM apps.

[1] https://developer.wordpress.org/playground
[2] https://github.com/WordPress/wordpress-playground/pull/548
[3] https://github.com/WordPress/wordpress-playground/pull/547
[4] https://wptavern.com/blocknotes-app-runs-wordpress-natively-on-ios-now-in-public-beta

[Alex Titarenko]

FileSystemWritableFileStream will be helpful for my note-taking app as well. It's working in Chrome but not Safari, and currently only working solution with using Web Worker is not very practical for me, so I need this to be implemented in Safari.

[Andrew Hodel]

This is needed in Safari, requiring HTTP range requests and a request per file does not allow a protocol buffer implementation to handle transmission priority.

This works in every browser except Safari and allows the ability to prioritize messages, that's a requirement for control messages.

Protocol Buffer JavaScript Example:
```
var inbound_message_callback = function(m) {

	console.log('inbound_message_callback() called.', m);

	var c = create_message_element(inbound_element, m, 'inbound');

	var file_handle = null;
	var writable_stream = null;
	var file_mem_buf = [];
	var file_data_index = 0;
	var data_chunks_read_into_memory = 0;
	var downloaded = false;

	c.c.addEventListener('dblclick', function(e) {

		if (downloaded === true) {
			alert('already downloaded');
			return;
		}

		// create new file handle with name
		var fh_promise = window.showSaveFilePicker();

		fh_promise.then(function(fh_fullfillment) {

			// if the fullfillment happens of window.showSaveFilePicker()
			// do not allow it to happen again, in order to regain the memory used by file_mem_buf
			// as an inbound message can be terabytes in size
			// and the download to file can be started during and after the message arrival
			downloaded = true;

			file_handle = fh_fullfillment

			console.log('created file handle for message_type 1 message data', file_handle);

			var wrs_promise = fh_fullfillment.createWritable();

			wrs_promise.then(function(wrs_fullfillment) {

				writable_stream = wrs_fullfillment;
				console.log('created writable stream for message_type 1 message data', writable_stream);

				if (data_chunks_read_into_memory === m.total_chunks) {

					// data is already in memory
					// no more activity_callback() invocations

					// add data from memory buffer to file_handle
					for (var l in file_mem_buf) {
						writable_stream.write(file_mem_buf[l]);
					}

					// empty file_mem_buf, this is in the main thread and this activity_callback function must complete before another is run
					file_mem_buf = [];

					// close the writable stream
					writable_stream.close();

				}

			}, function(wrs_rejection) {

				console.log('wrs_rejection', wrs_rejection);

			});

		}, function(fh_rejection) {

			console.log('fh_rejection', fh_rejection);
		
		});

	});

	m.activity_callback = function(e, data_progress, data) {

		// (event String, data_progress Number, data ArrayBuffer)
		// data is always nil unless it is a "data_progress" or "data_complete" event of a message in the ReceiveQueue

		//console.log('inbound message activity_callback', e, data_progress, data, m.transfer_rate_bits_per_microsecond + 'mbps');

		c.update(data_progress);

		if (e === 'data_progress') {

			c.data_chunk(data);

			if (writable_stream === null) {

				// add data to memory buffer
				file_mem_buf.push(data);

				data_chunks_read_into_memory++;

			} else {

				// add data from memory buffer to file_handle
				for (var l in file_mem_buf) {
					writable_stream.write(file_mem_buf[l]);
				}

				// empty file_mem_buf, this is in the main thread and this activity_callback function must complete before another is run
				file_mem_buf = [];

				// add data to file_handle
				writable_stream.write(data);

			}

			// increment the file_data_index
			file_data_index++;

		} else if (e === 'data_complete') {

			if (writable_stream !== null && file_mem_buf.length === 0) {

				// all data in file_mem_buf has been written to a file

				// close the writable stream
				writable_stream.close();

			}

		}

	}

}
```

[Andrew Hodel]

It isn't reasonable that `FileSystemFileHandle.createSyncAccessHandle()` is the only choice to write a file in `OPFS` with Safari because of the requirement of using a `Worker` thread.

The problem with a `Worker` thread is the wasted resource if you are already receiving data to write in blocks in the main thread, it's not worth opening a new `Worker` because the intent is confusion, not work.

[Andrew Hodel]

Without access to this in the main thread, any module that writes files uses non required resource to write files by using a `Worker` thread.

If any other module provides data it is either going to provide it to the main thread or be included in a `Worker` itself where this wouldn't be a problem as the sync handle can write the file from the `Worker`.

The problem with the `Worker` thread is that many times the module cannot be included in the `Worker` with consideration of RAM because if you are requiring a worker pool every worker must have a copy of the module in RAM.

This is very problematic when you get into 3D fonts, because you have 1-4GB of RAM allocated to each worker having the ability to draw fonts and that significantly takes from what is available.

Worker's aren't like subroutines (Golang) or threads (C99 pthread) with regards to modules being accessible globally.  That is why the main thread is often used and what I think comments such as #4 are not considering.

[Andrew Hodel]

It's confusing because when you read the documentation of the specification that the browser claims to implement and then test in a Safari and result failure; you must then read to find out that instead of changing the spec, the browser developer's chose to partially implement it and didn't make an error message that explains their reasoning when `FileSystemFileHandle.createWritable` is used as specified.

[Andrew Hodel]

@Sihui Liu - thank you

https://github.com/WebKit/WebKit/commit/92cb5665c378bef8d73a61a782ad57291520f77d