Unable to postMessage a WebAssembly module to a worklet

• Component: Web Audio
• Status: RESOLVED FIXED

[Richard Newman]

WebAssembly modules can be sent between workers and pages via postMessage.

Both Firefox and Chrome also allow WebAssembly modules to be sent from a page to an AudioWorklet via its MessagePort:

  https://developer.mozilla.org/en-US/docs/Web/API/AudioWorkletNode/port

E.g.,

```
  this.port.postMessage({ message: 'hey', module });
```

in Safari Technology Preview (117, WebKit 15611.1.7.2), this throws 

```
  DataCloneError: The object can not be cloned.
```

(FWIW, adding `module` to the transfer list doesn't help; I guess Safari doesn't support transfer lists either?)

The lack of this capability prevents a common code loading approach for AudioWorklets, where some large chunk of WebAssembly is loaded in a worker and sent via the page down to the AudioWorklet via the MessagePort. 

I have not yet verified whether there's a workaround by posting a multi-megabyte ArrayBuffer and compiling it inside the worklet.

[Radar WebKit Bug Importer]

<rdar://problem/72596556>

[Yusuke Suzuki]

Currently, MessagePort postMessage w/ WebAssembly.Module is not supported.

We need to know the notion of the followings.

1. This pair of MessagePorts are in the same process so that we can send SerializedScriptValue without using IPC back-and-force
2. This pair of MessagePorts are in the different process so that we should just throw an onmessageerror exception.

Chris, do you know about MessagePort's remote/not-remote flags? Is it specified?

Note: https://bugzilla.mozilla.org/show_bug.cgi?id=1605566

[Yusuke Suzuki]

Note that LayoutTests/imported/w3c/web-platform-tests/wasm/serialization/module/window-messagechannel-success.html covers this.

[Chris Dumez]

(In reply to Yusuke Suzuki from comment #2)
> Currently, MessagePort postMessage w/ WebAssembly.Module is not supported.
> 
> We need to know the notion of the followings.
> 
> 1. This pair of MessagePorts are in the same process so that we can send
> SerializedScriptValue without using IPC back-and-force
> 2. This pair of MessagePorts are in the different process so that we should
> just throw an onmessageerror exception.
> 
> Chris, do you know about MessagePort's remote/not-remote flags? Is it
> specified?
> 
> Note: https://bugzilla.mozilla.org/show_bug.cgi?id=1605566

@Yusuke: I don't quite understand the need for "different-process". Our worklets always run in process.

The only case where we use message port cross-process is with service workers, which are different from worklets.

[Chris Dumez]

(In reply to Chris Dumez from comment #4)
> (In reply to Yusuke Suzuki from comment #2)
> > Currently, MessagePort postMessage w/ WebAssembly.Module is not supported.
> > 
> > We need to know the notion of the followings.
> > 
> > 1. This pair of MessagePorts are in the same process so that we can send
> > SerializedScriptValue without using IPC back-and-force
> > 2. This pair of MessagePorts are in the different process so that we should
> > just throw an onmessageerror exception.
> > 
> > Chris, do you know about MessagePort's remote/not-remote flags? Is it
> > specified?
> > 
> > Note: https://bugzilla.mozilla.org/show_bug.cgi?id=1605566
> 
> @Yusuke: I don't quite understand the need for "different-process". Our
> worklets always run in process.
> 
> The only case where we use message port cross-process is with service
> workers, which are different from worklets.

Based on the error, it looks like nobody implemented the serialization of modules in SerializedScriptValue? If so, sending the module to an iframe via postMessage would fail too (has this been tested?)

[Richard Newman]

I have not tested iframe messaging.

Web worker to page seems to work correctly.

[Chris Dumez]

(In reply to Richard Newman from comment #6)
> I have not tested iframe messaging.
> 
> Web worker to page seems to work correctly.

Oh I see. If worker to page works then I would imagine iframe messaging would work too. This must be specific to worklets somehow.

[Chris Dumez]

(In reply to Chris Dumez from comment #7)
> (In reply to Richard Newman from comment #6)
> > I have not tested iframe messaging.
> > 
> > Web worker to page seems to work correctly.
> 
> Oh I see. If worker to page works then I would imagine iframe messaging
> would work too. This must be specific to worklets somehow.

In SerializedScriptValue:

                if (m_context != SerializationContext::WorkerPostMessage) {
                    code = SerializationReturnCode::DataCloneError;
                    return true;
                }

When encoding wasm modules. We are likely not using the WorkerPostMessage SerializationContext when messaging audio worklets?

[Chris Dumez]

(In reply to Chris Dumez from comment #8)
> (In reply to Chris Dumez from comment #7)
> > (In reply to Richard Newman from comment #6)
> > > I have not tested iframe messaging.
> > > 
> > > Web worker to page seems to work correctly.
> > 
> > Oh I see. If worker to page works then I would imagine iframe messaging
> > would work too. This must be specific to worklets somehow.
> 
> In SerializedScriptValue:
> 
>                 if (m_context != SerializationContext::WorkerPostMessage) {
>                     code = SerializationReturnCode::DataCloneError;
>                     return true;
>                 }
> 
> When encoding wasm modules. We are likely not using the WorkerPostMessage
> SerializationContext when messaging audio worklets?

MessagePort::postMessage() seems to use the default serialization context, no matter in which context the message port is used:

===
ExceptionOr<void> MessagePort::postMessage(JSC::JSGlobalObject& state, JSC::JSValue messageValue, PostMessageOptions&& options)
{
    LOG(MessagePorts, "Attempting to post message to port %s (to be received by port %s)", m_identifier.logString().utf8().data(), m_remoteIdentifier.logString().utf8().data());

    registerLocalActivity();

    Vector<RefPtr<MessagePort>> ports;
    auto messageData = SerializedScriptValue::create(state, messageValue, WTFMove(options.transfer), ports);
===

[Yusuke Suzuki]

(In reply to Chris Dumez from comment #9)
> (In reply to Chris Dumez from comment #8)
> > (In reply to Chris Dumez from comment #7)
> > > (In reply to Richard Newman from comment #6)
> > > > I have not tested iframe messaging.
> > > > 
> > > > Web worker to page seems to work correctly.
> > > 
> > > Oh I see. If worker to page works then I would imagine iframe messaging
> > > would work too. This must be specific to worklets somehow.
> > 
> > In SerializedScriptValue:
> > 
> >                 if (m_context != SerializationContext::WorkerPostMessage) {
> >                     code = SerializationReturnCode::DataCloneError;
> >                     return true;
> >                 }
> > 
> > When encoding wasm modules. We are likely not using the WorkerPostMessage
> > SerializationContext when messaging audio worklets?
> 
> MessagePort::postMessage() seems to use the default serialization context,
> no matter in which context the message port is used:
> 
> ===
> ExceptionOr<void> MessagePort::postMessage(JSC::JSGlobalObject& state,
> JSC::JSValue messageValue, PostMessageOptions&& options)
> {
>     LOG(MessagePorts, "Attempting to post message to port %s (to be received
> by port %s)", m_identifier.logString().utf8().data(),
> m_remoteIdentifier.logString().utf8().data());
> 
>     registerLocalActivity();
> 
>     Vector<RefPtr<MessagePort>> ports;
>     auto messageData = SerializedScriptValue::create(state, messageValue,
> WTFMove(options.transfer), ports);
> ===

I changed it to WorkerPostMessage serialization context, and it didn't work because MessagePort always serialize (via IPC encoder) SerializedScriptValue and deserialized it even though both ports are used by the workers/worklets in the same process.
The problem is that we do not support full serialization of WebAssembly.Module.

1. It becomes super huge size
2. It is really complicated
3. Serialize and deserialize it will double memory size while WebAssembly.Module can be shared between multiple threads safely (it is designed to be a thread-safe).

So, we need a notion of "MessageChannel is not used among different processes", and we should avoid using IPC encoder / decoder to serialize SerializedScriptValue for MessagePort. And I think this is what gecko is doing.

[Chris Dumez]

(In reply to Yusuke Suzuki from comment #10)
> (In reply to Chris Dumez from comment #9)
> > (In reply to Chris Dumez from comment #8)
> > > (In reply to Chris Dumez from comment #7)
> > > > (In reply to Richard Newman from comment #6)
> > > > > I have not tested iframe messaging.
> > > > > 
> > > > > Web worker to page seems to work correctly.
> > > > 
> > > > Oh I see. If worker to page works then I would imagine iframe messaging
> > > > would work too. This must be specific to worklets somehow.
> > > 
> > > In SerializedScriptValue:
> > > 
> > >                 if (m_context != SerializationContext::WorkerPostMessage) {
> > >                     code = SerializationReturnCode::DataCloneError;
> > >                     return true;
> > >                 }
> > > 
> > > When encoding wasm modules. We are likely not using the WorkerPostMessage
> > > SerializationContext when messaging audio worklets?
> > 
> > MessagePort::postMessage() seems to use the default serialization context,
> > no matter in which context the message port is used:
> > 
> > ===
> > ExceptionOr<void> MessagePort::postMessage(JSC::JSGlobalObject& state,
> > JSC::JSValue messageValue, PostMessageOptions&& options)
> > {
> >     LOG(MessagePorts, "Attempting to post message to port %s (to be received
> > by port %s)", m_identifier.logString().utf8().data(),
> > m_remoteIdentifier.logString().utf8().data());
> > 
> >     registerLocalActivity();
> > 
> >     Vector<RefPtr<MessagePort>> ports;
> >     auto messageData = SerializedScriptValue::create(state, messageValue,
> > WTFMove(options.transfer), ports);
> > ===
> 
> I changed it to WorkerPostMessage serialization context, and it didn't work
> because MessagePort always serialize (via IPC encoder) SerializedScriptValue
> and deserialized it even though both ports are used by the workers/worklets
> in the same process.
> The problem is that we do not support full serialization of
> WebAssembly.Module.
> 
> 1. It becomes super huge size
> 2. It is really complicated
> 3. Serialize and deserialize it will double memory size while
> WebAssembly.Module can be shared between multiple threads safely (it is
> designed to be a thread-safe).
> 
> So, we need a notion of "MessageChannel is not used among different
> processes", and we should avoid using IPC encoder / decoder to serialize
> SerializedScriptValue for MessagePort. And I think this is what gecko is
> doing.

From MessagePort, you can query identifier() and remoteIdentifier(). Each returns a MessagePortIdentifier which is a struct that contains both a processIdentifier and a portIdentifier. It should therefore be trivial to figure out if the local and remote ports are hosted in the same process check comparing identifier().processIdentifier and remoteIdentifier().processIdentifier.

[Chris Dumez]

(In reply to Chris Dumez from comment #11)
> (In reply to Yusuke Suzuki from comment #10)
> > (In reply to Chris Dumez from comment #9)
> > > (In reply to Chris Dumez from comment #8)
> > > > (In reply to Chris Dumez from comment #7)
> > > > > (In reply to Richard Newman from comment #6)
> > > > > > I have not tested iframe messaging.
> > > > > > 
> > > > > > Web worker to page seems to work correctly.
> > > > > 
> > > > > Oh I see. If worker to page works then I would imagine iframe messaging
> > > > > would work too. This must be specific to worklets somehow.
> > > > 
> > > > In SerializedScriptValue:
> > > > 
> > > >                 if (m_context != SerializationContext::WorkerPostMessage) {
> > > >                     code = SerializationReturnCode::DataCloneError;
> > > >                     return true;
> > > >                 }
> > > > 
> > > > When encoding wasm modules. We are likely not using the WorkerPostMessage
> > > > SerializationContext when messaging audio worklets?
> > > 
> > > MessagePort::postMessage() seems to use the default serialization context,
> > > no matter in which context the message port is used:
> > > 
> > > ===
> > > ExceptionOr<void> MessagePort::postMessage(JSC::JSGlobalObject& state,
> > > JSC::JSValue messageValue, PostMessageOptions&& options)
> > > {
> > >     LOG(MessagePorts, "Attempting to post message to port %s (to be received
> > > by port %s)", m_identifier.logString().utf8().data(),
> > > m_remoteIdentifier.logString().utf8().data());
> > > 
> > >     registerLocalActivity();
> > > 
> > >     Vector<RefPtr<MessagePort>> ports;
> > >     auto messageData = SerializedScriptValue::create(state, messageValue,
> > > WTFMove(options.transfer), ports);
> > > ===
> > 
> > I changed it to WorkerPostMessage serialization context, and it didn't work
> > because MessagePort always serialize (via IPC encoder) SerializedScriptValue
> > and deserialized it even though both ports are used by the workers/worklets
> > in the same process.
> > The problem is that we do not support full serialization of
> > WebAssembly.Module.
> > 
> > 1. It becomes super huge size
> > 2. It is really complicated
> > 3. Serialize and deserialize it will double memory size while
> > WebAssembly.Module can be shared between multiple threads safely (it is
> > designed to be a thread-safe).
> > 
> > So, we need a notion of "MessageChannel is not used among different
> > processes", and we should avoid using IPC encoder / decoder to serialize
> > SerializedScriptValue for MessagePort. And I think this is what gecko is
> > doing.
> 
> From MessagePort, you can query identifier() and remoteIdentifier(). Each
> returns a MessagePortIdentifier which is a struct that contains both a
> processIdentifier and a portIdentifier. It should therefore be trivial to
> figure out if the local and remote ports are hosted in the same process
> check comparing identifier().processIdentifier and
> remoteIdentifier().processIdentifier.

Indeed, it seems we are going to the NetworkProcess via IPC even though the source and destination processes are identical. This is definitely suboptimal in general and breaking for Wasm modules since those are currently not IPC-encoded.

[Chris Dumez]

With the WIP patch at Bug 220627 (to avoid IPC when both MessagePorts are in the same process) and with a change to MessagePort::postMessage() to use SerializationContext::WorkerPostMessage when necessary, LayoutTests/imported/w3c/web-platform-tests/wasm/serialization/module/window-messagechannel-success.html starts to PASS.

[Yusuke Suzuki]

(In reply to Chris Dumez from comment #13)
> With the WIP patch at Bug 220627 (to avoid IPC when both MessagePorts are in
> the same process) and with a change to MessagePort::postMessage() to use
> SerializationContext::WorkerPostMessage when necessary,
> LayoutTests/imported/w3c/web-platform-tests/wasm/serialization/module/window-
> messagechannel-success.html starts to PASS.

Perfect!

[Chris Dumez]

Created attachment 418682 [details]
WIP Patch

[Chris Dumez]

Comment on attachment 418682 [details]
WIP Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=418682&action=review

> Source/WebCore/dom/MessagePort.cpp:141
> +    // FIXME: At the time of serialization, we don't know if the target is a window, a worker or something else. We will only know at the time of

@Yusuke: Not sure how to best deal with this. Passing SerializationContext::WorkerPostMessage unconditionally here is a hack.

[Yusuke Suzuki]

Comment on attachment 418682 [details]
WIP Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=418682&action=review

> LayoutTests/imported/w3c/web-platform-tests/wasm/serialization/module/window-messagechannel-success-expected.txt:2
> +PASS postMessaging to a dedicated worker via MessageChannel allows them to instantiate

Can we have a test case that is posting wasm module to message-ports tied to the different process and getting an error in deserialization side?
Ideally, we should throw an error, and we should report onmessageerror since we failed deserialization.
Maybe, we should add a test for checking that error, and after that, we should file a bug for `onmessageerror` handler invocation and implement it.
https://developer.mozilla.org/en-US/docs/Web/API/Worker/onmessageerror
https://developer.mozilla.org/en-US/docs/Web/API/MessagePort/onmessageerror

>> Source/WebCore/dom/MessagePort.cpp:141
>> +    // FIXME: At the time of serialization, we don't know if the target is a window, a worker or something else. We will only know at the time of
> 
> @Yusuke: Not sure how to best deal with this. Passing SerializationContext::WorkerPostMessage unconditionally here is a hack.

This hack looks OK to me! If we attempt to deserialize non-sharable-between-processes objects, then we will get an error at deserialization (not causing catastrophic crash) since our SerializedScriptValue's deserialization is robust enough against that.

Created attachment 435429 [details]
Test case

Note that WebAssembly modules should be serialisable to be used in processorOptions at construction time between an AudioWorkletNote and and AudioWorkletProcessor.

[Chris Dumez]

Comment on attachment 418682 [details]
WIP Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=418682&action=review

>> LayoutTests/imported/w3c/web-platform-tests/wasm/serialization/module/window-messagechannel-success-expected.txt:2
>> +PASS postMessaging to a dedicated worker via MessageChannel allows them to instantiate
> 
> Can we have a test case that is posting wasm module to message-ports tied to the different process and getting an error in deserialization side?
> Ideally, we should throw an error, and we should report onmessageerror since we failed deserialization.
> Maybe, we should add a test for checking that error, and after that, we should file a bug for `onmessageerror` handler invocation and implement it.
> https://developer.mozilla.org/en-US/docs/Web/API/Worker/onmessageerror
> https://developer.mozilla.org/en-US/docs/Web/API/MessagePort/onmessageerror

WebKit currently doesn't support onmessageerror AFAIK

[Chris Dumez]

Created attachment 435798 [details]
WIP patch

[Chris Dumez]

Created attachment 435799 [details]
WIP patch

[Chris Dumez]

Created attachment 435812 [details]
WIP patch

[Chris Dumez]

Created attachment 435820 [details]
WIP patch

[Chris Dumez]

Created attachment 435860 [details]
WIP patch

[Chris Dumez]

Created attachment 435862 [details]
Patch

[Chris Dumez]

Created attachment 435865 [details]
Patch

[Chris Dumez]

Created attachment 435882 [details]
Patch

[Chris Dumez]

Created attachment 436196 [details]
Patch

[Chris Dumez]

Created attachment 436207 [details]
Patch

[Ian Kettlewell]

A key detail worth noting: It looks like WebAssembly.Memory objects cannot be sent between workers via postMessage either.

An important use for postMessaging WebAssembly modules / memory is to use an Audio Worklet in a way similar to how a non-web dedicated audio thread would work. The Audio Worklet runs a Wasm instance backed by a SharedArrayBuffer that communicates with the main thread via Wasm atomics. That flow makes it *much* easier to for things like game engines to share audio backend code between non-web and web.

I think (but haven't verified) the module being un-postMessageable can be worked around by postMessaging the code and re-instantiating the module on the Worklet.

However as far as I'm aware the WebAssembly.Memory object being non-postMessagable does not have a workaround. That blocks the use-case I've described here.

Any news on this issue? Any expected date for a proper fix to appear? Thanks.

[Tibor Klajnscek]

+1 for a status update - this is blocking us from using our audio worklet backend which is critical to consistent and crackle-free audio in our products.

[Attila Haraszti]

This is currently a blocker for us, a fix would be highly appreciated.

[Tibor Klajnscek]

This seems to work in the latest Safari dev preview, but I haven't found any release notes mentioning it. 

Is it just a fluke or has it been fixed? And if so, will it make it into Safari 16?

[Ahmad Saleem]

@Chris - it seems this patch passes all tests and only have "style" issues? Is it something on your radar and "see also" bug is also fixed? Thanks!

[Chris Dumez]

Pull request: https://github.com/WebKit/WebKit/pull/6596

[EWS]

Committed 256853@main (a9d61c9d9e2e): <https://commits.webkit.org/256853@main>

Reviewed commits have been landed. Closing PR #6596 and removing active labels.