Getting "TypeError: Underlying ArrayBuffer has been detached from the view" error with WebAssembly code

• Component: WebAssembly
• Status: RESOLVED INVALID

I get "TypeError: Underlying ArrayBuffer has been detached from the view" error when loading and WebAssembly code compiled with emscripten.

1) On this test page: https://faust.grame.fr/dynamic/faustlive-wasm.html

2) Open this URL on another Web page : https://faust.grame.fr/modules

3) Drop the https://faust.grame.fr/modules/AtonalSoftHarp.dsp URL on the faustlive-wasm.html drop zone (this can be done by drag/drop the FAUST button)

4) This activate an Emscripten WebAssembly compiled piece of code, and cause this "TypeError: Underlying ArrayBuffer has been detached from the view". message.

5) Drag/drop the https://faust.grame.fr/modules/AtonalSoftHarp.dsp URL again finally works (we see several "enlarged memory arrays from 16777216 to 33554432, took 0 ms (has ArrayBuffer.transfer? false)" kind of messages in the JavaScript console.

5) This issue does not appear under Chrome Canary or Firefox Nightly. This seems to be a Webkit only issue.

[JF Bastien]

I follow your instructions, but on 3) I get "FaustDSP : 1 : ERROR : syntax error, unexpected SEQ, expecting LPAR or DEF".

From what you've described it looks like you're using one of validate / compile / compileAndInstantiate, passing it an ArrayBuffer or ArrayBufferView, and we think it's been neutered. Can you instrument your code before these functions are used, and print out info about the buffer you're trying to compile (is it an ArrayBuffer or ArrayBufferView, and was it neutered?). Are you using Workers at all (say, download in one and compile in another?).

Also, what browser and version are you using?

You can actually reproduce it in a simple way using another tool:

1) got to : https://faust.grame.fr/editor/

2) copy/paste the faust source code from here : http://faust.grame.fr/modules/AtonalSoftHarp.dsp in the Faust Editor

3) then compile it with the left upper button on the lead (arrow)

4) first time you'll have the "TypeError: Underlying ArrayBuffer has been detached from the view" message, and second time the Faust source code is correctly compiled and runs


"and was it neutered" ==>  what does "neutered" means? how can I display that? 

We don't use Workers.

[Keith Miller]

It looks like you are growing your memory causes any TypedArray looking at the old memory to be detached (inaccessible). This is causing the issue you see, if you make a new typed array looking at the same wasm memory range that should solve you issue.

Maybe you don't grow memory on FF or Chrome? It could also be they both have a bug.

See: https://webassembly.github.io/spec/js-api/index.html#dom-memory-grow

I am not growing myself. We have compiled our C++ code with Emscripten. We do dynamic memory allocation in our C++ code (the Faust compiler that build an internal tree structure and so son..). So this cause the Emscripten memory allocator to handle the situation. 
The same test run on FireFox or Chrome causes the same "enlarged memory arrays from 16777216 to 33554432" message displayed by Emscripten, but do not end up in this "TypeError: Underlying ArrayBuffer has been detached from the view", that is Webkit specific. 
The same test on FireFox or Chrome  works the first time, but on Webkit we get this  "TypeError: Underlying ArrayBuffer has been detached from the view" the first time the Faust compiler handes the test. The second time, I understand that the Emscripten memory allocator has already "enlarged its memory", then the Faust compilation finally works.
Since the issue only happened on Webkit, the seems like an issue in "Emscripten running on Webkit". Should I report the problem on the Emscripten developer list also?

[Alon Zakai]

When emscripten grows memory, it should update all the views for the new buffer, making it impossible to access the neutered data. That happens here: https://github.com/kripken/emscripten/blob/incoming/src/preamble.js#L874

However, it's possible in theory that user code would keep a reference to one of those heap views, like HEAP8, which gets outdated. That seems unlikely to only show up on one browser, though. But it's possible in theory that a browser difference could exist in timing of event callbacks.

To investigate this, I'd recommend one or both of

 * Building the code with --profiling so that it will give useful stack traces. Then hopefully WebKit provides a full stack trace of the location it throws that error, and maybe that will make it obvious what's going on.

 * Remove all possible sources of timing differences: while you can't avoid the async compilation, make sure that you start to use the code and do everything to show the bug in synchronous code. When you have that, you can add some console.logs in a way that should make it clear if this is a WebKit bug or not, as everything should be deterministic.

"Building the code with --profiling" ==> do you mean building the WebKit code here yes?

[Keith Miller]

(In reply to Alon Zakai from comment #5)
> When emscripten grows memory, it should update all the views for the new
> buffer, making it impossible to access the neutered data. That happens here:
> https://github.com/kripken/emscripten/blob/incoming/src/preamble.js#L874
> 
> However, it's possible in theory that user code would keep a reference to
> one of those heap views, like HEAP8, which gets outdated. That seems
> unlikely to only show up on one browser, though. But it's possible in theory
> that a browser difference could exist in timing of event callbacks.
> 
> To investigate this, I'd recommend one or both of
> 
>  * Building the code with --profiling so that it will give useful stack
> traces. Then hopefully WebKit provides a full stack trace of the location it
> throws that error, and maybe that will make it obvious what's going on.
> 
>  * Remove all possible sources of timing differences: while you can't avoid
> the async compilation, make sure that you start to use the code and do
> everything to show the bug in synchronous code. When you have that, you can
> add some console.logs in a way that should make it clear if this is a WebKit
> bug or not, as everything should be deterministic.

The code is keeping a reference to the buffer. IIRC, the code was something like:

argv_something_ptr = new Int32Array(Module.HEAP32.buffer,...);

[Alon Zakai]

> The code is keeping a reference to the buffer. IIRC, the code was something like: argv_something_ptr = new Int32Array(Module.HEAP32.buffer,...);

That looks like it can explain this, yeah. It's not valid to keep using argv_something_ptr after a memory growth event. letz: you should use Module.HEAP32 each time, so you always use the current buffer.

However, it is still quite odd this only happens in one browser. But maybe event timing can explain that somehow.

> "Building the code with --profiling" ==> do you mean building the WebKit code here yes?

Sorry, no, I meant to build the project to JS/wasm that way (so that the JS and wasm have useful stack traces etc.). But maybe not necessary any more given the above.

"It's not valid to keep using argv_something_ptr after a memory growth event. letz: you should use Module.HEAP32 each time, so you always use the current buffer."

==> so how to get this "memory growth event." ? Can I use it to redo my code again?

OK problem solved in my code. Thanks all you guys for the help!

[JF Bastien]

(In reply to letz from comment #10)
> OK problem solved in my code. Thanks all you guys for the help!

Can you elaborate on what the fix was? It sounds like there might be a browser inconsistency. We should fix it.

I was doing:

- var argv_something_ptr = new Int32Array(Module.HEAP32.buffer,...);

- then calling an Emscripten exported function that was actually causing memory growth

- then using the argv_something_ptr after that call
 
Doing argv_something_ptr = new Int32Array(Module.HEAP32.buffer,...) again, and using the updated argv_something_ptr solved the issue.

[JF Bastien]

(In reply to letz from comment #12)
> I was doing:
> 
> - var argv_something_ptr = new Int32Array(Module.HEAP32.buffer,...);
> 
> - then calling an Emscripten exported function that was actually causing
> memory growth
> 
> - then using the argv_something_ptr after that call
>  
> Doing argv_something_ptr = new Int32Array(Module.HEAP32.buffer,...) again,
> and using the updated argv_something_ptr solved the issue.

Great, thanks for the clarification. Sounds like either there's a race or other browsers are buggy. Will ping them.