Bug 106658 - Web Audio Api CreateBuffer and decodeAudioData DOM exception 12 when not on frame boundary
Summary: Web Audio Api CreateBuffer and decodeAudioData DOM exception 12 when not on f...
Status: UNCONFIRMED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Web Audio (show other bugs)
Version: 528+ (Nightly build)
Hardware: All All
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-01-11 05:30 PST by Ladislav Nevery
Modified: 2024-03-08 03:41 PST (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ladislav Nevery 2013-01-11 05:30:38 PST
Lot of people have trouble convincing Web Audio Api to play their files and are receiving weird DOM exception 12.

tested:
mobile safari 6.0.1
latest win7 chrome 24.0.1312.52m

reproduction steps: 

prepend  for example 100  bytes in front of any mp3.
observe DOM exception 12. on xhr/createBuffer/decodeAudioData attempt to decode or play.



I am aware that this is new and evolving technology and  I am very  thankful even for web audio api as it is now since its small miracle that happened to us.

both apis  miss  stream syncing on header boundary which is not used on pure stream files(i.e. no performance hit). But world is not perfect and there is bunch of data in streams these days. And mp3 or many aac/adts files are streaming file formats. streaming means that you can cut them anywhere or insert append anything (various tags even image artwork) decoder shouldnt care about unknown data. decoder should just seek until he finds header he knows and can decode.

I thrown together this temporary solution that seeks to nearest frame header start and passes data from this offset only.

mp3 or mp2 all start their header with 0XFFE and aac(adts) on oxFFF sync-word .therefore both will sync on 0xFFE. Yes its oversimplification
Here is the code I currently use to play previously not played files.

What I hate is that arrayBuffer doesnt have subarray() like its typed childs to return just different view from different offset instead of whole new array copy that slice() returns. if only webaudio api accepted typed arrays like Uint8Array alike as input but unfortunately the only way to create arrayBuffer back to keep type check happy seems huge slice() copy. thankfully usually only one or two seeks are needed. and frames are small i.e. no more than 300 or 600 bytes checked.
Still seeking or pre-decoding mp3 or aac in js doesn't seem right. So i have feeling that frame-sync should be part of api. After all any reasonable streaming format decoder starts with it. Yes there is media element source but that is disabled on iOS and i thing one shot sources shouldn't be picky about files. many people have no idea how to frame align their files in hex editor. probably  mentioning of mandatory frame alignment in documentation will help steer people. what you guys think?

// current temporary quick JS hack that plays files with tags or artwork or partial cuts too
// simply prepend bunch of bytes  in front of previously playable mpg/aac(adts) to simulate the  ones with embedded artwork or tags.

function syncStream(node){
	var buf8 = new Uint8Array(node.buf); 
	buf8.indexOf = Array.prototype.indexOf;
	var i=node.sync, b=buf8;
	while(1) {
		node.retry++;
		i=b.indexOf(0xFF,i); if(i==-1 || (b[i+1] & 0xE0 == 0xE0 )) break;
		i++;
	}
	if(i!=-1) {
		var tmp=node.buf.slice(i); //carefull there it returns copy
		delete(node.buf); node.buf=null;
		node.buf=tmp;
		node.sync=i;
		return true;
	}
	return false;
}

function decode(node) {
	try{
		context.decodeAudioData(node.buf,
		function(decoded){
			node.source  = context.createBufferSource();
			node.source.connect(context.destination);
			node.source.buffer=decoded; 
			node.source.noteOn(node.time);
		},
		function(){ 
			if(syncStream(node)) decode(node);
		});
	} catch(e) {
		log('decode exception',e.message);
	}
}

function playSound(node) { 
	node.xhr = new XMLHttpRequest();
	node.xhr.onload=function(){  
		node.buf=node.xhr.response;
		node.sync=0;
		node.retry=0;
		decode(node);
	}
	node.xhr.open("GET", node.url, true); 
	node.xhr.responseType = "arraybuffer"; 
	node.xhr.send();
}
Comment 1 Chris Rogers 2013-05-10 18:01:39 PDT
decodeAudioData() is only meant to decode entire audio assets, and not ones spliced from the middle.
Comment 4 extrawin 2024-03-07 20:54:22 PST
So how to decodeAudioData the concatenated content from the middle?
https://geometrydashlite.net/
Comment 5 Ladislav Nevery 2024-03-08 03:41:28 PST
Dear Chris. Most Creators are seeing this error on complete files be it mp3 aac or m4a. As I explained containers of streamed formats contain all kind of  stuff and it is perfectly legitimate after 2decades of evolution for files to contain tags artworks embedded by iTunes or other tools. some production tools embed cue points during production  like Mixedinkey or various DJ software embeds key bpm etc . Ie if 2 decades old winamp windows media player or even ITunes can play streamed formats like mp3 m4a with embedded cover art without issue then why web-kit API has decade problem to do so after so many years with response Wont Fix? That's not very developer friendly approach. I guess Apple is protecting it's 30% App-store walled-garden by sabotaging web-audio as it did with refusing to support webgl 2 for so long. Why I event bother.