Bug 228732 - Audio buffer may contain more frames than decoded.
Summary: Audio buffer may contain more frames than decoded.
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Web Audio (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Jean-Yves Avenard [:jya]
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-08-02 23:35 PDT by Jean-Yves Avenard [:jya]
Modified: 2021-08-11 17:46 PDT (History)
9 users (show)

See Also:


Attachments
Patch (6.67 KB, patch)
2021-08-09 22:04 PDT, Jean-Yves Avenard [:jya]
no flags Details | Formatted Diff | Diff
Patch (441.23 KB, patch)
2021-08-10 06:27 PDT, Jean-Yves Avenard [:jya]
no flags Details | Formatted Diff | Diff
Patch (441.28 KB, patch)
2021-08-11 17:16 PDT, Jean-Yves Avenard [:jya]
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jean-Yves Avenard [:jya] 2021-08-02 23:35:00 PDT
In AudioFileReader we read the number of available frames via a call to ExtAudioFileGetProperty [1]

However this provides the amount of frames as reported by the container, which may not always match the number of decoded frames, particularly once trimming is apply (such as when applying the XING header trimming properties).

We then decode the data via a call to ExtAudioFileRead [2], which will return the actual number of frames read. However we continue to return an AudioBus with the same length as originally returned, this is typically too long and will incorrectly contain silence.

the audioBus length should be set to the number of frames actually read.

[1] https://webkit-search.igalia.com/webkit/rev/96dd9c630bcd253a03e0844e7b8c440e5cd1a2b8/Source/WebCore/platform/audio/cocoa/AudioFileReaderCocoa.cpp#396-399
[2] https://webkit-search.igalia.com/webkit/rev/96dd9c630bcd253a03e0844e7b8c440e5cd1a2b8/Source/WebCore/platform/audio/cocoa/AudioFileReaderCocoa.cpp#563-567
Comment 1 Radar WebKit Bug Importer 2021-08-02 23:41:24 PDT
<rdar://problem/81447014>
Comment 2 Jean-Yves Avenard [:jya] 2021-08-09 22:04:33 PDT
Created attachment 435241 [details]
Patch
Comment 3 Jean-Yves Avenard [:jya] 2021-08-09 22:15:30 PDT
Turned out that our use of EXTAudioFileRead was incorrect.

The callers should call `ExtAudioFileRead` again when the initial call reads fewer bytes than requested.

From the docs:
```
@param ioNumberFrames
On entry, ioNumberFrames is the number of frames to be read from the file.
On exit, it is the number of frames actually read. A number of factors may
cause a fewer number of frames to be read, including the supplied buffers
not being large enough, and internal optimizations. If 0 frames are
returned, however, this indicates that end-of-file was reached.
```
Comment 4 Jean-Yves Avenard [:jya] 2021-08-10 06:27:26 PDT
Created attachment 435252 [details]
Patch

Add extra test
Comment 5 Eric Carlson 2021-08-10 07:58:32 PDT
Comment on attachment 435252 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=435252&action=review

> Source/WebCore/platform/audio/cocoa/AudioFileReaderCocoa.cpp:590
> +            framesToRead = std::min<size_t>(std::numeric_limits<UInt32>::max(), framesLeftToRead);

Shouldn't this be `std::min<UInt32>`?

> Source/WebCore/platform/audio/cocoa/AudioFileReaderCocoa.cpp:599
> +            for (size_t i = 0; i < numberOfChannels; ++i) {
> +                bufferList->mBuffers[i].mDataByteSize = (numberOfFrames - framesRead) * sizeof(float);
> +                bufferList->mBuffers[i].mData = static_cast<float*>(bufferList->mBuffers[i].mData) + framesToRead;
> +            }

If we go through the `while` loop more than once, won't we end up with just the last chunk returned by ExtAudioFileRead?
Comment 6 Jean-Yves Avenard [:jya] 2021-08-10 16:18:08 PDT
Comment on attachment 435252 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=435252&action=review

>> Source/WebCore/platform/audio/cocoa/AudioFileReaderCocoa.cpp:590
>> +            framesToRead = std::min<size_t>(std::numeric_limits<UInt32>::max(), framesLeftToRead);
> 
> Shouldn't this be `std::min<UInt32>`?

framesLeftToRead is a size_t, doing std::min<UInt32< would cast framesLeftToRead to UInt32 before the comparison which could cause loss. So we make sure to never have more than UINT32_MAX

>> Source/WebCore/platform/audio/cocoa/AudioFileReaderCocoa.cpp:599
>> +            }
> 
> If we go through the `while` loop more than once, won't we end up with just the last chunk returned by ExtAudioFileRead?

bufferList is only an accessor to the underlying AudioBus memory buffer ; it's content is irrelevant once we exit the loop.
Comment 7 EWS 2021-08-11 17:10:33 PDT
/Volumes/Data/worker/Commit-Queue/build/LayoutTests/ChangeLog neither lists a valid reviewer nor contains the string "Unreviewed" or "Rubber stamp" (case insensitive).
Comment 8 Ryosuke Niwa 2021-08-11 17:13:30 PDT
Comment on attachment 435252 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=435252&action=review

> Source/WebCore/ChangeLog:11
> +        New test: webaudio/decode-audio-data-wav.html.

It should look like this instead:
Test: webaudio/decode-audio-data-wav.html

> LayoutTests/ChangeLog:6
> +

You're missing: Reviewed by NOBODY (OOPS!)
Comment 9 Jean-Yves Avenard [:jya] 2021-08-11 17:16:52 PDT
Created attachment 435388 [details]
Patch
Comment 10 EWS 2021-08-11 17:46:47 PDT
Committed r280948 (240456@main): <https://commits.webkit.org/240456@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 435388 [details].