Bug 211720 - Allow WebAudioBufferList to dynamically change its number of frames
Summary: Allow WebAudioBufferList to dynamically change its number of frames
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebRTC (show other bugs)
Version: WebKit Local Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: youenn fablet
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2020-05-11 04:22 PDT by youenn fablet
Modified: 2020-05-13 11:02 PDT (History)
11 users (show)

See Also:


Attachments
Patch (12.83 KB, patch)
2020-05-11 06:17 PDT, youenn fablet
no flags Details | Formatted Diff | Diff
Patch (13.02 KB, patch)
2020-05-11 10:50 PDT, youenn fablet
no flags Details | Formatted Diff | Diff
Patch (13.02 KB, patch)
2020-05-11 11:18 PDT, youenn fablet
no flags Details | Formatted Diff | Diff
Patch for landing (14.25 KB, patch)
2020-05-13 10:34 PDT, youenn fablet
no flags Details | Formatted Diff | Diff
Patch for landing (14.25 KB, patch)
2020-05-13 10:35 PDT, youenn fablet
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description youenn fablet 2020-05-11 04:22:06 PDT
Allow WebAudioBufferList to dynamically change its number of frames
Comment 1 youenn fablet 2020-05-11 06:17:48 PDT
Created attachment 399014 [details]
Patch
Comment 2 Eric Carlson 2020-05-11 09:36:18 PDT
Comment on attachment 399014 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=399014&action=review

> Source/WebCore/Modules/webaudio/MediaStreamAudioSource.cpp:36
> +#if USE(AVFOUNDATION)
> +#include "WebAudioBufferList.h"
> +#include <CoreAudio/CoreAudioTypes.h>
> +#endif

Do we really need to include these here in the cross platform file?

> Source/WebCore/Modules/webaudio/MediaStreamAudioSource.h:65
>  #if USE(AVFOUNDATION)
> +    std::unique_ptr<WebAudioBufferList> m_audioBuffer;
>      size_t m_numberOfFrames { 0 };
>  #endif

This seems like a mistake. It's probably time to actually subclass for Cocoa rather than just implementing parts of the class in MediaStreamAudioSourceCocoa.cpp

> Source/WebCore/platform/audio/cocoa/WebAudioBufferList.cpp:71
> +    uint32_t bufferCount = m_canonicalList->mNumberBuffers;
> +    if (!bufferCount)
> +        return;
>  
> -    size_t bytesPerBuffer = sampleCount * channelCount * format.bytesPerFrame();
> -    m_flatBuffer.reserveInitialCapacity(bufferCount * bytesPerBuffer);
> +    size_t bytesPerBuffer = sampleCount * m_channelCount * m_bytesPerFrame;
> +    m_flatBuffer.reserveCapacity(bufferCount * bytesPerBuffer);
>      auto data = m_flatBuffer.data();

Can this return early if nothing has changed so we don't call reset() needlessly?
Comment 3 youenn fablet 2020-05-11 10:50:18 PDT
Created attachment 399027 [details]
Patch
Comment 4 youenn fablet 2020-05-11 10:51:45 PDT
(In reply to Eric Carlson from comment #2)
> Comment on attachment 399014 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=399014&action=review
> 
> > Source/WebCore/Modules/webaudio/MediaStreamAudioSource.cpp:36
> > +#if USE(AVFOUNDATION)
> > +#include "WebAudioBufferList.h"
> > +#include <CoreAudio/CoreAudioTypes.h>
> > +#endif
> 
> Do we really need to include these here in the cross platform file?

Fixed it by using PlatformAudioData.

> > Source/WebCore/Modules/webaudio/MediaStreamAudioSource.h:65
> >  #if USE(AVFOUNDATION)
> > +    std::unique_ptr<WebAudioBufferList> m_audioBuffer;
> >      size_t m_numberOfFrames { 0 };
> >  #endif
> 
> This seems like a mistake. It's probably time to actually subclass for Cocoa
> rather than just implementing parts of the class in
> MediaStreamAudioSourceCocoa.cpp

I made a try at keeping it like it is but, yes maybe we should have a cocoa specialisation.

> > Source/WebCore/platform/audio/cocoa/WebAudioBufferList.cpp:71
> > +    uint32_t bufferCount = m_canonicalList->mNumberBuffers;
> > +    if (!bufferCount)
> > +        return;
> >  
> > -    size_t bytesPerBuffer = sampleCount * channelCount * format.bytesPerFrame();
> > -    m_flatBuffer.reserveInitialCapacity(bufferCount * bytesPerBuffer);
> > +    size_t bytesPerBuffer = sampleCount * m_channelCount * m_bytesPerFrame;
> > +    m_flatBuffer.reserveCapacity(bufferCount * bytesPerBuffer);
> >      auto data = m_flatBuffer.data();
> 
> Can this return early if nothing has changed so we don't call reset()
> needlessly?

Added an if check.
Comment 5 youenn fablet 2020-05-11 11:18:35 PDT
Created attachment 399033 [details]
Patch
Comment 6 EWS 2020-05-12 09:45:36 PDT
Committed r261557: <https://trac.webkit.org/changeset/261557>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 399033 [details].
Comment 7 Radar WebKit Bug Importer 2020-05-12 09:46:16 PDT
<rdar://problem/63140841>
Comment 8 Jacob Uphoff 2020-05-12 11:23:43 PDT
Reverted r261557 for reason:

This commit caused testing to exit early due to too many crashes on macOS Catalina Asan

Committed r261566: <https://trac.webkit.org/changeset/261566>
Comment 9 youenn fablet 2020-05-13 10:24:45 PDT
ASAN issue is a false positive.
We preallocate a Vector with reserveCapacity and write within the bounds of the allocated buffer, but past the end of the Vector (which is equal to begin) since the vector size is zero.
We cannot use reserveCapacity here since it is calling asanSetInitialBufferSizeTo.
Comment 10 youenn fablet 2020-05-13 10:34:47 PDT
Created attachment 399277 [details]
Patch for landing
Comment 11 youenn fablet 2020-05-13 10:35:46 PDT
Created attachment 399278 [details]
Patch for landing
Comment 12 youenn fablet 2020-05-13 10:38:39 PDT
> We cannot use reserveCapacity here since it is calling
> asanSetInitialBufferSizeTo.

I switched to resize() which is not that much expensive and will make sure we can write on the whole allocated buffer.
Comment 13 EWS 2020-05-13 11:02:13 PDT
Committed r261626: <https://trac.webkit.org/changeset/261626>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 399278 [details].