Bug 211192 - USB microphone not recognized iOS Safari
Summary: USB microphone not recognized iOS Safari
Status: REOPENED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebRTC (show other bugs)
Version: Safari 13
Hardware: iPhone / iPad iOS 13
: P2 Blocker
Assignee: Eric Carlson
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2020-04-29 10:12 PDT by Marc Bakos
Modified: 2022-11-11 10:02 PST (History)
19 users (show)

See Also:


Attachments
Audio Tools (and other apps) discovers USB microphone connected to iOS, Safari does not. (166.87 KB, image/jpeg)
2020-04-29 10:12 PDT, Marc Bakos
no flags Details
Patch (29.27 KB, patch)
2020-11-10 08:30 PST, Eric Carlson
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch (30.12 KB, patch)
2020-11-10 10:12 PST, Eric Carlson
no flags Details | Formatted Diff | Diff
Patch (31.34 KB, patch)
2020-11-10 12:41 PST, Eric Carlson
no flags Details | Formatted Diff | Diff
Patch (30.04 KB, patch)
2020-11-10 15:57 PST, Eric Carlson
no flags Details | Formatted Diff | Diff
Patch for landing (30.70 KB, patch)
2020-11-11 08:47 PST, Eric Carlson
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Marc Bakos 2020-04-29 10:12:51 PDT
Created attachment 397975 [details]
Audio Tools (and other apps) discovers USB microphone connected to iOS, Safari does not.

Safari (iOS) doesn’t allow USB microphone devices through to any WebRTC enabled websites (getUserMedia). 
The Apple USB to lightning adapter is recognized by iOS, but is unable to perform with Safari. The USB microphone (in this case, a ATR2100); is detected on every other native app which enables microphones. 

The USB microphone is picked up in all native apps, attached example shows the microphone being correctly being discovered by iOS as described in an app called 'AudioTools'. However, with two WebRTC websites, (cleanfeed.net and WebRTC Samples); this microphone isn't being identified: https://webrtc.github.io/samples/src/content/devices/input-output/

Regards
Comment 1 Radar WebKit Bug Importer 2020-04-29 10:23:40 PDT
<rdar://problem/62607313>
Comment 2 Marc Bakos 2020-07-06 09:18:46 PDT
Hello, is there any further feedback on this report?
Comment 3 Bill 2020-08-31 10:32:06 PDT
Noticed this issue as well. Seems like a fairly big problem for certain types of apps. Any updates?
Comment 4 Eric Carlson 2020-11-10 08:30:30 PST
Created attachment 413698 [details]
Patch
Comment 5 youenn fablet 2020-11-10 08:53:46 PST
Comment on attachment 413698 [details]
Patch

Great to have the fix!

View in context: https://bugs.webkit.org/attachment.cgi?id=413698&action=review

> Source/WebCore/platform/mediastream/mac/CoreAudioCaptureDevice.h:52
> +    bool defaultDevice();

isDefaultDevice

> Source/WebCore/platform/mediastream/mac/CoreAudioCaptureDeviceManager.cpp:292
> +    if (haveDeviceChanges && notify == NotifyIfDevicesHaveChanged::Notify) {

Why not firing a device change event in case default changes?
This might be useful so that applications call again enumerateDevices and might notice the change.
Comment 6 Eric Carlson 2020-11-10 10:12:35 PST
Created attachment 413713 [details]
Patch
Comment 7 Eric Carlson 2020-11-10 12:41:44 PST
Created attachment 413725 [details]
Patch
Comment 8 Eric Carlson 2020-11-10 15:57:14 PST
Created attachment 413753 [details]
Patch
Comment 9 youenn fablet 2020-11-11 02:49:16 PST
Comment on attachment 413753 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=413753&action=review

> Source/WebCore/platform/mediastream/ios/AVAudioSessionCaptureDevice.mm:44
> +    setIsDefault(defaultInput && [defaultInput.UID isEqualToString:deviceInput.UID]);

Having two AVAudioSessionPortDescription* might be a bit confusing
Since we are only using defaultInput to check whether deviceInput is default, we could just pass isDefault value here or add a setter for this value that would be called later.

> Source/WebCore/platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.mm:52
> +    [[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(routeDidChange:) name:PAL::get_AVFoundation_AVAudioSessionRouteChangeNotification() object:session];

Is it always calling from the main thread?

> Source/WebCore/platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.mm:158
> +        LOG_ERROR("Failed to activate audio session with error: %@.", error.localizedDescription);

RELEASE_LOG?

> Source/WebCore/platform/mediastream/mac/CoreAudioCaptureDevice.cpp:70
> +        err = AudioObjectGetPropertyData(static_cast<UInt32>(deviceID), &address, 0, nullptr, &dataSize, &localizedName);

Logging maybe?

> Source/WebCore/platform/mediastream/mac/CoreAudioCaptureDevice.cpp:110
> +    setIsDefault(!err && defaultID == m_deviceID);

logging?

> Source/WebCore/platform/mediastream/mac/CoreAudioCaptureDeviceManager.cpp:154
> +                CoreAudioCaptureDeviceManager::singleton().scheduleUpdateCaptureDevices();

Sounds good to do this indeed.
The other changes of this method are refactoring and are not really changing any behavior, is that correct?

> Source/WebCore/platform/mediastream/mac/CoreAudioCaptureDeviceManager.cpp:267
> +            auto& newState = audioDevices[cptr];

s/oldState/oldDevice/
s/newState/newDevice/
Comment 10 Eric Carlson 2020-11-11 08:47:54 PST
Created attachment 413825 [details]
Patch for landing
Comment 11 Eric Carlson 2020-11-11 09:43:09 PST
Comment on attachment 413753 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=413753&action=review

>> Source/WebCore/platform/mediastream/ios/AVAudioSessionCaptureDevice.mm:44
>> +    setIsDefault(defaultInput && [defaultInput.UID isEqualToString:deviceInput.UID]);
> 
> Having two AVAudioSessionPortDescription* might be a bit confusing
> Since we are only using defaultInput to check whether deviceInput is default, we could just pass isDefault value here or add a setter for this value that would be called later.

It might be slightly confusing, but it seems better to me to keep the logic for what makes an input default in the class that represents the input.

>> Source/WebCore/platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.mm:52
>> +    [[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(routeDidChange:) name:PAL::get_AVFoundation_AVAudioSessionRouteChangeNotification() object:session];
> 
> Is it always calling from the main thread?

Good point, I don't know if it is or not so I'll add a check.

>> Source/WebCore/platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.mm:158
>> +        LOG_ERROR("Failed to activate audio session with error: %@.", error.localizedDescription);
> 
> RELEASE_LOG?

Changed.

>> Source/WebCore/platform/mediastream/mac/CoreAudioCaptureDevice.cpp:110
>> +    setIsDefault(!err && defaultID == m_deviceID);
> 
> logging?

Fixed.

>> Source/WebCore/platform/mediastream/mac/CoreAudioCaptureDeviceManager.cpp:154
>> +                CoreAudioCaptureDeviceManager::singleton().scheduleUpdateCaptureDevices();
> 
> Sounds good to do this indeed.
> The other changes of this method are refactoring and are not really changing any behavior, is that correct?

Correct, the other changes are just minor cleanup.

>> Source/WebCore/platform/mediastream/mac/CoreAudioCaptureDeviceManager.cpp:267
>> +            auto& newState = audioDevices[cptr];
> 
> s/oldState/oldDevice/
> s/newState/newDevice/

Fixed
Comment 12 EWS 2020-11-11 10:07:09 PST
Committed r269688: <https://trac.webkit.org/changeset/269688>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 413825 [details].
Comment 13 Bill 2021-01-13 20:25:12 PST
Thanks for fixing this. Do you know what version of iOS will have the change?
Comment 14 Bill 2021-02-24 20:15:30 PST
Hey everyone, I'm still seeing this issue on iPad OS 14.5 beta. Should that version have the fix? It seems like this was merged awhile ago.
Comment 15 Marc Bakos 2021-04-07 08:13:28 PDT
As per my email to Eric, I upgraded to 14.5 beta 6 on my spare iPhone XS.

Unfortunately I’m still not seeing the USB device enumerated. 

I’m using a Lighting cable to USB adapter (official Apple) and have an Audio Technica ATR2100-USB plugged in. The device works in other apps; just not Safari. 

Both my test on Cleanfeed and WebRTC example pages (https://webrtc.github.io/samples/src/content/devices/input-output) do not display the input device.

I've reopened the ticket for confirmation.
Comment 16 Marc Bakos 2021-08-12 03:32:20 PDT
Tested on iOS 15 Beta (build 19A5318f); Safari is able to enumerate the device correctly; however upon selecting it, it doesn't open the device. 

In fact it just reverts the input back to the built in microphone of the device, despite its selection in the device selection menu.

If you'd like some more information, please let me know. It seems incredibly close!
Comment 17 Eric Carlson 2021-08-12 08:44:48 PDT
(In reply to Marc Bakos from comment #16)
> Tested on iOS 15 Beta (build 19A5318f); Safari is able to enumerate the
> device correctly; however upon selecting it, it doesn't open the device. 
> 
> In fact it just reverts the input back to the built in microphone of the
> device, despite its selection in the device selection menu.
> 
> If you'd like some more information, please let me know. It seems incredibly
> close!

r280702 (bug 228753) should fix the inability to select some devices.
Comment 18 Marc Bakos 2021-09-02 05:06:06 PDT
Thanks Eric. I have reviewed the latest iOS 15 Beta 8 build 19A5340a on iPhone Xs and I now see the USB device listed as 'Advanced USB Audio Device', though it is still not being used.

I'm guessing this is just a case of waiting for the patch 280702 to land in a build to correct this? I'll keep my eyes peeled for Beta 9, or the gold seed.
Comment 19 tim.proegler 2021-10-07 08:28:13 PDT
Dear all,

chiming in on this.

Has this change landed in either iOS 15 release version or 15.1 beta 3?

I'm not able to access an external audio interface in these version using an iPhone 6s. The interface is listed correctly but still the built-in mic is used, no matter if I select the external audio interface or not.

Tested with https://webrtc.github.io/samples/src/content/devices/input-output/ and cleanfeed.

Any help would be appreciated!


Regards
Tim
Comment 20 Marc Bakos 2022-01-23 13:18:19 PST
Hi all,

iOS 15.2.1 has been tested, with still no resolve. 
The USB device is enumerated, but still cannot be used. 

I made a video to demonstrate the issue: 
https://www.youtube.com/watch?v=V5BobU8vaBg

I can confirm that the correct device ID is selected, but the the device is still not opened correctly. It continues using the audio from the built in microphone.

If there is further information I can give, please let me know.

Thanks
Marc
Comment 21 Antti Laitinen 2022-02-22 04:30:33 PST
Hello everyone!

We're having this same issue in Soundtrap's iOS client.
You can't use an iRig for example. It's detected as an input device, but if you try to use it, then the incoming audio will come from the built-in mic.

getUserMedia() returns a MediaStream which has iRig in its label, but the audio still comes from the built-in mic.

Here's a simple piece of code which should be able to reproduce the problem: https://codepen.io/laiskimus/pen/WNXJYGb

That works in desktop browsers, but in mobile Safari you always get audio from the built-in mic.
Comment 22 Marc Bakos 2022-11-11 10:02:34 PST
Hello. I see there has been a lot of input to this ticket, though I'm afraid the user experience has recently got worse in recent iOS releases. 

For example, USB audio devices connected to iOS devices now don't even output audio from Safari. 

It is still not possible to correctly use the enumerated device.

Could someone kindly update this ticket as to whether it is possible to iron out these issues and finally get this bug resolved?