Bug 240692 - REGRESSION (r293829): [ iOS ] TestWebKitAPI.GPUProcess.WebProcessTerminationAfterTooManyGPUProcessCrashes is a flaky crash
Summary: REGRESSION (r293829): [ iOS ] TestWebKitAPI.GPUProcess.WebProcessTerminationA...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Karl Rackler
URL:
Keywords: InRadar
Depends on: 241415 241449
Blocks:
  Show dependency treegraph
 
Reported: 2022-05-19 16:15 PDT by Karl Rackler
Modified: 2022-06-08 21:39 PDT (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Karl Rackler 2022-05-19 16:15:52 PDT
Description:
TestWebKitAPI.GPUProcess.WebProcessTerminationAfterTooManyGPUProcessCrashes

The first failure that I saw on the dashboard was on 9/22/2021 at r282903, but this does not appear relevant to causing this issue.  This test has been flaky for its entire history on the dashboard.

History:
https://results.webkit.org/?platform=ios&limit=50000&suite=api-tests&test=TestWebKitAPI.GPUProcess.WebProcessTerminationAfterTooManyGPUProcessCrashes

Crash Log:
Child process terminated with signal 9: Killed
Comment 1 Radar WebKit Bug Importer 2022-05-19 16:16:09 PDT
<rdar://problem/93611089>
Comment 2 Karl Rackler 2022-05-19 17:01:31 PDT
REPRODUCTION STEPS
I can reproduce this on r294506 ToT.

Command: 
run-api-tests --root 294506 --ios-simulator —release --iterations 2000 TestWebKitAPI.GPUProcess.WebProcessTerminationAfterTooManyGPUProcessCrashes

Result: 
Child process terminated with signal 9: Killed
worker/0 TestWebKitAPI.GPUProcess.WebProcessTerminationAfterTooManyGPUProcessCrashes Crashed
LEAK: 1 WebProcessPool
LEAK: 1 WebPageProxy

ASSERTION FAILED: !result
/Volumes/Data/build/OpenSource/Source/WebCore/platform/audio/ios/AudioOutputUnitAdaptorIOS.cpp(70) : void WebCore::AudioOutputUnitAdaptor::configure(float, unsigned int)
1   0x1289d3129 WTFCrash
2   0x1905b9dcb WTFCrashWithInfo(int, char const*, char const*, int)
3   0x190666b07 WebCore::AudioOutputUnitAdaptor::configure(float, unsigned int)
4   0x14247da83 WebKit::RemoteAudioDestination::RemoteAudioDestination(WebKit::GPUConnectionToWebProcess&, WTF::ObjectIdentifier<WebKit::RemoteAudioDestinationIdentifierType>, WTF::String const&, unsigned int, unsigned int, float, float, IPC::Semaphore&&)
Comment 3 Karl Rackler 2022-06-06 15:34:35 PDT
Full Stack: 
ASSERTION FAILED: !result
/Volumes/Data/worker/trunk-OpenSource-ios-simulator-debug-archive/build/OpenSource/Source/WebCore/platform/audio/ios/AudioOutputUnitAdaptorIOS.cpp(70) : void WebCore::AudioOutputUnitAdaptor::configure(float, unsigned int)
1   0x1289d3129 WTFCrash
2   0x1905b9dcb WTFCrashWithInfo(int, char const*, char const*, int)
3   0x190666b07 WebCore::AudioOutputUnitAdaptor::configure(float, unsigned int)
4   0x14247da83 WebKit::RemoteAudioDestination::RemoteAudioDestination(WebKit::GPUConnectionToWebProcess&, WTF::ObjectIdentifier<WebKit::RemoteAudioDestinationIdentifierType>, WTF::String const&, unsigned int, unsigned int, float, float, IPC::Semaphore&&)
5   0x14247d92d WebKit::RemoteAudioDestination::RemoteAudioDestination(WebKit::GPUConnectionToWebProcess&, WTF::ObjectIdentifier<WebKit::RemoteAudioDestinationIdentifierType>, WTF::String const&, unsigned int, unsigned int, float, float, IPC::Semaphore&&)
6   0x14247d866 WTF::UniqueRef<WebKit::RemoteAudioDestination> WTF::makeUniqueRefWithoutFastMallocCheck<WebKit::RemoteAudioDestination, WebKit::GPUConnectionToWebProcess&, WTF::ObjectIdentifier<WebKit::RemoteAudioDestinationIdentifierType>&, WTF::String const&, unsigned int&, unsigned int&, float&, float&, IPC::Semaphore>(WebKit::GPUConnectionToWebProcess&, WTF::ObjectIdentifier<WebKit::RemoteAudioDestinationIdentifierType>&, WTF::String const&, unsigned int&, unsigned int&, float&, float&, IPC::Semaphore&&)
7   0x14247b211 WTF::UniqueRef<WebKit::RemoteAudioDestination> WTF::makeUniqueRef<WebKit::RemoteAudioDestination, WebKit::GPUConnectionToWebProcess&, WTF::ObjectIdentifier<WebKit::RemoteAudioDestinationIdentifierType>&, WTF::String const&, unsigned int&, unsigned int&, float&, float&, IPC::Semaphore>(WebKit::GPUConnectionToWebProcess&, WTF::ObjectIdentifier<WebKit::RemoteAudioDestinationIdentifierType>&, WTF::String const&, unsigned int&, unsigned int&, float&, float&, IPC::Semaphore&&)
8   0x14247afa9 WebKit::RemoteAudioDestinationManager::createAudioDestination(WTF::String const&, unsigned int, unsigned int, float, float, IPC::Semaphore&&, WTF::CompletionHandler<void (WTF::ObjectIdentifier<WebKit::RemoteAudioDestinationIdentifierType>)>&&)
9   0x142ae29d8 void IPC::callMemberFunctionImpl<WebKit::RemoteAudioDestinationManager, void (WebKit::RemoteAudioDestinationManager::*)(WTF::String const&, unsigned int, unsigned int, float, float, IPC::Semaphore&&, WTF::CompletionHandler<void (WTF::ObjectIdentifier<WebKit::RemoteAudioDestinationIdentifierType>)>&&), void (WTF::ObjectIdentifier<WebKit::RemoteAudioDestinationIdentifierType>), std::__1::tuple<WTF::String, unsigned int, unsigned int, float, float, IPC::Semaphore>, 0ul, 1ul, 2ul, 3ul, 4ul, 5ul>(WebKit::RemoteAudioDestinationManager*, void (WebKit::RemoteAudioDestinationManager::*)(WTF::String const&, unsigned int, unsigned int, float, float, IPC::Semaphore&&, WTF::CompletionHandler<void (WTF::ObjectIdentifier<WebKit::RemoteAudioDestinationIdentifierType>)>&&), WTF::CompletionHandler<void (WTF::ObjectIdentifier<WebKit::RemoteAudioDestinationIdentifierType>)>&&, std::__1::tuple<WTF::String, unsigned int, unsigned int, float, float, IPC::Semaphore>&&, std::__1::integer_sequence<unsigned long, 0ul, 1ul, 2ul, 3ul, 4ul, 5ul>)
10  0x142ade4f2 void IPC::callMemberFunction<WebKit::RemoteAudioDestinationManager, void (WebKit::RemoteAudioDestinationManager::*)(WTF::String const&, unsigned int, unsigned int, float, float, IPC::Semaphore&&, WTF::CompletionHandler<void (WTF::ObjectIdentifier<WebKit::RemoteAudioDestinationIdentifierType>)>&&), void (WTF::ObjectIdentifier<WebKit::RemoteAudioDestinationIdentifierType>), std::__1::tuple<WTF::String, unsigned int, unsigned int, float, float, IPC::Semaphore>, std::__1::integer_sequence<unsigned long, 0ul, 1ul, 2ul, 3ul, 4ul, 5ul> >(std::__1::tuple<WTF::String, unsigned int, unsigned int, float, float, IPC::Semaphore>&&, WTF::CompletionHandler<void (WTF::ObjectIdentifier<WebKit::RemoteAudioDestinationIdentifierType>)>&&, WebKit::RemoteAudioDestinationManager*, void (WebKit::RemoteAudioDestinationManager::*)(WTF::String const&, unsigned int, unsigned int, float, float, IPC::Semaphore&&, WTF::CompletionHandler<void (WTF::ObjectIdentifier<WebKit::RemoteAudioDestinationIdentifierType>)>&&))
11  0x142a96471 bool IPC::handleMessageSynchronous<Messages::RemoteAudioDestinationManager::CreateAudioDestination, WebKit::RemoteAudioDestinationManager, void (WebKit::RemoteAudioDestinationManager::*)(WTF::String const&, unsigned int, unsigned int, float, float, IPC::Semaphore&&, WTF::CompletionHandler<void (WTF::ObjectIdentifier<WebKit::RemoteAudioDestinationIdentifierType>)>&&)>(IPC::Connection&, IPC::Decoder&, WTF::UniqueRef<IPC::Encoder>&, WebKit::RemoteAudioDestinationManager*, void (WebKit::RemoteAudioDestinationManager::*)(WTF::String const&, unsigned int, unsigned int, float, float, IPC::Semaphore&&, WTF::CompletionHandler<void (WTF::ObjectIdentifier<WebKit::RemoteAudioDestinationIdentifierType>)>&&))
12  0x142a96232 WebKit::RemoteAudioDestinationManager::didReceiveSyncMessage(IPC::Connection&, IPC::Decoder&, WTF::UniqueRef<IPC::Encoder>&)
13  0x142bab68e WebKit::RemoteAudioDestinationManager::didReceiveSyncMessageFromWebProcess(IPC::Connection&, IPC::Decoder&, WTF::UniqueRef<IPC::Encoder>&)
14  0x142bab2cb WebKit::GPUConnectionToWebProcess::dispatchSyncMessage(IPC::Connection&, IPC::Decoder&, WTF::UniqueRef<IPC::Encoder>&)
15  0x1429f04d1 WebKit::GPUConnectionToWebProcess::didReceiveSyncMessage(IPC::Connection&, IPC::Decoder&, WTF::UniqueRef<IPC::Encoder>&)
16  0x1433bbd6a IPC::Connection::dispatchSyncMessage(IPC::Decoder&)
17  0x1433bc96b IPC::Connection::dispatchMessage(std::__1::unique_ptr<IPC::Decoder, std::__1::default_delete<IPC::Decoder> >)
18  0x1433b5e03 IPC::Connection::SyncMessageState::ConnectionAndIncomingMessage::dispatch()
19  0x1433b6098 IPC::Connection::SyncMessageState::dispatchMessagesAndResetDidScheduleDispatchMessagesForConnection(IPC::Connection&)
20  0x1433ce29c IPC::Connection::SyncMessageState::processIncomingMessage(IPC::Connection&, std::__1::unique_ptr<IPC::Decoder, std::__1::default_delete<IPC::Decoder> >&)::$_5::operator()()
21  0x1433ce1c9 WTF::Detail::CallableWrapper<IPC::Connection::SyncMessageState::processIncomingMessage(IPC::Connection&, std::__1::unique_ptr<IPC::Decoder, std::__1::default_delete<IPC::Decoder> >&)::$_5, void>::call()
22  0x1289ff002 WTF::Function<void ()>::operator()() const
23  0x128a8e072 WTF::RunLoop::performWork()
24  0x128a9199e WTF::RunLoop::performWork(void*)
25  0x10935a833 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__
26  0x10935a72b __CFRunLoopDoSource0
27  0x109359bf8 __CFRunLoopDoSources0
28  0x1093542f4 __CFRunLoopRun
29  0x109353a90 CFRunLoopRunSpecific
30  0x109ca3e31 -[NSRunLoop(NSRunLoop) runMode:beforeDate:]
31  0x109ca404f -[NSRunLoop(NSRunLoop) run]
LEAK: 1 WebProcessPool

/Volumes/Data/worker/trunk-OpenSource-ios-simulator-debug-archive/build/OpenSource/Tools/TestWebKitAPI/Tests/WebKitCocoa/GPUProcess.mm:217
Value of: [webView _isPlayingAudio]
  Actual: false
Expected: true

LEAK: 1 WebPageProxy
worker/0 TestWebKitAPI.GPUProcess.WebProcessTerminationAfterTooManyGPUProcessCrashes Failed
Comment 4 Karl Rackler 2022-06-06 16:49:49 PDT
REPRODUCTION STEPS
I can reproduce this on r293829 (250302@Main) but cannot reproduce it on r293828.

Command: 
run-api-tests --ios-simulator —debug --iterations 10 

Result: 
worker/0 TestWebKitAPI.GPUProcess.WebProcessTerminationAfterTooManyGPUProcessCrashes Crashed
ASSERTION FAILED: m_sendPort
/Volumes/Data/worker/trunk-OpenSource-ios-simulator-debug-archive/build/OpenSource/Source/WebKit/Platform/IPC/cocoa/ConnectionCocoa.mm(200) : bool IPC::Connection::open()
1   0x113d751e9 WTFCrash
2   0x13cb3af6b WTFCrashWithInfo(int, char const*, char const*, int)
3   0x13d5fa40c IPC::Connection::open()
4   0x13d7f9f2f WebKit::GPUConnectionToWebProcess::GPUConnectionToWebProcess(WebKit::GPUProcess&, WTF::ObjectIdentifier<WebCore::ProcessIdentifierType>, PAL::SessionID, IPC::Connection::Identifier&&, WebKit::GPUProcessConnectionParameters&&)
5   0x13d7f9acd WebKit::GPUConnectionToWebProcess::GPUConnectionToWebProcess(WebKit::GPUProcess&, WTF::ObjectIdentifier<WebCore::ProcessIdentifierType>, PAL::SessionID, IPC::Connection::Identifier&&, WebKit::GPUProcessConnectionParameters&&)
6   0x13d7f9a03 WebKit::GPUConnectionToWebProcess::create(WebKit::GPUProcess&, WTF::ObjectIdentifier<WebCore::ProcessIdentifierType>, PAL::SessionID, IPC::Connection::Identifier&&, WebKit::GPUProcessConnectionParameters&&)
7   0x13d80089d WebKit::GPUProcess::createGPUConnectionToWebProcess(WTF::ObjectIdentifier<WebCore::ProcessIdentifierType>, PAL::SessionID, IPC::Attachment&&, WebKit::GPUProcessConnectionParameters&&, WTF::CompletionHandler<void ()>&&)
8   0x13d6f1d20 void IPC::callMemberFunctionImpl<WebKit::GPUProcess, void (WebKit::GPUProcess::*)(WTF::ObjectIdentifier<WebCore::ProcessIdentifierType>, PAL::SessionID, IPC::Attachment&&, WebKit::GPUProcessConnectionParameters&&, WTF::CompletionHandler<void ()>&&), void (), std::__1::tuple<WTF::ObjectIdentifier<WebCore::ProcessIdentifierType>, PAL::SessionID, IPC::Attachment, WebKit::GPUProcessConnectionParameters>, 0ul, 1ul, 2ul, 3ul>(WebKit::GPUProcess*, void (WebKit::GPUProcess::*)(WTF::ObjectIdentifier<WebCore::ProcessIdentifierType>, PAL::SessionID, IPC::Attachment&&, WebKit::GPUProcessConnectionParameters&&, WTF::CompletionHandler<void ()>&&), WTF::CompletionHandler<void ()>&&, std::__1::tuple<WTF::ObjectIdentifier<WebCore::ProcessIdentifierType>, PAL::SessionID, IPC::Attachment, WebKit::GPUProcessConnectionParameters>&&, std::__1::integer_sequence<unsigned long, 0ul, 1ul, 2ul, 3ul>)
9   0x13d6ed762 void IPC::callMemberFunction<WebKit::GPUProcess, void (WebKit::GPUProcess::*)(WTF::ObjectIdentifier<WebCore::ProcessIdentifierType>, PAL::SessionID, IPC::Attachment&&, WebKit::GPUProcessConnectionParameters&&, WTF::CompletionHandler<void ()>&&), void (), std::__1::tuple<WTF::ObjectIdentifier<WebCore::ProcessIdentifierType>, PAL::SessionID, IPC::Attachment, WebKit::GPUProcessConnectionParameters>, std::__1::integer_sequence<unsigned long, 0ul, 1ul, 2ul, 3ul> >(std::__1::tuple<WTF::ObjectIdentifier<WebCore::ProcessIdentifierType>, PAL::SessionID, IPC::Attachment, WebKit::GPUProcessConnectionParameters>&&, WTF::CompletionHandler<void ()>&&, WebKit::GPUProcess*, void (WebKit::GPUProcess::*)(WTF::ObjectIdentifier<WebCore::ProcessIdentifierType>, PAL::SessionID, IPC::Attachment&&, WebKit::GPUProcessConnectionParameters&&, WTF::CompletionHandler<void ()>&&))
10  0x13d6bc994 void IPC::handleMessageAsync<Messages::GPUProcess::CreateGPUConnectionToWebProcess, WebKit::GPUProcess, void (WebKit::GPUProcess::*)(WTF::ObjectIdentifier<WebCore::ProcessIdentifierType>, PAL::SessionID, IPC::Attachment&&, WebKit::GPUProcessConnectionParameters&&, WTF::CompletionHandler<void ()>&&)>(IPC::Connection&, IPC::Decoder&, WebKit::GPUProcess*, void (WebKit::GPUProcess::*)(WTF::ObjectIdentifier<WebCore::ProcessIdentifierType>, PAL::SessionID, IPC::Attachment&&, WebKit::GPUProcessConnectionParameters&&, WTF::CompletionHandler<void ()>&&))
11  0x13d6bbcf5 WebKit::GPUProcess::didReceiveGPUProcessMessage(IPC::Connection&, IPC::Decoder&)
12  0x13d800728 WebKit::GPUProcess::didReceiveMessage(IPC::Connection&, IPC::Decoder&)
13  0x13e00d2d0 IPC::Connection::dispatchMessage(IPC::Decoder&)
14  0x13e00d9c5 IPC::Connection::dispatchMessage(std::__1::unique_ptr<IPC::Decoder, std::__1::default_delete<IPC::Decoder> >)
15  0x13e006e43 IPC::Connection::SyncMessageState::ConnectionAndIncomingMessage::dispatch()
16  0x13e0070d8 IPC::Connection::SyncMessageState::dispatchMessagesAndResetDidScheduleDispatchMessagesForConnection(IPC::Connection&)
17  0x13e01f2dc IPC::Connection::SyncMessageState::processIncomingMessage(IPC::Connection&, std::__1::unique_ptr<IPC::Decoder, std::__1::default_delete<IPC::Decoder> >&)::$_5::operator()()
18  0x13e01f209 WTF::Detail::CallableWrapper<IPC::Connection::SyncMessageState::processIncomingMessage(IPC::Connection&, std::__1::unique_ptr<IPC::Decoder, std::__1::default_delete<IPC::Decoder> >&)::$_5, void>::call()
19  0x113da0d22 WTF::Function<void ()>::operator()() const
20  0x113e2fbf2 WTF::RunLoop::performWork()
21  0x113e3351e WTF::RunLoop::performWork(void*)
22  0x10bd9e833 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__
23  0x10bd9e72b __CFRunLoopDoSource0
24  0x10bd9dbf8 __CFRunLoopDoSources0
25  0x10bd982f4 __CFRunLoopRun
26  0x10bd97a90 CFRunLoopRunSpecific
27  0x10c6e7e31 -[NSRunLoop(NSRunLoop) runMode:beforeDate:]
28  0x10c6e804f -[NSRunLoop(NSRunLoop) run]
29  0x10d1f5feb _xpc_objc_main
30  0x10d1f7fd4 xpc_main
31  0x13d89de5a WebKit::XPCServiceMain(int, char const**)
LEAK: 1 WebProcessPool
LEAK: 1 WebPageProxy
Comment 5 Karl Rackler 2022-06-08 15:59:56 PDT
I have disabled this test for iOS while this issue is investigated.
Comment 6 Karl Rackler 2022-06-08 16:03:20 PDT
Pull request: https://github.com/WebKit/WebKit/pull/1396
Comment 7 EWS 2022-06-08 17:37:25 PDT
Test gardening commit r295404 (251410@main): <https://commits.webkit.org/251410@main>

Reviewed commits have been landed. Closing PR #1396 and removing active labels.
Comment 8 Simon Fraser (smfr) 2022-06-08 21:39:40 PDT
I filed bug 241449 for the AudioOutputUnitAdaptor crashes.