Bug 222277 – [ macOS Wk2 ] media/media-fragments/TC0051.html is flakey crashing

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Robert Jenner]

media/media-fragments/TC0051.html

HISTORY URL:
https://results.webkit.org/?suite=layout-tests&test=media%2Fmedia-fragments%2FTC0051.html&platform=mac

CRASH LOG URL:
https://build.webkit.org/results/Apple-BigSur-Release-WK2-Tests/r273239%20(409)/media/media-fragments/TC0051-crash-log.txt

CRAHS LOG TEXT:
No crash log found for GPUProcess:41578.

stdout:

stderr:
2021-02-22 05:16:22.066 com.apple.WebKit.GPU.Development[41578:82907104] *** Terminating app due to uncaught exception 'NSRangeException', reason: '*** -[__NSArrayM objectAtIndex:]: index 3 beyond bounds [0 .. 2]'
*** First throw call stack:
(
	0   CoreFoundation                      0x00007fff204d46af __exceptionPreprocess + 242
	1   libobjc.A.dylib                     0x00007fff2020c3c9 objc_exception_throw + 48
	2   CoreFoundation                      0x00007fff20588a9a -[__NSCFString characterAtIndex:].cold.1 + 0
	3   CoreFoundation                      0x00007fff203fb163 -[NSTaggedPointerString hash] + 0
	4   Photos                              0x00007fff3e627453 +[AVAssetCollection(Utilities) fragmentParamsFromURL:] + 219
	5   Photos                              0x00007fff3e6272c6 +[AVAssetCollection(Utilities) isURLForAssetInCollection:] + 102
	6   AVFCore                             0x00007fff3167e1b0 -[AVURLAsset initWithURL:options:] + 136
	7   WebCore                             0x000000010545df34 _ZN7WebCore34MediaPlayerPrivateAVFoundationObjC19createAVAssetForURLERKN3WTF3URLENS1_9RetainPtrI19NSMutableDictionaryEE + 2724
	8   WebCore                             0x000000010545d447 _ZN7WebCore34MediaPlayerPrivateAVFoundationObjC19createAVAssetForURLERKN3WTF3URLE + 87
	9   WebCore                             0x0000000105451bd4 _ZN7WebCore30MediaPlayerPrivateAVFoundation10setPreloadENS_16MediaPlayerEnums7PreloadE + 212
	10  WebCore                             0x0000000105450349 _ZN7WebCore30MediaPlayerPrivateAVFoundation4loadERKN3WTF6StringE + 457
	11  WebCore                             0x0000000106afa8e3 _ZN7WebCore11MediaPlayer23loadWithNextMediaEngineEPKNS_18MediaPlayerFactoryE + 691
	12  WebCore                             0x0000000106afa545 _ZN7WebCore11MediaPlayer4loadERKN3WTF3URLERKNS_11ContentTypeERKNS1_6StringE + 1269
	13  WebKit                              0x00000001031d4129 _ZN6WebKit22RemoteMediaPlayerProxy4loadEON3WTF3URLEONS1_8OptionalINS_16SandboxExtension6HandleEEERKN7WebCore11ContentTypeERKNS1_6StringEONS1_17CompletionHandlerIFvONS_30RemoteMediaPlayerConfigurationEEEE + 185
	14  WebKit                              0x00000001031521c0 _ZN3IPC22callMemberFunctionImplIN6WebKit22RemoteMediaPlayerProxyEMS2_FvON3WTF3URLEONS3_8OptionalINS1_16SandboxExtension6HandleEEERKN7WebCore11ContentTypeERKNS3_6StringEONS3_17CompletionHandlerIFvONS1_30RemoteMediaPlayerConfigurationEEEEEFvRKSJ_ENSt3__15tupleIJS4_S9_SC_SF_EEEJLm0ELm1ELm2ELm3EEEEvPT_T0_ONSI_IT1_EEOT2_NST_16integer_sequenceImJXspT3_EEEE + 110
	15  WebKit                              0x0000000103150d4f _ZN3IPC18handleMessageAsyncIN8Messages22RemoteMediaPlayerProxy4LoadEN6WebKit22RemoteMediaPlayerProxyEMS5_FvON3WTF3URLEONS6_8OptionalINS4_16SandboxExtension6HandleEEERKN7WebCore11ContentTypeERKNS6_6StringEONS6_17CompletionHandlerIFvONS4_30RemoteMediaPlayerConfigurationEEEEEEEvRNS_10ConnectionERNS_7DecoderEPT0_T1_ + 196
	16  WebKit                              0x00000001031d38bc _ZN6WebKit29RemoteMediaPlayerManagerProxy23didReceivePlayerMessageERN3IPC10ConnectionERNS1_7DecoderE + 56
	17  WebKit                              0x00000001031b78ad _ZN6WebKit25GPUConnectionToWebProcess15dispatchMessageERN3IPC10ConnectionERNS1_7DecoderE + 121
	18  WebKit                              0x00000001031752e6 _ZN6WebKit25GPUConnectionToWebProcess17didReceiveMessageERN3IPC10ConnectionERNS1_7DecoderE + 434
	19  WebKit                              0x000000010309dd61 _ZN3IPC10Connection15dispatchMessageENSt3__110unique_ptrINS_7DecoderENS1_14default_deleteIS3_EEEE + 221
	20  WebKit                              0x000000010309dfb5 _ZN3IPC10Connection26dispatchOneIncomingMessageEv + 169
	21  JavaScriptCore                      0x000000010bc8c151 _ZN3WTF7RunLoop11performWorkEv + 545
	22  JavaScriptCore                      0x000000010bc8c9a2 _ZN3WTF7RunLoop11performWorkEPv + 34
	23  CoreFoundation                      0x00007fff2045aa0c __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
	24  CoreFoundation                      0x00007fff2045a974 __CFRunLoopDoSource0 + 180
	25  CoreFoundation                      0x00007fff2045a6ef __CFRunLoopDoSources0 + 248
	26  CoreFoundation                      0x00007fff20459121 __CFRunLoopRun + 890
	27  CoreFoundation                      0x00007fff204586ce CFRunLoopRunSpecific + 563
	28  Foundation                          0x00007fff211e5fa1 -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 212
	29  Foundation                          0x00007fff21274384 -[NSRunLoop(NSRunLoop) run] + 76
	30  libxpc.dylib                        0x00007fff200af3dd _xpc_objc_main + 825
	31  libxpc.dylib                        0x00007fff200aee65 _xpc_copy_xpcservice_dictionary + 0
	32  WebKit                              0x000000010325664f _ZN6WebKit14XPCServiceMainEiPPKc + 310
	33  libdyld.dylib                       0x00007fff2037d621 start + 1
)
libc++abi.dylib: terminating with uncaught exception of type NSException

[Radar WebKit Bug Importer]

<rdar://problem/74600790>

[Robert Jenner]

Reproduced the crashing at tip of tree by generating a test list from the stdio file. Tested the test list using: 

run-webkit-test --root <path to revision> <path to test list> --child-process=1 

I have attached the test list I generated and used to duplicate the crashing.

[Robert Jenner]

Created attachment 421248 [details]
test list used to reproduce crash.

[Robert Jenner]

Narrowed Problem down to one particular test. 

When: 
imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https.html

is ran before:
media/media-fragments/TC0051.html

it causes "media/media-fragments/TC0051.html" to crash.

When "imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https.html" is removed, then the TC0051 test does not crash.

[Robert Jenner]

imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-no-sink.https.html

Also causes "media/media-fragments/TC0051.html" to crash.

[Robert Jenner]

*** Bug 222185 has been marked as a duplicate of this bug. ***

[Robert Jenner]

*** Bug 221930 has been marked as a duplicate of this bug. ***

[Robert Jenner]

Consolidating two other bugs that are GPUP crashes to this one, as they all appear related. This bug has more reproduction steps, and will make everything easier to track. The two other tests that appear related to this are:

media/media-fragments/TC0034.html 
media/media-extension-with-fragment.html*

*This particular test was only crashing on EWS. However, it does appear related to the GPUP crashes.

[Robert Jenner]

Created attachment 421459 [details]
Patch

[Ryan Haddad]

Comment on attachment 421459 [details]
Patch

Marking CQ- because if we skip this test, the next one will start crashing. We should see if there is a particular test (or set of tests) that can be skipped to prevent this from happening, or, better yet, get a fix for the underlying issue.

[Robert Jenner]

Looking into this further, and narrowing down. It appears that there are three tests that cause "media/media-fragments/TC0051.html" to crash. All of which belong to the same family of tests. The three tests are:
imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-no-sink.https.html

imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https.html

imported/w3c/web-platform-tests/mediacapture-record/idlharness.window.html

A fourth test related to that family of tests, is also flakey text failing:
imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-error.html

[Robert Jenner]

Created attachment 421479 [details]
Patch

[Truitt Savell]

Comment on attachment 421479 [details]
Patch

Clearing flags on attachment: 421479

Committed r273497 (234574@main): <https://commits.webkit.org/234574@main>

[youenn fablet]

I reproduced the bug and played with options a bit:
- Crash disappears if encoding is done in WebProcess.
- Crash happens as soon as we create a VTB session in RTCVideoEncoderH264 resetCompressionSessionWithPixelFormat.
- Crash does not happen if the encoder is not using VCP.

I do not understand why this issue happens only with GPUProcess, maybe there is some init code that does not run in the same order.

[Ryan Haddad]

If https://bugs.webkit.org/show_bug.cgi?id=223277 was supposed to be the workaround for this crash, I don't think it worked. We're still seeing this test crash on the bots.

[Ryan Haddad]

<rdar://75842806>

[Jer Noble]

Created attachment 424545 [details]
Patch

[Jer Noble]

Created attachment 424546 [details]
Patch

[Eric Carlson]

Comment on attachment 424546 [details]
Patch

r=me once the bots can deal with the changes

[Jer Noble]

Created attachment 424551 [details]
Patch

[Daniel Bates]

Comment on attachment 424551 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=424551&action=review

> Source/WebCore/ChangeLog:14
> +        r274734 attempted to be efficient, by only checking whether calls to parse a malforgmed fragment

Very minor, no change needed. Malforgmed is misspelled.

> Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:814
> +    NeverDestroyed<Optional<bool>> value;

This should be static.

> Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:995
> +        m_avAsset = adoptNS([PAL::allocAVURLAssetInstance() initWithURL:cocoaURL options:options.get()]);

Could this throw an exception?

[Daniel Bates]

Comment on attachment 424551 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=424551&action=review

>> Source/WebCore/ChangeLog:14
>> +        r274734 attempted to be efficient, by only checking whether calls to parse a malforgmed fragment
> 
> Very minor, no change needed. Malforgmed is misspelled.

...or maybe malformed 😀

[Ryan Haddad]

Since this patch still needs work, I've skipped the media/media-fragments directory again in media/media-fragments to speed up EWS.

[Ryan Haddad]

(In reply to Ryan Haddad from comment #23)
> Since this patch still needs work, I've skipped the media/media-fragments
> directory again in media/media-fragments to speed up EWS.
in https://trac.webkit.org/changeset/275526/webkit

[Robert Jenner]

I received a comment yesterday from the Photos team that the issues were fixed. I re-enabled the tests that were disabled, and they are still crashing. I reverted my change here, and will be putting it back on the Photos team:
https://bugs.webkit.org/show_bug.cgi?id=224724

[Jer Noble]

Created attachment 428461 [details]
Patch

[Eric Carlson]

Comment on attachment 428461 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=428461&action=review

> Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:961
> +        m_avAsset = adoptNS([PAL::allocAVURLAssetInstance() initWithURL:cocoaURL options:options.get()]);

It is probably worth restructuring this so you can wrap the second allocation attempt in another @try/@catch so we don't crash if it throws another exception for some reason.

[Jer Noble]

Created attachment 428517 [details]
Patch for landing

[EWS]

Committed r277442 (237690@main): <https://commits.webkit.org/237690@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 428517 [details].