Bug 222277 - [ macOS Wk2 ] media/media-fragments/TC0051.html is flakey crashing
Summary: [ macOS Wk2 ] media/media-fragments/TC0051.html is flakey crashing
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: youenn fablet
URL:
Keywords: InRadar
: 221930 222185 (view as bug list)
Depends on: 224724
Blocks:
  Show dependency treegraph
 
Reported: 2021-02-22 09:56 PST by Robert Jenner
Modified: 2021-04-17 16:55 PDT (History)
11 users (show)

See Also:


Attachments
test list used to reproduce crash. (189.79 KB, text/plain)
2021-02-22 15:28 PST, Robert Jenner
no flags Details
Patch (1.47 KB, patch)
2021-02-24 15:00 PST, Robert Jenner
no flags Details | Formatted Diff | Diff
Patch (2.60 KB, patch)
2021-02-24 16:17 PST, Robert Jenner
no flags Details | Formatted Diff | Diff
Patch (5.05 KB, patch)
2021-03-29 10:51 PDT, Jer Noble
no flags Details | Formatted Diff | Diff
Patch (5.05 KB, patch)
2021-03-29 10:52 PDT, Jer Noble
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch (5.06 KB, patch)
2021-03-29 11:19 PDT, Jer Noble
dbates: review+
ews-feeder: commit-queue-
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Jenner 2021-02-22 09:56:43 PST
media/media-fragments/TC0051.html

HISTORY URL:
https://results.webkit.org/?suite=layout-tests&test=media%2Fmedia-fragments%2FTC0051.html&platform=mac

CRASH LOG URL:
https://build.webkit.org/results/Apple-BigSur-Release-WK2-Tests/r273239%20(409)/media/media-fragments/TC0051-crash-log.txt

CRAHS LOG TEXT:
No crash log found for GPUProcess:41578.

stdout:

stderr:
2021-02-22 05:16:22.066 com.apple.WebKit.GPU.Development[41578:82907104] *** Terminating app due to uncaught exception 'NSRangeException', reason: '*** -[__NSArrayM objectAtIndex:]: index 3 beyond bounds [0 .. 2]'
*** First throw call stack:
(
	0   CoreFoundation                      0x00007fff204d46af __exceptionPreprocess + 242
	1   libobjc.A.dylib                     0x00007fff2020c3c9 objc_exception_throw + 48
	2   CoreFoundation                      0x00007fff20588a9a -[__NSCFString characterAtIndex:].cold.1 + 0
	3   CoreFoundation                      0x00007fff203fb163 -[NSTaggedPointerString hash] + 0
	4   Photos                              0x00007fff3e627453 +[AVAssetCollection(Utilities) fragmentParamsFromURL:] + 219
	5   Photos                              0x00007fff3e6272c6 +[AVAssetCollection(Utilities) isURLForAssetInCollection:] + 102
	6   AVFCore                             0x00007fff3167e1b0 -[AVURLAsset initWithURL:options:] + 136
	7   WebCore                             0x000000010545df34 _ZN7WebCore34MediaPlayerPrivateAVFoundationObjC19createAVAssetForURLERKN3WTF3URLENS1_9RetainPtrI19NSMutableDictionaryEE + 2724
	8   WebCore                             0x000000010545d447 _ZN7WebCore34MediaPlayerPrivateAVFoundationObjC19createAVAssetForURLERKN3WTF3URLE + 87
	9   WebCore                             0x0000000105451bd4 _ZN7WebCore30MediaPlayerPrivateAVFoundation10setPreloadENS_16MediaPlayerEnums7PreloadE + 212
	10  WebCore                             0x0000000105450349 _ZN7WebCore30MediaPlayerPrivateAVFoundation4loadERKN3WTF6StringE + 457
	11  WebCore                             0x0000000106afa8e3 _ZN7WebCore11MediaPlayer23loadWithNextMediaEngineEPKNS_18MediaPlayerFactoryE + 691
	12  WebCore                             0x0000000106afa545 _ZN7WebCore11MediaPlayer4loadERKN3WTF3URLERKNS_11ContentTypeERKNS1_6StringE + 1269
	13  WebKit                              0x00000001031d4129 _ZN6WebKit22RemoteMediaPlayerProxy4loadEON3WTF3URLEONS1_8OptionalINS_16SandboxExtension6HandleEEERKN7WebCore11ContentTypeERKNS1_6StringEONS1_17CompletionHandlerIFvONS_30RemoteMediaPlayerConfigurationEEEE + 185
	14  WebKit                              0x00000001031521c0 _ZN3IPC22callMemberFunctionImplIN6WebKit22RemoteMediaPlayerProxyEMS2_FvON3WTF3URLEONS3_8OptionalINS1_16SandboxExtension6HandleEEERKN7WebCore11ContentTypeERKNS3_6StringEONS3_17CompletionHandlerIFvONS1_30RemoteMediaPlayerConfigurationEEEEEFvRKSJ_ENSt3__15tupleIJS4_S9_SC_SF_EEEJLm0ELm1ELm2ELm3EEEEvPT_T0_ONSI_IT1_EEOT2_NST_16integer_sequenceImJXspT3_EEEE + 110
	15  WebKit                              0x0000000103150d4f _ZN3IPC18handleMessageAsyncIN8Messages22RemoteMediaPlayerProxy4LoadEN6WebKit22RemoteMediaPlayerProxyEMS5_FvON3WTF3URLEONS6_8OptionalINS4_16SandboxExtension6HandleEEERKN7WebCore11ContentTypeERKNS6_6StringEONS6_17CompletionHandlerIFvONS4_30RemoteMediaPlayerConfigurationEEEEEEEvRNS_10ConnectionERNS_7DecoderEPT0_T1_ + 196
	16  WebKit                              0x00000001031d38bc _ZN6WebKit29RemoteMediaPlayerManagerProxy23didReceivePlayerMessageERN3IPC10ConnectionERNS1_7DecoderE + 56
	17  WebKit                              0x00000001031b78ad _ZN6WebKit25GPUConnectionToWebProcess15dispatchMessageERN3IPC10ConnectionERNS1_7DecoderE + 121
	18  WebKit                              0x00000001031752e6 _ZN6WebKit25GPUConnectionToWebProcess17didReceiveMessageERN3IPC10ConnectionERNS1_7DecoderE + 434
	19  WebKit                              0x000000010309dd61 _ZN3IPC10Connection15dispatchMessageENSt3__110unique_ptrINS_7DecoderENS1_14default_deleteIS3_EEEE + 221
	20  WebKit                              0x000000010309dfb5 _ZN3IPC10Connection26dispatchOneIncomingMessageEv + 169
	21  JavaScriptCore                      0x000000010bc8c151 _ZN3WTF7RunLoop11performWorkEv + 545
	22  JavaScriptCore                      0x000000010bc8c9a2 _ZN3WTF7RunLoop11performWorkEPv + 34
	23  CoreFoundation                      0x00007fff2045aa0c __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
	24  CoreFoundation                      0x00007fff2045a974 __CFRunLoopDoSource0 + 180
	25  CoreFoundation                      0x00007fff2045a6ef __CFRunLoopDoSources0 + 248
	26  CoreFoundation                      0x00007fff20459121 __CFRunLoopRun + 890
	27  CoreFoundation                      0x00007fff204586ce CFRunLoopRunSpecific + 563
	28  Foundation                          0x00007fff211e5fa1 -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 212
	29  Foundation                          0x00007fff21274384 -[NSRunLoop(NSRunLoop) run] + 76
	30  libxpc.dylib                        0x00007fff200af3dd _xpc_objc_main + 825
	31  libxpc.dylib                        0x00007fff200aee65 _xpc_copy_xpcservice_dictionary + 0
	32  WebKit                              0x000000010325664f _ZN6WebKit14XPCServiceMainEiPPKc + 310
	33  libdyld.dylib                       0x00007fff2037d621 start + 1
)
libc++abi.dylib: terminating with uncaught exception of type NSException
Comment 1 Radar WebKit Bug Importer 2021-02-22 09:57:16 PST
<rdar://problem/74600790>
Comment 2 Robert Jenner 2021-02-22 15:27:57 PST
Reproduced the crashing at tip of tree by generating a test list from the stdio file. Tested the test list using: 

run-webkit-test --root <path to revision> <path to test list> --child-process=1 

I have attached the test list I generated and used to duplicate the crashing.
Comment 3 Robert Jenner 2021-02-22 15:28:24 PST
Created attachment 421248 [details]
test list used to reproduce crash.
Comment 4 Robert Jenner 2021-02-22 18:10:55 PST
Narrowed Problem down to one particular test. 

When: 
imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https.html

is ran before:
media/media-fragments/TC0051.html

it causes "media/media-fragments/TC0051.html" to crash.

When "imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https.html" is removed, then the TC0051 test does not crash.
Comment 5 Robert Jenner 2021-02-23 10:05:45 PST
imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-no-sink.https.html

Also causes "media/media-fragments/TC0051.html" to crash.
Comment 6 Robert Jenner 2021-02-23 10:56:17 PST
*** Bug 222185 has been marked as a duplicate of this bug. ***
Comment 7 Robert Jenner 2021-02-23 10:56:39 PST
*** Bug 221930 has been marked as a duplicate of this bug. ***
Comment 8 Robert Jenner 2021-02-23 11:02:59 PST
Consolidating two other bugs that are GPUP crashes to this one, as they all appear related. This bug has more reproduction steps, and will make everything easier to track. The two other tests that appear related to this are:

media/media-fragments/TC0034.html 
media/media-extension-with-fragment.html*

*This particular test was only crashing on EWS. However, it does appear related to the GPUP crashes.
Comment 9 Robert Jenner 2021-02-24 15:00:26 PST
Created attachment 421459 [details]
Patch
Comment 10 Ryan Haddad 2021-02-24 15:07:57 PST
Comment on attachment 421459 [details]
Patch

Marking CQ- because if we skip this test, the next one will start crashing. We should see if there is a particular test (or set of tests) that can be skipped to prevent this from happening, or, better yet, get a fix for the underlying issue.
Comment 11 Robert Jenner 2021-02-24 16:01:10 PST
Looking into this further, and narrowing down. It appears that there are three tests that cause "media/media-fragments/TC0051.html" to crash. All of which belong to the same family of tests. The three tests are:
imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-no-sink.https.html

imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https.html

imported/w3c/web-platform-tests/mediacapture-record/idlharness.window.html

A fourth test related to that family of tests, is also flakey text failing:
imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-error.html
Comment 12 Robert Jenner 2021-02-24 16:17:16 PST
Created attachment 421479 [details]
Patch
Comment 13 Truitt Savell 2021-02-25 11:09:43 PST
Comment on attachment 421479 [details]
Patch

Clearing flags on attachment: 421479

Committed r273497 (234574@main): <https://commits.webkit.org/234574@main>
Comment 14 youenn fablet 2021-03-01 08:54:33 PST
I reproduced the bug and played with options a bit:
- Crash disappears if encoding is done in WebProcess.
- Crash happens as soon as we create a VTB session in RTCVideoEncoderH264 resetCompressionSessionWithPixelFormat.
- Crash does not happen if the encoder is not using VCP.

I do not understand why this issue happens only with GPUProcess, maybe there is some init code that does not run in the same order.
Comment 15 Ryan Haddad 2021-03-23 21:54:57 PDT
If https://bugs.webkit.org/show_bug.cgi?id=223277 was supposed to be the workaround for this crash, I don't think it worked. We're still seeing this test crash on the bots.
Comment 16 Ryan Haddad 2021-03-25 10:39:06 PDT
<rdar://75842806>
Comment 17 Jer Noble 2021-03-29 10:51:22 PDT
Created attachment 424545 [details]
Patch
Comment 18 Jer Noble 2021-03-29 10:52:22 PDT
Created attachment 424546 [details]
Patch
Comment 19 Eric Carlson 2021-03-29 10:56:20 PDT
Comment on attachment 424546 [details]
Patch

r=me once the bots can deal with the changes
Comment 20 Jer Noble 2021-03-29 11:19:06 PDT
Created attachment 424551 [details]
Patch
Comment 21 Daniel Bates 2021-03-30 00:22:52 PDT
Comment on attachment 424551 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=424551&action=review

> Source/WebCore/ChangeLog:14
> +        r274734 attempted to be efficient, by only checking whether calls to parse a malforgmed fragment

Very minor, no change needed. Malforgmed is misspelled.

> Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:814
> +    NeverDestroyed<Optional<bool>> value;

This should be static.

> Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:995
> +        m_avAsset = adoptNS([PAL::allocAVURLAssetInstance() initWithURL:cocoaURL options:options.get()]);

Could this throw an exception?
Comment 22 Daniel Bates 2021-03-30 00:25:50 PDT
Comment on attachment 424551 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=424551&action=review

>> Source/WebCore/ChangeLog:14
>> +        r274734 attempted to be efficient, by only checking whether calls to parse a malforgmed fragment
> 
> Very minor, no change needed. Malforgmed is misspelled.

...or maybe malformed 😀
Comment 23 Ryan Haddad 2021-04-06 09:38:06 PDT
Since this patch still needs work, I've skipped the media/media-fragments directory again in media/media-fragments to speed up EWS.
Comment 24 Ryan Haddad 2021-04-06 09:38:48 PDT
(In reply to Ryan Haddad from comment #23)
> Since this patch still needs work, I've skipped the media/media-fragments
> directory again in media/media-fragments to speed up EWS.
in https://trac.webkit.org/changeset/275526/webkit
Comment 25 Robert Jenner 2021-04-17 16:55:13 PDT
I received a comment yesterday from the Photos team that the issues were fixed. I re-enabled the tests that were disabled, and they are still crashing. I reverted my change here, and will be putting it back on the Photos team:
https://bugs.webkit.org/show_bug.cgi?id=224724