Bug 190991 – REGRESSION(r237180/r237489): [GTK] Layout test regressions from EME enablement

• Status: NEW
• Component: WebKitGTK

[Michael Catanzaro]

I'm going to use this bug to track regressions caused by EME enablement in r237489. We'll have to decide what to do about these.

So far I've noticed fast/dom/navigator-detached-no-crash.html has broken. Text diff:

--- /home/slave/webkitgtk/gtk-linux-64-release-tests/build/layout-test-results/fast/dom/navigator-detached-no-crash-expected.txt
+++ /home/slave/webkitgtk/gtk-linux-64-release-tests/build/layout-test-results/fast/dom/navigator-detached-no-crash-actual.txt
@@ -1,3 +1,4 @@
+CONSOLE MESSAGE: Unhandled Promise Rejection: TypeError: Not enough arguments
 This tests that the navigator object of a deleted frame is disconnected properly. Accessing fields or methods shouldn't crash the browser. 
  Check Navigator
 navigator.activeVRDisplays is OK
@@ -17,6 +18,7 @@
 navigator.plugins is OK
 navigator.product is OK
 navigator.productSub is OK
+navigator.requestMediaKeySystemAccess() is OK
 navigator.sendBeacon() threw err TypeError: Not enough arguments
 navigator.serviceWorker is OK
 navigator.userAgent is OK
@@ -40,6 +42,7 @@
 navigator.plugins is OK
 navigator.product is OK
 navigator.productSub is OK
+navigator.requestMediaKeySystemAccess() is OK
 navigator.sendBeacon() threw err TypeError: Not enough arguments
 navigator.serviceWorker is OK
 navigator.userAgent is OK

Looks like a couple expected changes, but the unhandled promise rejection looks bad.

[Michael Catanzaro]

Hm, in addition to r237489, I see tests failing since r237180.

r237180: "[EME] Multiple ClearKey tests crashing in gst_qtdemux_request_protection_context"
r237489: "[GTK] Enable experimental encrypted media support"

Here are two failing since r237180:

imported/w3c/web-platform-tests/encrypted-media/clearkey-mp4-waiting-for-a-key.https.html
imported/w3c/web-platform-tests/encrypted-media/idlharness.https.html




--- /home/slave/webkitgtk/gtk-linux-64-release-tests/build/layout-test-results/imported/w3c/web-platform-tests/encrypted-media/clearkey-mp4-waiting-for-a-key.https-expected.txt
+++ /home/slave/webkitgtk/gtk-linux-64-release-tests/build/layout-test-results/imported/w3c/web-platform-tests/encrypted-media/clearkey-mp4-waiting-for-a-key.https-actual.txt
@@ -1,3 +1,5 @@
 
-TIMEOUT Waiting for a key. timeout. message = generateRequest(), encrypted: true, waitingforkey: true, timeupdate count: 0
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT Waiting for a key. Test timed out
+



--- /home/slave/webkitgtk/gtk-linux-64-release-tests/build/layout-test-results/imported/w3c/web-platform-tests/encrypted-media/idlharness.https-expected.txt
+++ /home/slave/webkitgtk/gtk-linux-64-release-tests/build/layout-test-results/imported/w3c/web-platform-tests/encrypted-media/idlharness.https-actual.txt
@@ -1,107 +1,11 @@
+CONSOLE MESSAGE: line 832: Unhandled Promise Rejection: MediaKeyMessageEventInit inherits EventInit, but EventInit is undefined.
 Description
 
 This test verifies that implementations of the Encrypted Media Extensions API match its WebIDL definition.
 
 
-PASS Navigator interface: operation requestMediaKeySystemAccess(DOMString, [object Object])
-PASS Unscopable handled correctly for requestMediaKeySystemAccess(DOMString, [object Object]) on Navigator
-PASS HTMLMediaElement interface: attribute mediaKeys
-PASS Unscopable handled correctly for mediaKeys property on HTMLMediaElement
-PASS HTMLMediaElement interface: attribute onencrypted
-PASS Unscopable handled correctly for onencrypted property on HTMLMediaElement
-PASS HTMLMediaElement interface: attribute onwaitingforkey
-PASS Unscopable handled correctly for onwaitingforkey property on HTMLMediaElement
-PASS HTMLMediaElement interface: operation setMediaKeys(MediaKeys)
-PASS Unscopable handled correctly for setMediaKeys(MediaKeys) on HTMLMediaElement
-PASS MediaKeySystemAccess interface: existence and properties of interface object
-PASS MediaKeySystemAccess interface object length
-PASS MediaKeySystemAccess interface object name
-PASS MediaKeySystemAccess interface: existence and properties of interface prototype object
-PASS MediaKeySystemAccess interface: existence and properties of interface prototype object's "constructor" property
-PASS MediaKeySystemAccess interface: existence and properties of interface prototype object's @@unscopables property
-PASS MediaKeySystemAccess interface: attribute keySystem
-PASS Unscopable handled correctly for keySystem property on MediaKeySystemAccess
-PASS MediaKeySystemAccess interface: operation getConfiguration()
-PASS Unscopable handled correctly for getConfiguration() on MediaKeySystemAccess
-PASS MediaKeySystemAccess interface: operation createMediaKeys()
-PASS Unscopable handled correctly for createMediaKeys() on MediaKeySystemAccess
-PASS MediaKeys interface: existence and properties of interface object
-PASS MediaKeys interface object length
-PASS MediaKeys interface object name
-PASS MediaKeys interface: existence and properties of interface prototype object
-PASS MediaKeys interface: existence and properties of interface prototype object's "constructor" property
-PASS MediaKeys interface: existence and properties of interface prototype object's @@unscopables property
-PASS MediaKeys interface: operation createSession(MediaKeySessionType)
-PASS Unscopable handled correctly for createSession(MediaKeySessionType) on MediaKeys
-PASS MediaKeys interface: operation setServerCertificate(BufferSource)
-PASS Unscopable handled correctly for setServerCertificate(BufferSource) on MediaKeys
-PASS MediaKeySession interface: existence and properties of interface object
-PASS MediaKeySession interface object length
-PASS MediaKeySession interface object name
-PASS MediaKeySession interface: existence and properties of interface prototype object
-PASS MediaKeySession interface: existence and properties of interface prototype object's "constructor" property
-PASS MediaKeySession interface: existence and properties of interface prototype object's @@unscopables property
-PASS MediaKeySession interface: attribute sessionId
-PASS Unscopable handled correctly for sessionId property on MediaKeySession
-PASS MediaKeySession interface: attribute expiration
-PASS Unscopable handled correctly for expiration property on MediaKeySession
-PASS MediaKeySession interface: attribute closed
-PASS Unscopable handled correctly for closed property on MediaKeySession
-PASS MediaKeySession interface: attribute keyStatuses
-PASS Unscopable handled correctly for keyStatuses property on MediaKeySession
-PASS MediaKeySession interface: attribute onkeystatuseschange
-PASS Unscopable handled correctly for onkeystatuseschange property on MediaKeySession
-PASS MediaKeySession interface: attribute onmessage
-PASS Unscopable handled correctly for onmessage property on MediaKeySession
-PASS MediaKeySession interface: operation generateRequest(DOMString, BufferSource)
-PASS Unscopable handled correctly for generateRequest(DOMString, BufferSource) on MediaKeySession
-PASS MediaKeySession interface: operation load(DOMString)
-PASS Unscopable handled correctly for load(DOMString) on MediaKeySession
-PASS MediaKeySession interface: operation update(BufferSource)
-PASS Unscopable handled correctly for update(BufferSource) on MediaKeySession
-PASS MediaKeySession interface: operation close()
-PASS Unscopable handled correctly for close() on MediaKeySession
-PASS MediaKeySession interface: operation remove()
-PASS Unscopable handled correctly for remove() on MediaKeySession
-PASS MediaKeyStatusMap interface: existence and properties of interface object
-PASS MediaKeyStatusMap interface object length
-PASS MediaKeyStatusMap interface object name
-PASS MediaKeyStatusMap interface: existence and properties of interface prototype object
-PASS MediaKeyStatusMap interface: existence and properties of interface prototype object's "constructor" property
-PASS MediaKeyStatusMap interface: existence and properties of interface prototype object's @@unscopables property
-PASS Testing Symbol.iterator property of iterable interface MediaKeyStatusMap
-PASS Testing pair iterable interface MediaKeyStatusMap
-PASS MediaKeyStatusMap interface: attribute size
-PASS Unscopable handled correctly for size property on MediaKeyStatusMap
-PASS MediaKeyStatusMap interface: operation has(BufferSource)
-PASS Unscopable handled correctly for has(BufferSource) on MediaKeyStatusMap
-PASS MediaKeyStatusMap interface: operation get(BufferSource)
-PASS Unscopable handled correctly for get(BufferSource) on MediaKeyStatusMap
-PASS MediaKeyStatusMap interface: operation entries()
-PASS Unscopable handled correctly for entries() on MediaKeyStatusMap
-PASS MediaKeyStatusMap interface: operation keys()
-PASS Unscopable handled correctly for keys() on MediaKeyStatusMap
-PASS MediaKeyStatusMap interface: operation values()
-PASS Unscopable handled correctly for values() on MediaKeyStatusMap
-PASS MediaKeyStatusMap interface: operation forEach(function, any)
-PASS Unscopable handled correctly for forEach(function, any) on MediaKeyStatusMap
-PASS MediaKeyMessageEvent interface: existence and properties of interface object
-PASS MediaKeyMessageEvent interface object length
-PASS MediaKeyMessageEvent interface object name
-PASS MediaKeyMessageEvent interface: existence and properties of interface prototype object
-PASS MediaKeyMessageEvent interface: existence and properties of interface prototype object's "constructor" property
-PASS MediaKeyMessageEvent interface: existence and properties of interface prototype object's @@unscopables property
-PASS MediaKeyMessageEvent interface: attribute messageType
-PASS Unscopable handled correctly for messageType property on MediaKeyMessageEvent
-PASS MediaKeyMessageEvent interface: attribute message
-PASS Unscopable handled correctly for message property on MediaKeyMessageEvent
-PASS MediaEncryptedEvent interface: existence and properties of interface object
-PASS MediaEncryptedEvent interface object length
-PASS MediaEncryptedEvent interface object name
-PASS MediaEncryptedEvent interface: existence and properties of interface prototype object
-PASS MediaEncryptedEvent interface: existence and properties of interface prototype object's "constructor" property
-PASS MediaEncryptedEvent interface: existence and properties of interface prototype object's @@unscopables property
-PASS MediaEncryptedEvent interface: attribute initDataType
-PASS Unscopable handled correctly for initDataType property on MediaEncryptedEvent
-PASS MediaEncryptedEvent interface: attribute initData
-PASS Unscopable handled correctly for initData property on MediaEncryptedEvent
+Harness Error (FAIL), message = MediaKeyMessageEventInit inherits EventInit, but EventInit is undefined.
+
+PASS Partial interface Navigator: original interface defined 
+PASS Partial interface HTMLMediaElement: original interface defined 
+

[Michael Catanzaro]

More failing since r237180:

media/encrypted-media/encrypted-media-constants.html
media/encrypted-media/encrypted-media-is-type-supported.html


--- /home/slave/webkitgtk/gtk-linux-64-release-tests/build/layout-test-results/media/encrypted-media/encrypted-media-constants-expected.txt
+++ /home/slave/webkitgtk/gtk-linux-64-release-tests/build/layout-test-results/media/encrypted-media/encrypted-media-constants-actual.txt
@@ -1,12 +1,12 @@
 Test WebKitMediaKeyError constants and additional MediaError constant.
 
-EXPECTED (WebKitMediaKeyError.MEDIA_KEYERR_UNKNOWN == '1') OK
-EXPECTED (WebKitMediaKeyError.MEDIA_KEYERR_CLIENT == '2') OK
-EXPECTED (WebKitMediaKeyError.MEDIA_KEYERR_SERVICE == '3') OK
-EXPECTED (WebKitMediaKeyError.MEDIA_KEYERR_OUTPUT == '4') OK
-EXPECTED (WebKitMediaKeyError.MEDIA_KEYERR_HARDWARECHANGE == '5') OK
-EXPECTED (WebKitMediaKeyError.MEDIA_KEYERR_DOMAIN == '6') OK
+ReferenceError: Can't find variable: WebKitMediaKeyError
+ReferenceError: Can't find variable: WebKitMediaKeyError
+ReferenceError: Can't find variable: WebKitMediaKeyError
+ReferenceError: Can't find variable: WebKitMediaKeyError
+ReferenceError: Can't find variable: WebKitMediaKeyError
+ReferenceError: Can't find variable: WebKitMediaKeyError
 
-EXPECTED (MediaError.MEDIA_ERR_ENCRYPTED == '5') OK
+EXPECTED (MediaError.MEDIA_ERR_ENCRYPTED == '5'), OBSERVED 'undefined' FAIL
 END OF TEST
 

--- /home/slave/webkitgtk/gtk-linux-64-release-tests/build/layout-test-results/media/encrypted-media/encrypted-media-is-type-supported-expected.txt
+++ /home/slave/webkitgtk/gtk-linux-64-release-tests/build/layout-test-results/media/encrypted-media/encrypted-media-is-type-supported-actual.txt
@@ -1,22 +1,23 @@
+CONSOLE MESSAGE: line 11: TypeError: internals.initializeMockCDM is not a function. (In 'internals.initializeMockCDM()', 'internals.initializeMockCDM' is undefined)
 Test Encrypted Media extension of WebKitMediaKeys isTypeSupported() method.
 
 Always return false for a null type.
-EXPECTED (WebKitMediaKeys.isTypeSupported(null) == 'false') OK
+ReferenceError: Can't find variable: WebKitMediaKeys
 
 Always return false for an empty type.
-EXPECTED (WebKitMediaKeys.isTypeSupported('') == 'false') OK
-EXPECTED (WebKitMediaKeys.isTypeSupported('', '') == 'false') OK
-EXPECTED (WebKitMediaKeys.isTypeSupported('', null) == 'false') OK
-EXPECTED (WebKitMediaKeys.isTypeSupported('', undefined) == 'false') OK
-EXPECTED (WebKitMediaKeys.isTypeSupported('', 'video/invalid') == 'false') OK
+ReferenceError: Can't find variable: WebKitMediaKeys
+ReferenceError: Can't find variable: WebKitMediaKeys
+ReferenceError: Can't find variable: WebKitMediaKeys
+ReferenceError: Can't find variable: WebKitMediaKeys
+ReferenceError: Can't find variable: WebKitMediaKeys
 
 Always return true for supported types.
-EXPECTED (WebKitMediaKeys.isTypeSupported('com.webcore.mock') == 'true') OK
+ReferenceError: Can't find variable: WebKitMediaKeys
 
 Always return true for supported types and containers.
-EXPECTED (WebKitMediaKeys.isTypeSupported('com.webcore.mock', 'video/mock') == 'true') OK
+ReferenceError: Can't find variable: WebKitMediaKeys
 
 Always return false for supported types but unsupported containers.
-EXPECTED (WebKitMediaKeys.isTypeSupported('com.webcore.mock', 'video/invalid') == 'false') OK
+ReferenceError: Can't find variable: WebKitMediaKeys
 END OF TEST

[Michael Catanzaro]

There are also 52 tests with missing expectations. That's too many for regular gardening to deal with, and I see many of those tests are not doing very great either.

Charlie, I think these commits need to be rolled out, unless you have time to add the missing expectations?

[Michael Catanzaro]

For now I'll just add a [ Skip ] expectation back to imported/w3c/web-platform-tests/encrypted-media/ until you have time to deal with it; that should do well enough.

[Michael Catanzaro]

More regressions:

media/encrypted-media/encrypted-media-events.html
media/encrypted-media/encrypted-media-session-lifetime.html
media/encrypted-media/encrypted-media-v2-events.html
media/encrypted-media/encrypted-media-v2-syntax.html

[Xabier Rodríguez Calvar]

Tests that use prefixed API should be skipped because they use LEGACY_ENCRYPTED_MEDIA aka V2, not like ENCRYPTED_MEDIA aka V3.

[Diego Pino]

media/encrypted-media/encrypted-media-events.html sometimes crashes (See: https://build.webkit.org/results/GTK%20Linux%2064-bit%20Release%20(Tests)/r259906%20(13284)/media/encrypted-media/encrypted-media-events-crash-log.txt):

Thread 1 (Thread 0x7ff80cb7c9c0 (LWP 51230)):
#0  0x00007ff818f6f680 in WebCore::VideoTrackPrivateGStreamer::markAsActive() () from /home/slave/webkitgtk/gtk-linux-64-release/build/WebKitBuild/Release/lib/libwebkit2gtk-4.0.so.37
#1  0x00007ff818f679e6 in WebCore::MediaPlayerPrivateGStreamer::handleMessage(_GstMessage*) () from /home/slave/webkitgtk/gtk-linux-64-release/build/WebKitBuild/Release/lib/libwebkit2gtk-4.0.so.37
#2  0x00007ff80d0f88ee in ffi_call_unix64 () from /usr/lib/x86_64-linux-gnu/libffi.so.6
#3  0x00007ff80d0f82bf in ffi_call () from /usr/lib/x86_64-linux-gnu/libffi.so.6
#4  0x00007ff80ec74509 in g_cclosure_marshal_generic () at ../../Source/glib-2.58.1/gobject/gclosure.c:1496
#5  0x00007ff80ec73d3d in g_closure_invoke () at ../../Source/glib-2.58.1/gobject/gclosure.c:810
#6  0x00007ff80ec86d35 in signal_emit_unlocked_R () at ../../Source/glib-2.58.1/gobject/gsignal.c:3635
#7  0x00007ff80ec8fdbe in g_signal_emit_valist () at ../../Source/glib-2.58.1/gobject/gsignal.c:3391
#8  0x00007ff80ec90452 in g_signal_emit () at ../../Source/glib-2.58.1/gobject/gsignal.c:3447
#9  0x00007ff810c984f4 in gst_bus_async_signal_func () at ../../Source/gstreamer-1.16.1/gst/gstbus.c:1261
#10 0x00007ff810c9930d in gst_bus_source_dispatch () at ../../Source/gstreamer-1.16.1/gst/gstbus.c:839
#11 0x00007ff80eb916b8 in g_main_dispatch () at ../../Source/glib-2.58.1/glib/gmain.c:3182
#12 g_main_context_dispatch () at ../../Source/glib-2.58.1/glib/gmain.c:3847
#13 0x00007ff80eb91a78 in g_main_context_iterate () at ../../Source/glib-2.58.1/glib/gmain.c:3920
#14 0x00007ff80eb91d62 in g_main_loop_run () at ../../Source/glib-2.58.1/glib/gmain.c:4116
#15 0x00007ff814fb4170 in WTF::RunLoop::run() () from /home/slave/webkitgtk/gtk-linux-64-release/build/WebKitBuild/Release/lib/libjavascriptcoregtk-4.0.so.18
#16 0x00007ff8177e28fd in int WebKit::AuxiliaryProcessMain<WebKit::WebProcess, WebKit::WebProcessMainGtk>(int, char**) () from /home/slave/webkitgtk/gtk-linux-64-release/build/WebKitBuild/Release/lib/libwebkit2gtk-4.0.so.37
#17 0x00007ff80e64409b in __libc_start_main (main=0x5618945e4c10 <main>, argc=4, argv=0x7ffe1bb369f8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffe1bb369e8) at ../csu/libc-start.c:308
#18 0x00005618945e4c8a in _start ()

[Charlie Turner]

(In reply to Diego Pino from comment #7)
> media/encrypted-media/encrypted-media-events.html sometimes crashes (See:
> https://build.webkit.org/results/GTK%20Linux%2064-bit%20Release%20(Tests)/
> r259906%20(13284)/media/encrypted-media/encrypted-media-events-crash-log.
> txt):

You have quoted a different backtrace to the one linked.

The trace pasted here should be investigated as a separate bug, can you confirm it was created by this test case? It's not relevant to EME specifically, though an EME test may have triggered it.

The trace linked is the infamous Xvfb issue, which happens for reasons I don't think anyone has figured out on lots of different tests.

[Diego Pino]

(In reply to Charlie Turner from comment #8)
> (In reply to Diego Pino from comment #7)
> > media/encrypted-media/encrypted-media-events.html sometimes crashes (See:
> > https://build.webkit.org/results/GTK%20Linux%2064-bit%20Release%20(Tests)/
> > r259906%20(13284)/media/encrypted-media/encrypted-media-events-crash-log.
> > txt):
> 
> You have quoted a different backtrace to the one linked.
> 
> The trace pasted here should be investigated as a separate bug, can you
> confirm it was created by this test case? It's not relevant to EME
> specifically, though an EME test may have triggered it.
> 
> The trace linked is the infamous Xvfb issue, which happens for reasons I
> don't think anyone has figured out on lots of different tests.


Sorry, my bad.

The pasted stacktrace corresponds to https://build.webkit.org/results/GTK%20Linux%2064-bit%20Release%20(Tests)/r259865%20(13278)/fast/mediastream/stream-switch-crash-log.txt. There's a bug created for this issue at https://bugs.webkit.org/show_bug.cgi?id=210385.