Bug 254706 – [ iOS, macOS ] imported/w3c/web-platform-tests/html/dom/idlharness.https.html near constant failure.

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Bri Harris]

imported/w3c/web-platform-tests/html/dom/idlharness.https.html

is a near constant failure. It is failing on iOS 16 and is a flaky time out / failure on macOS Debug.

HISTORY:

https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fdom%2Fidlharness.https.html%3Fexclude%3D(Document%7CWindow%7CHTML.*)


DIFF TEXT:
@@ -1275,9 +1275,9 @@
 PASS Worklet interface object length
 PASS Worklet interface object name
 PASS Worklet interface: existence and properties of interface prototype object
-FAIL Worklet interface: existence and properties of interface prototype object's "constructor" property assert_own_property: Worklet.prototype does not have own property "constructor" expected property "constructor" missing
+PASS Worklet interface: existence and properties of interface prototype object's "constructor" property
 PASS Worklet interface: existence and properties of interface prototype object's @@unscopables property
-FAIL Worklet interface: operation addModule(USVString, optional WorkletOptions) assert_own_property: interface prototype object missing non-static operation expected property "addModule" missing
+PASS Worklet interface: operation addModule(USVString, optional WorkletOptions)
 PASS Storage interface: existence and properties of interface object
 PASS Storage interface object length
 PASS Storage interface object name

[Bri Harris]

I was able to replicate this failure on iOS 16 ToT as follows:

run-webkit-tests --iOS-simulator imported/w3c/web-platform-tests/html/dom/idlharness.https.html

[Radar WebKit Bug Importer]

<rdar://problem/107393338>

[Alexey Proskuryakov]

There are many problems with this test, but AFAICT the diff posted above just means that iOS has incorrect expectations for it.

Other problems that I see is that the test is super slow in debug, frequently timing out, and diffs of at least one more kind:

-PASS MediaError must be primary interface of errorVideo.error
-PASS Stringification of errorVideo.error
-PASS MediaError interface: errorVideo.error must inherit property "MEDIA_ERR_ABORTED" with the proper type
-PASS MediaError interface: errorVideo.error must inherit property "MEDIA_ERR_NETWORK" with the proper type
-PASS MediaError interface: errorVideo.error must inherit property "MEDIA_ERR_DECODE" with the proper type
-PASS MediaError interface: errorVideo.error must inherit property "MEDIA_ERR_SRC_NOT_SUPPORTED" with the proper type
-PASS MediaError interface: errorVideo.error must inherit property "code" with the proper type
-PASS MediaError interface: errorVideo.error must inherit property "message" with the proper type
+FAIL Stringification of errorVideo.error assert_class_string: class string of errorVideo.error expected "[object MediaError]" but got "[object Null]"
+FAIL MediaError interface: errorVideo.error must inherit property "MEDIA_ERR_ABORTED" with the proper type assert_inherits: provided value is not an object
+FAIL MediaError interface: errorVideo.error must inherit property "MEDIA_ERR_NETWORK" with the proper type assert_inherits: provided value is not an object
+FAIL MediaError interface: errorVideo.error must inherit property "MEDIA_ERR_DECODE" with the proper type assert_inherits: provided value is not an object
+FAIL MediaError interface: errorVideo.error must inherit property "MEDIA_ERR_SRC_NOT_SUPPORTED" with the proper type assert_inherits: provided value is not an object
+FAIL MediaError interface: errorVideo.error must inherit property "code" with the proper type assert_inherits: provided value is not an object
+FAIL MediaError interface: errorVideo.error must inherit property "message" with the proper type assert_inherits: provided value is not an object

[EWS]

Test gardening commit 262595@main (4798b5317d9d): <https://commits.webkit.org/262595@main>

Reviewed commits have been landed. Closing PR #12396 and removing active labels.

[EWS]

Test gardening commit 259548.602@safari-7615-branch (1fd08d3625dd): <https://commits.webkit.org/259548.602@safari-7615-branch>

Reviewed commits have been landed. Closing PR #537 and removing active labels.

[youenn fablet]

Looking at the test results, on iOS it might require a specific expectation for iPad vs. iPhone due to MSE support.

For Debug in general, idlharness test should be marked as slow.

[youenn fablet]

Pull request: https://github.com/WebKit/WebKit/pull/19164

[EWS]

Committed 269461@main (2e494e5298f7): <https://commits.webkit.org/269461@main>

Reviewed commits have been landed. Closing PR #19164 and removing active labels.