Regression (~r252261): [ Mac wk2 ] inspector/heap/getPreview.html is flaky failing

[Jason Lawrence]

inspector/heap/getPreview.html 

Description:
This test is flaky failing on Mac wk2. It appears to show a regression at 252273.

History:
https://results.webkit.org/?suite=layout-tests&test=inspector%2Fheap%2FgetPreview.html&platform=mac&limit=50000

Diff:
I was not able to gather a diff.

[Radar WebKit Bug Importer]

<rdar://problem/59152840>

[Jason Lawrence]

Created attachment 389685 [details]
Update Test Expectations

[Jason Lawrence]

Here's a diff:

--- /Volumes/Data/slave/catalina-debug-tests-wk2/build/layout-test-results/inspector/heap/getPreview-expected.txt
+++ /Volumes/Data/slave/catalina-debug-tests-wk2/build/layout-test-results/inspector/heap/getPreview-actual.txt
@@ -29,23 +29,12 @@
 OBJECT PREVIEW: {
     "type": "object",
     "description": "Map",
-    "lossless": true,
-    "subtype": "map",
-    "overflow": false,
-    "properties": [],
-    "size": 1,
-    "entries": [
+    "lossless": false,
+    "properties": [
         {
-            "key": {
-                "type": "string",
-                "description": "key",
-                "lossless": true
-            },
-            "value": {
-                "type": "string",
-                "description": "value",
-                "lossless": true
-            }
+            "name": "Symbol(Symbol.toStringTag)",
+            "type": "string",
+            "value": "Map"
         }
     ]
 }

[Jason Lawrence]

Created attachment 389687 [details]
Update Test Expectations

[Truitt Savell]

Comment on attachment 389687 [details]
Update Test Expectations

Clearing flags on attachment: 389687

Committed r255690: <https://trac.webkit.org/changeset/255690>

[Jacob Uphoff]

I ran this test in iterations x5000 and did not reproduce a failure.

[Jason Lawrence]

I was able to reproduce this issue with 255856 using the following command.


run-webkit-tests --root xxx inspector/heap/getPreview.html --iterations 5000 -f --exit-after-n-failures 5 -g --force --time-out-ms 10000

[15/5000] inspector/heap/getPreview.html failed unexpectedly (text diff)
[17/5000] inspector/heap/getPreview.html failed unexpectedly (text diff)
[18/5000] inspector/heap/getPreview.html failed unexpectedly (text diff)
[22/5000] inspector/heap/getPreview.html failed unexpectedly (text diff)
[23/5000] inspector/heap/getPreview.html failed unexpectedly (text diff)
Exiting early after 5 failures. 20 tests run.


15 tests ran as expected, 5 didn't (4980 didn't run):

[Joseph Pecoraro]

Hmm, I don't like the idea of papering over this potential issue. It sounds like a (In reply to Jason Lawrence from comment #3)
> Here's a diff:
> 
> ---
> /Volumes/Data/slave/catalina-debug-tests-wk2/build/layout-test-results/
> inspector/heap/getPreview-expected.txt
> +++
> /Volumes/Data/slave/catalina-debug-tests-wk2/build/layout-test-results/
> inspector/heap/getPreview-actual.txt
> @@ -29,23 +29,12 @@
>  OBJECT PREVIEW: {
>      "type": "object",
>      "description": "Map",
> -    "lossless": true,
> -    "subtype": "map",
> -    "overflow": false,
> -    "properties": [],
> -    "size": 1,
> -    "entries": [
> +    "lossless": false,
> +    "properties": [
>          {
> -            "key": {
> -                "type": "string",
> -                "description": "key",
> -                "lossless": true
> -            },
> -            "value": {
> -                "type": "string",
> -                "description": "value",
> -                "lossless": true
> -            }
> +            "name": "Symbol(Symbol.toStringTag)",
> +            "type": "string",
> +            "value": "Map"
>          }
>      ]
>  }

Does this always fail this way? Can we just rebaseline?

[Alexey Proskuryakov]

> 15 tests ran as expected, 5 didn't (4980 didn't run):

This suggests that no, we cannot rebaseline.

[Alexey Proskuryakov]

I don't see any failures at all before early November (roughly r252261), although there were always many timeouts.

[Jacob Uphoff]

Corrected test expectations on r256174