|Summary:||REGRESSION (r234330): [mac-wk1] Layout Test inspector/layers/layer-tree-manager.html and inspector/layers/layers-anonymous.html are flaky|
|Product:||WebKit||Reporter:||Truitt Savell <tsavell>|
|Component:||Tools / Tests||Assignee:||Nobody <webkit-unassigned>|
|Severity:||Normal||CC:||ap, bburg, lforschler, realdawei, ryanhaddad, simon.fraser, webkit-bug-importer|
|Version:||WebKit Nightly Build|
Description Truitt Savell 2018-08-07 16:42:02 PDT
The following layout tests are flaky on High Sierra Leaks inspector/layers/layer-tree-manager.html inspector/layers/layers-anonymous.html Probable cause: These two tests fail in tandem. A state is being set somewhere in the test runner. I have not reproduced yet. This started somewhere closely around r234344 Flakiness Dashboard: https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=inspector%2Flayers%2Flayers-anonymous.html%20inspector%2Flayers%2Flayer-tree-manager.html
Comment 1 Ryan Haddad 2018-08-07 16:56:46 PDT
inspector/layers/layer-tree-manager.html is a text failure: --- /Volumes/Data/slave/sierra-release-tests-wk1/build/layout-test-results/inspector/layers/layer-tree-manager-expected.txt +++ /Volumes/Data/slave/sierra-release-tests-wk1/build/layout-test-results/inspector/layers/layer-tree-manager-actual.txt @@ -3,7 +3,9 @@ == Running test suite: LayerTreeManager -- Running test case: LayerTreeManager.layersForNode.root -PASS: Returned array should include all layers. +FAIL: Returned array should include all layers. + Expected: 5 + Actual: 0 PASS: Array elements should be Layer instances. -- Running test case: LayerTreeManager.layerTreeMutations.overlappingNodeIds --- /Volumes/Data/slave/sierra-release-tests-wk1/build/layout-test-results/inspector/layers/layers-anonymous-expected.txt +++ /Volumes/Data/slave/sierra-release-tests-wk1/build/layout-test-results/inspector/layers/layers-anonymous-actual.txt @@ -1,3 +1,5 @@ +CONSOLE MESSAGE: line 58: TypeError: undefined is not an object (evaluating 'anonymousLayers.nodeId') +FAIL: Timed out waiting for notifyDone to be called This is a test inspector/layers/layers-anonymous.html times out with output: @@ -15,14 +17,5 @@ === Check layers === -PASS: Expected number of anonymous layers. -PASS: The sole anonymous layer has a non-zero node id. -PASS: The sole anonymous layer has a :first-letter pseudo-element. +FAIL: Expected number of anonymous layers. Expected 1 but got 0 -=== Check node === - -PASS: Node was found. -PASS: Node has expected localName. -PASS: Node has id. -PASS: Node has expected id. - Also, this isn't limited to leaks, but it definitely happens far more often there.
Comment 2 Ryan Haddad 2018-08-07 16:57:02 PDT
Possibly related to https://trac.webkit.org/changeset/234330/webkit?
Comment 3 Truitt Savell 2018-08-08 14:14:30 PDT
Created attachment 346793 [details] Test List I was able to reproduce the two failures by running the tests from the imported/ directory that ran directly before the inspector/ tests from the same test runner. I am going to work on bisecting down the blamed tests. Command Used: run-webkit-tests --test-list Test_list_directory --child-processes 1 -1 --leaks
Comment 4 Truitt Savell 2018-08-09 10:52:51 PDT
Created attachment 346846 [details] Test List New list of tests. Noting that the imported/ directory is no longer ran. The results are flakey at around a 2/3 chance of having the two tests fail.
Comment 5 Truitt Savell 2018-08-09 11:05:31 PDT
(In reply to Truitt Savell from comment #4) > Created attachment 346846 [details] > Test List > > New list of tests. Noting that the imported/ directory is no longer ran. The > results are flakey at around a 2/3 chance of having the two tests fail. For now I think this is the best list I can provide. removing any more tests from it either stops the failure condition or makes the test failure far less consistent.
Comment 6 Ryan Haddad 2018-08-23 17:17:09 PDT
Updating title to reflect that this isn't limited to the leaks bot, but it does happen far more frequently there.
Comment 7 Truitt Savell 2018-09-04 16:15:57 PDT
Found that these three tests run in succession makes the failure occur around 90% of the time: imported/w3c/web-platform-tests/html/dom/interfaces.html inspector/layers/layer-tree-manager.html inspector/layers/layers-anonymous.html I used the command: run-webkit-tests --root testbuild-235621 imported/w3c/web-platform-tests/html/dom/interfaces.html inspector/layers/layer-tree-manager.html inspector/layers/layers-anonymous.html --child-processes 1 -1 --iterations 10 Using this I was able to begin bisecting the webkit revisions to find the start of the regression. I found the revision that caused these failures to occur to be https://trac.webkit.org/changeset/234330/webkit. If you run the above command on r234329 no failures will occur, if it is ran on r234330 than the failure reproduces.