RESOLVED CONFIGURATION CHANGED Bug 244395
[ macOS X86_64 wk1 release ] 103x editing/execCommand and fast/domHTMLElement (layout-tests) are flaky text failures - update 
https://bugs.webkit.org/show_bug.cgi?id=244395
Summary [ macOS X86_64 wk1 release ] 103x editing/execCommand and fast/domHTMLElement...
Robert Jenner
Reported 2022-08-26 10:49:31 PDT
Then following 103 editing/execCommand and fast/dom/HTMLElement layout-tests are flaky text failures on macOS BigSur and higher on wk1 release Intel only: editing/execCommand/primitive-value-cleanup-minimal.html editing/execCommand/primitive-value.html editing/execCommand/print.html editing/execCommand/query-command-state.html editing/execCommand/query-command-value-background-color.html editing/execCommand/query-font-size-with-typing-style.html editing/execCommand/query-font-size.html editing/execCommand/query-format-block.html editing/execCommand/query-text-alignment.html editing/execCommand/query-text-decoration-with-typing-style.html editing/execCommand/queryCommandState-01.html editing/execCommand/queryCommandState-02.html editing/execCommand/queryCommandValue-unsupported-commands.html editing/execCommand/remove-format-background-color.html editing/execCommand/remove-format-elements.html editing/execCommand/remove-format-image.html editing/execCommand/remove-format-in-table-cell.html editing/execCommand/remove-format-multiple-elements-mac.html editing/execCommand/remove-format-multiple-elements-win.html editing/execCommand/remove-format-non-html-element-crash.html editing/execCommand/remove-format-orphaned-list-item.html editing/execCommand/remove-format-textdecoration-in-iframe.html editing/execCommand/remove-formatting-2.html editing/execCommand/remove-formatting-from-iframe-in-button.html editing/execCommand/remove-formatting.html editing/execCommand/remove-list-1.html editing/execCommand/remove-list-from-multi-list-items.html editing/execCommand/remove-list-from-range-selection.html editing/execCommand/remove-list-item-1.html editing/execCommand/remove-list-items.html editing/execCommand/remove-node-during-command-crash.html editing/execCommand/reset-direction-after-breaking-blockquote.html editing/execCommand/reset-values-after-navigation.html editing/execCommand/selectAll-copy-crash.html editing/execCommand/selectAll-including-marquee-crash.html editing/execCommand/selectAll.html editing/execCommand/set-backColor-with-color-filter-from-scripts.html editing/execCommand/set-foreColor-with-color-filter-from-scripts.html editing/execCommand/strikethrough-uses-strike-tag.html editing/execCommand/strikethroughSelection.html editing/execCommand/style-with-css.html editing/execCommand/switch-list-type-with-inner-list.html editing/execCommand/switch-list-type-with-orphaned-li.html editing/execCommand/switch-list-type.html editing/execCommand/switch-multiple-list-items-crash.html editing/execCommand/switch-multiple-list-items.html editing/execCommand/toggle-compound-styles.html editing/execCommand/toggle-link-mac.html editing/execCommand/toggle-link-win.html editing/execCommand/toggle-mixed-text-decorations.html editing/execCommand/toggle-style-2.html editing/execCommand/toggle-style-3.html editing/execCommand/toggle-styles.html editing/execCommand/toggle-text-decorations.html editing/execCommand/toggle-unlink-mac.html editing/execCommand/toggle-unlink-win.html editing/execCommand/transpose-backslash-with-euc.html editing/execCommand/typing-should-not-trigger-scrolling-when-selection-is-visible.html editing/execCommand/underline-selection-containing-image.html editing/execCommand/unindent-nested-blockquote-with-inner-div.html editing/execCommand/unlink.html editing/execCommand/use-css.html editing/execCommand/value-without-selection-crash.html fast/dom/HTMLElement/attr-dir-auto-change-before-text-node.html fast/dom/HTMLElement/attr-dir-auto-change-child-node.html fast/dom/HTMLElement/attr-dir-auto-change-text-form-control.html fast/dom/HTMLElement/attr-dir-auto-change-text.html fast/dom/HTMLElement/attr-dir-auto-children.html fast/dom/HTMLElement/attr-dir-auto-remove-add-children.html fast/dom/HTMLElement/attr-dir-auto-text-form-control-child.html fast/dom/HTMLElement/attr-dir-auto-text-form-control.html fast/dom/HTMLElement/attr-dir-auto.html fast/dom/HTMLElement/attr-dir-value-change.html fast/dom/HTMLElement/attr-empty-string.html fast/dom/HTMLElement/attr-false-string.html fast/dom/HTMLElement/attr-invalid-string.html fast/dom/HTMLElement/attr-missing-ancestor-false.html fast/dom/HTMLElement/attr-missing-ancestor-true.html fast/dom/HTMLElement/attr-missing-parent-ancestor-missing.html fast/dom/HTMLElement/attr-missing-parent-false.html fast/dom/HTMLElement/attr-missing-parent-true.html fast/dom/HTMLElement/attr-true-string.html fast/dom/HTMLElement/bdo.html fast/dom/HTMLElement/class-list-gc.html fast/dom/HTMLElement/class-list-quirks.html fast/dom/HTMLElement/dynamic-editability-change.html fast/dom/HTMLElement/innerHTML-selection-crash.html fast/dom/HTMLElement/insertAdjacentHTML-errors.html fast/dom/HTMLElement/iscontenteditable-designmodeon-allinherit-subframe.html fast/dom/HTMLElement/iscontenteditable-designmodeon-allinherit.html fast/dom/HTMLElement/iscontenteditable-designmodeon-ancestor.html fast/dom/HTMLElement/iscontenteditable-designmodeon.html fast/dom/HTMLElement/set-and-clear-dir-attribute.html fast/dom/HTMLElement/set-false.html fast/dom/HTMLElement/set-inherit-parent-false.html fast/dom/HTMLElement/set-inherit-parent-true.html fast/dom/HTMLElement/set-inner-outer-optimization.html fast/dom/HTMLElement/set-invalid-value.html fast/dom/HTMLElement/set-true.html fast/dom/HTMLElement/set-value-caseinsensitive.html fast/dom/HTMLElement/spellcheck.html fast/dom/HTMLElement/translate.html I have verified that when all of these tests fail, they fail together. Below is a link to the results of one of the test runs where the failures occurred together: https://build.webkit.org/results/Apple-Monterey-Release-WK1-Tests/253721@main%20(5074)/results.html Additionally, I have verified that all of these tests failure history match. So instead of one large link that contains history for 103 tests that will take forever to load you can see the history with just this one test, and I have verified that it is the same for all 103: https://results.webkit.org/?suite=layout-tests&test=editing%2FexecCommand%2Fprimitive-value-cleanup-minimal.html&flavor=wk1&architecture=x86_64&style=release
Attachments
Test List for reproduction (27.44 KB, text/plain)
2022-08-26 11:02 PDT, Robert Jenner 		no flags
Details
View All Add attachment proposed patch, testcase, etc.
Robert Jenner
Comment 1 2022-08-26 10:51:12 PDT
<rdar://problem/94300021>
Robert Jenner
Comment 2 2022-08-26 10:56:40 PDT
I was able to reproduce the failures running in a test list in a specific order. I have attached the test list to this bug. Running it as follows on Monterey Production ToT: Run-webkit-tests --test-list <path to test list goes here> --child-process=1 -1 This is specific to Intel machines, and it does not reproduce on Apple Silicon.
Robert Jenner
Comment 3 2022-08-26 11:02:28 PDT
Created attachment 461887 [details] Test List for reproduction
Robert Jenner
Comment 4 2022-08-26 11:05:58 PDT
Being that this reproduces only in a test list, that indicates that this may be a test-list order issue. So I began to eliminate tests to see if I could determine which caused this, and it appears so far that the following test: "editing/execCommand/paste-as-quotation-disconnected-paragraph-ancestor-crash.html" When run before the others in this list are what causes all of these to fail. That test was introduced here by an external contributor: https://commits.webkit.org/246072@main I ran the test list without the test mentioned above in this comment in a loop for 24 hours, and in that time the failures never occurred.
EWS
Comment 5 2022-08-26 11:17:39 PDT
Test gardening commit 253825@main (dec9b492fbb9): <https://commits.webkit.org/253825@main> Reviewed commits have been landed. Closing PR #3707 and removing active labels.
Robert Jenner
Comment 6 2022-08-26 11:31:31 PDT
I have skipped this test in the commit above on all Mac wk1 Intel queues to stop the failures from occurring. Starting this on Frédéric who introduced the test that started causing the failures in the commit below: https://commits.webkit.org/246072@main
Robert Jenner
Comment 7 2022-09-15 13:00:24 PDT
I think this was actually all addressed in: https://bugs.webkit.org/show_bug.cgi?id=235733 Updating the expectation to include all release Mac wk1.
EWS
Comment 8 2022-09-15 13:10:54 PDT
Test gardening commit 254527@main (5c92b974c3d1): <https://commits.webkit.org/254527@main> Reviewed commits have been landed. Closing PR #4398 and removing active labels.
Note You need to log in before you can comment on or make changes to this bug.