234105 2021-12-09 14:44:06 -0800 REGRESSION (r286259): [ BigSur+ wk1 ] 2X imported/w3c/web-platform-tests/speech-api/SpeechSynthesis (layout-tests) are constant text failures 2021-12-14 13:49:28 -0800 1 1 1 Unclassified WebKit Accessibility Other Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=231895 InRadar P2 Normal --- 1 jenner webkit-unassigned andresg_22 cfleizach webkit-bot-watchers-bugzilla webkit-bug-importer oldest_to_newest 1822350 0 jenner 2021-12-09 14:44:06 -0800 The following two tests are constant text failures on Big Sur and later in WK 1: imported/w3c/web-platform-tests/speech-api/SpeechSynthesis-speak-events.html imported/w3c/web-platform-tests/speech-api/SpeechSynthesis-pause-resume.tentative.html HISTORY: https://results.webkit.org/?suite=layout-tests&suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fspeech-api%2FSpeechSynthesis-pause-resume.tentative.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fspeech-api%2FSpeechSynthesis-speak-events.html DIF URLS: https://build.webkit.org/results/Apple-Monterey-Release-WK1-Tests/r286796%20(580)/imported/w3c/web-platform-tests/speech-api/SpeechSynthesis-pause-resume.tentative-pretty-diff.html https://build.webkit.org/results/Apple-Monterey-Release-WK1-Tests/r286796%20(580)/imported/w3c/web-platform-tests/speech-api/SpeechSynthesis-speak-events-pretty-diff.html TEXT DIFFS: FOR "imported/w3c/web-platform-tests/speech-api/SpeechSynthesis-pause-resume.tentative.html": -PASS speechSynthesis.pause() and resume() state and events +Harness Error (TIMEOUT), message = null +TIMEOUT speechSynthesis.pause() and resume() state and events Test timed out + FOR "imported/w3c/web-platform-tests/speech-api/SpeechSynthesis-speak-events.html": -FAIL speechSynthesis.speak() fires start and end events with empty utterance assert_true: Not expecting event, but got end event expected true got false +FAIL speechSynthesis.speak() fires start and end events with empty utterance assert_true: Not expecting event, but got start event expected true got false FAIL speechSynthesis.speak() fires start and end events assert_true: Not expecting event, but got start event expected true got false 1822351 1 webkit-bug-importer 2021-12-09 14:44:30 -0800 <rdar://problem/86292065> 1822362 2 jenner 2021-12-09 14:56:20 -0800 I was able to reproduce the failure at Monterey Production ToT using the following test: run-webkit-tests <test name goes here> -1 I was able to also discover the regression point. The history shows that these tests weren't running on wk1 prior to r286259. When I ran the same test at r286258, I found that the tests were passing. They started failing after the introduction of r286259: https://trac.webkit.org/changeset/286259/webkit 1822367 3 cfleizach 2021-12-09 15:01:34 -0800 (In reply to Robert Jenner from comment #2) > I was able to reproduce the failure at Monterey Production ToT using the > following test: > > run-webkit-tests <test name goes here> -1 > > > > I was able to also discover the regression point. The history shows that > these tests weren't running on wk1 prior to r286259. When I ran the same > test at r286258, I found that the tests were passing. They started failing > after the introduction of r286259: > > https://trac.webkit.org/changeset/286259/webkit I re-enabled them with this change, but I'm testing on 12.0. I think we should skip on Big Sur and just run on Monterey if this is the case 1822466 4 jenner 2021-12-09 17:47:25 -0800 (In reply to chris fleizach from comment #3) > (In reply to Robert Jenner from comment #2) > > I was able to reproduce the failure at Monterey Production ToT using the > > following test: > > > > run-webkit-tests <test name goes here> -1 > > > > > > > > I was able to also discover the regression point. The history shows that > > these tests weren't running on wk1 prior to r286259. When I ran the same > > test at r286258, I found that the tests were passing. They started failing > > after the introduction of r286259: > > > > https://trac.webkit.org/changeset/286259/webkit > > I re-enabled them with this change, but I'm testing on 12.0. I think we > should skip on Big Sur and just run on Monterey if this is the case Okay, that sounds good for BigSur. I will get them skipped for wk1 there. What should be done for it failing on wk1 Monterey? 1822524 5 cfleizach 2021-12-09 22:47:16 -0800 (In reply to Robert Jenner from comment #4) > (In reply to chris fleizach from comment #3) > > (In reply to Robert Jenner from comment #2) > > > I was able to reproduce the failure at Monterey Production ToT using the > > > following test: > > > > > > run-webkit-tests <test name goes here> -1 > > > > > > > > > > > > I was able to also discover the regression point. The history shows that > > > these tests weren't running on wk1 prior to r286259. When I ran the same > > > test at r286258, I found that the tests were passing. They started failing > > > after the introduction of r286259: > > > > > > https://trac.webkit.org/changeset/286259/webkit > > > > I re-enabled them with this change, but I'm testing on 12.0. I think we > > should skip on Big Sur and just run on Monterey if this is the case > > > > Okay, that sounds good for BigSur. I will get them skipped for wk1 there. > > What should be done for it failing on wk1 Monterey? Can you skip all WK1. I will look into the Monterey case https://bugs.webkit.org/show_bug.cgi?id=234125 I feel like that should pass without issue 1823740 6 jenner 2021-12-13 14:05:32 -0800 (In reply to chris fleizach from comment #5) > (In reply to Robert Jenner from comment #4) > > (In reply to chris fleizach from comment #3) > > > (In reply to Robert Jenner from comment #2) > > > > I was able to reproduce the failure at Monterey Production ToT using the > > > > following test: > > > > > > > > run-webkit-tests <test name goes here> -1 > > > > > > > > > > > > > > > > I was able to also discover the regression point. The history shows that > > > > these tests weren't running on wk1 prior to r286259. When I ran the same > > > > test at r286258, I found that the tests were passing. They started failing > > > > after the introduction of r286259: > > > > > > > > https://trac.webkit.org/changeset/286259/webkit > > > > > > I re-enabled them with this change, but I'm testing on 12.0. I think we > > > should skip on Big Sur and just run on Monterey if this is the case > > > > > > > > Okay, that sounds good for BigSur. I will get them skipped for wk1 there. > > > > What should be done for it failing on wk1 Monterey? > > Can you skip all WK1. I will look into the Monterey case > https://bugs.webkit.org/show_bug.cgi?id=234125 > > I feel like that should pass without issue Disabled for all Mac wk1 here: https://trac.webkit.org/changeset/286980/webkit 1823741 7 cfleizach 2021-12-13 14:07:43 -0800 (In reply to Robert Jenner from comment #6) > (In reply to chris fleizach from comment #5) > > (In reply to Robert Jenner from comment #4) > > > (In reply to chris fleizach from comment #3) > > > > (In reply to Robert Jenner from comment #2) > > > > > I was able to reproduce the failure at Monterey Production ToT using the > > > > > following test: > > > > > > > > > > run-webkit-tests <test name goes here> -1 > > > > > > > > > > > > > > > > > > > > I was able to also discover the regression point. The history shows that > > > > > these tests weren't running on wk1 prior to r286259. When I ran the same > > > > > test at r286258, I found that the tests were passing. They started failing > > > > > after the introduction of r286259: > > > > > > > > > > https://trac.webkit.org/changeset/286259/webkit > > > > > > > > I re-enabled them with this change, but I'm testing on 12.0. I think we > > > > should skip on Big Sur and just run on Monterey if this is the case > > > > > > > > > > > > Okay, that sounds good for BigSur. I will get them skipped for wk1 there. > > > > > > What should be done for it failing on wk1 Monterey? > > > > Can you skip all WK1. I will look into the Monterey case > > https://bugs.webkit.org/show_bug.cgi?id=234125 > > > > I feel like that should pass without issue > > > Disabled for all Mac wk1 here: > > https://trac.webkit.org/changeset/286980/webkit Thanks for your help