Enable --treat-failing-as-flaky for the JSC post-commit bots
Created attachment 457634 [details] Patch
Created attachment 457638 [details] Patch
rdar://92099399
Created attachment 458261 [details] Patch
Created attachment 458262 [details] Patch
Committed r293430 (249990@main): <https://commits.webkit.org/249990@main> All reviewed patches have been landed. Closing bug and clearing flags on attachment 458262 [details].
This has been picked up by the build.webkit.org after yesterday's restart, and there's already been a run with a flaky test: https://build.webkit.org/#/builders/31/builds/3567 This is correctly captured in the per-test results: ❯ curl -sL 'http://results.webkit.org/api/results/javascriptcore-tests/stress/new-largeish-contiguous-array-with-size.js.ram-size-10000000?architecture=mips&uuid=165109908000' | jq . [ { "configuration": { "architecture": "mips", "is_simulator": false, "platform": "linux", "style": "release", "version": 4019000 }, "results": [ { "actual": "PASS", "expected": "PASS", "flakiness_num_passes": 7, "flakiness_num_tries": 8, "start_time": 1651099268, "uuid": 165109908000 } ] } ] However, when I look at the "suite" results, I get: ❯ curl -sL 'http://results.webkit.org/api/results/javascriptcore-tests?architecture=mips&limit=40' | jq -C . | grep build-number | grep 356 "build-number": "3560", "build-number": "3561", "build-number": "3563", "build-number": "3564", "build-number": "3568", "build-number": "3569", I.e. the result for the build number (3567) is missing. That said, I see that e.g. https://build.webkit.org/#/builders/31/builds/3565, which didn't have any flaky tests, is also missing, so I'm not sure what to think. FWIW I've locally verified again that when I submit a report with the current ToT (+ some flakiness injection) my local resultsdb can return the result just fine.
This feels like we dropped processing the build, but I would need to investigate to verify that. Since we have an example, I can put this in the staging instance to check.