217414 2020-10-06 16:51:32 -0700 REGRESSION (r267402): [ iOS Mojave+ Release ] imported/w3c/web-platform-tests/user-timing/measure-l3.any.html is a flaky failure 2021-11-16 17:08:40 -0800 1 1 1 Unclassified WebKit New Bugs WebKit Nightly Build Unspecified Unspecified RESOLVED FIXED https://github.com/web-platform-tests/wpt/pull/26192 InRadar P2 Normal --- 1 rackler dpino achristensen clopez dpino ews-watchlist ntim sam webkit-bot-watchers-bugzilla webkit-bug-importer youennf oldest_to_newest 1695387 0 rackler 2020-10-06 16:51:32 -0700 Description: imported/w3c/web-platform-tests/user-timing/measure-l3.any.html The first failure that I see this on the dashboard was 07/29/20 at r265034 for iOS 13, then 08/25/20 began to be consistent passing for iOS and macOS, with a few flaky failures on iOS. On 09/22/20 at r267403 macOS and then iOS began to flaky fail together, but this commit does not appear to be relevant to what would have caused this. I can reproduce this on r267403, but unable to reproduce on r267401 or earlier. This test was modified in the layout test import in r267402, so that probably introduced the flakiness. REPRODUCTION STEPS Command: run-webkit-tests --exit-after-n-failures 1 --exit-after-n-crashes-or-timeouts 1 --iterations 50 imported/w3c/web-platform-tests/user-timing/measure-l3.any.html Result: Regressions: Unexpected text-only failures (1) imported/w3c/web-platform-tests/user-timing/measure-l3.any.html [ Failure ] History: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fuser-timing%2Fmeasure-l3.any.html&platform=ios&platform=mac&limit=50000 Diff Log: Catalina wk2 --- /Volumes/Data/slave/catalina-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/user-timing/measure-l3.any-expected.txt +++ /Volumes/Data/slave/catalina-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/user-timing/measure-l3.any-actual.txt @@ -1,5 +1,5 @@ PASS When the end mark is given and the start is unprovided, the end time of the measure entry should be the end mark's time, the start time should be 0. -PASS When the start mark is given and the end is unprovided, the start time of the measure entry should be the start mark's time, the end should be now. +FAIL When the start mark is given and the end is unprovided, the start time of the measure entry should be the start mark's time, the end should be now. assert_greater_than_equal: expected a number greater than or equal to 9.000000000000002 but got 9 PASS When start and end mark are both given, the start time and end time of the measure entry should be the the marks' time, repectively Mojave wk2 --- /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/user-timing/measure-l3.any-expected.txt +++ /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/user-timing/measure-l3.any-actual.txt @@ -1,5 +1,5 @@ PASS When the end mark is given and the start is unprovided, the end time of the measure entry should be the end mark's time, the start time should be 0. -PASS When the start mark is given and the end is unprovided, the start time of the measure entry should be the start mark's time, the end should be now. +FAIL When the start mark is given and the end is unprovided, the start time of the measure entry should be the start mark's time, the end should be now. assert_greater_than_equal: expected a number greater than or equal to 9.000000000000002 but got 9 PASS When start and end mark are both given, the start time and end time of the measure entry should be the the marks' time, repectively iOS wk2 --- /Volumes/Data/slave/ios-simulator-14-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/user-timing/measure-l3.any-expected.txt +++ /Volumes/Data/slave/ios-simulator-14-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/user-timing/measure-l3.any-actual.txt @@ -1,5 +1,5 @@ PASS When the end mark is given and the start is unprovided, the end time of the measure entry should be the end mark's time, the start time should be 0. -PASS When the start mark is given and the end is unprovided, the start time of the measure entry should be the start mark's time, the end should be now. +FAIL When the start mark is given and the end is unprovided, the start time of the measure entry should be the start mark's time, the end should be now. assert_greater_than_equal: expected a number greater than or equal to 51.00000000000001 but got 51 PASS When start and end mark are both given, the start time and end time of the measure entry should be the the marks' time, repectively 1695388 1 webkit-bug-importer 2020-10-06 16:53:24 -0700 <rdar://problem/70024016> 1695396 2 rackler 2020-10-06 17:16:34 -0700 I have marked this test as failing while this issue is investigated. https://trac.webkit.org/changeset/268093/webkit 1700633 3 412095 dpino 2020-10-22 07:43:46 -0700 Created attachment 412095 Patch 1700635 4 ews-watchlist 2020-10-22 07:44:39 -0700 This patch modifies the imported WPT tests. Please ensure that any changes on the tests (not coming from a WPT import) are exported to WPT. Please see https://trac.webkit.org/wiki/WPTExportProcess 1700637 5 dpino 2020-10-22 07:46:05 -0700 Link to same change in WPT repo: https://github.com/web-platform-tests/wpt/pull/26192 1700817 6 412129 dpino 2020-10-22 13:51:32 -0700 Created attachment 412129 Patch 1701070 7 412129 achristensen 2020-10-23 11:00:42 -0700 Comment on attachment 412129 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=412129&action=review > LayoutTests/imported/w3c/ChangeLog:8 > + Sometimes lhs of comparison is higher by an infinitisimal margin. infinitisimal isn't quite correct but whatevs. 1701071 8 ews-feeder 2020-10-23 11:03:16 -0700 Committed r268928: <https://trac.webkit.org/changeset/268928> All reviewed patches have been landed. Closing bug and clearing flags on attachment 412129. 1718351 9 achristensen 2021-01-08 11:02:49 -0800 (In reply to Diego Pino from comment #5) > Link to same change in WPT repo: > https://github.com/web-platform-tests/wpt/pull/26192 Diego, would you mind including https://bugs.webkit.org/show_bug.cgi?id=217118 in your PR? 1815732 10 dpino 2021-11-16 17:08:40 -0800 (In reply to Alex Christensen from comment #9) > (In reply to Diego Pino from comment #5) > > Link to same change in WPT repo: > > https://github.com/web-platform-tests/wpt/pull/26192 > > Diego, would you mind including > https://bugs.webkit.org/show_bug.cgi?id=217118 in your PR? Thank you Alex, I added your changes to the PR https://github.com/web-platform-tests/wpt/pull/26192 412095 2020-10-22 07:43:46 -0700 2020-10-22 13:51:24 -0700 Patch bug-217414-20201022164345.patch text/plain 2085 dpino U3VidmVyc2lvbiBSZXZpc2lvbjogMjY4ODYzCmRpZmYgLS1naXQgYS9MYXlvdXRUZXN0cy9pbXBv cnRlZC93M2MvQ2hhbmdlTG9nIGIvTGF5b3V0VGVzdHMvaW1wb3J0ZWQvdzNjL0NoYW5nZUxvZwpp bmRleCA2ZDE3Mzc2MGQyNGY1MzMwOTdiNDEwMTU0MDY4MDMzZWQzNTI0MWM1Li41ZjAwMDUyMDhk NjA1ZWE2ZTY0NzMxZGFjMjUyM2JkMjY4Mjg1Yzk0IDEwMDY0NAotLS0gYS9MYXlvdXRUZXN0cy9p bXBvcnRlZC93M2MvQ2hhbmdlTG9nCisrKyBiL0xheW91dFRlc3RzL2ltcG9ydGVkL3czYy9DaGFu Z2VMb2cKQEAgLTEsMyArMSwxNyBAQAorMjAyMC0xMC0yMiAgRGllZ28gUGlubyBHYXJjaWEgIDxk cGlub0BpZ2FsaWEuY29tPgorCisgICAgICAgIFJFR1JFU1NJT04gKHIyNjc0MDIpOiBbIGlPUyBN b2phdmUrIFJlbGVhc2UgXSBpbXBvcnRlZC93M2Mvd2ViLXBsYXRmb3JtLXRlc3RzL3VzZXItdGlt aW5nL21lYXN1cmUtbDMuYW55Lmh0bWwgaXMgYSBmbGFreSBmYWlsdXJlCisgICAgICAgIGh0dHBz Oi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNnaT9pZD0yMTc0MTQKKworICAgICAgICBSZXZp ZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAgICAgICBTb21ldGltZXMgdGhlIHRlc3QgZmFp bHMgYmVjYXVzZSBvbmUgb2YgdGhlIGNvbXBhcmVkIHZhbHVlIGlzIGdyZWF0ZXIgdGhhbgorICAg ICAgICBleHBlY3RlZC4gVGhlIHRlc3QgZXhwZWN0cyBjb21wYXJlZCB2YWx1ZXMgdG8gYmUgZXF1 YWwgdGhvdWdoLiBVc2UKKyAgICAgICAgYXNzZXJ0X2FwcHJveF9lcXVhbHMgaW5zdGVhZCBvZiBh c3NlcnRfZ3JlYXRlcl90aGFuX2VxdWFsLgorCisgICAgICAgICogd2ViLXBsYXRmb3JtLXRlc3Rz L3VzZXItdGltaW5nL21lYXN1cmUtbDMuYW55LmpzOgorICAgICAgICAodGVzdCk6CisKIDIwMjAt MTAtMjIgIE1hcnRpbiBSb2JpbnNvbiAgPG1yb2JpbnNvbkBpZ2FsaWEuY29tPgogCiAgICAgICAg IFNjcm9sbCBzbmFwOiBkb24ndCBjcmVhdGUgaW1wbGljaXQgc25hcCBwb2ludHMgYXQgc2Nyb2xs bWluL3Njcm9sbG1heApkaWZmIC0tZ2l0IGEvTGF5b3V0VGVzdHMvaW1wb3J0ZWQvdzNjL3dlYi1w bGF0Zm9ybS10ZXN0cy91c2VyLXRpbWluZy9tZWFzdXJlLWwzLmFueS5qcyBiL0xheW91dFRlc3Rz L2ltcG9ydGVkL3czYy93ZWItcGxhdGZvcm0tdGVzdHMvdXNlci10aW1pbmcvbWVhc3VyZS1sMy5h bnkuanMKaW5kZXggMjRjMjdjNDgzNTE1ZWQ2NWUzODM3N2Q0ODA4NzdkZDRhZjlmZmRhYi4uNWNj MWJjY2Y3OTE1MWU2N2RhMWEyODNmODEzOWRhZTY5MGVjNWE5OCAxMDA2NDQKLS0tIGEvTGF5b3V0 VGVzdHMvaW1wb3J0ZWQvdzNjL3dlYi1wbGF0Zm9ybS10ZXN0cy91c2VyLXRpbWluZy9tZWFzdXJl LWwzLmFueS5qcworKysgYi9MYXlvdXRUZXN0cy9pbXBvcnRlZC93M2Mvd2ViLXBsYXRmb3JtLXRl c3RzL3VzZXItdGltaW5nL21lYXN1cmUtbDMuYW55LmpzCkBAIC0yMSw4ICsyMSw4IEBAIHRlc3Qo ZnVuY3Rpb24oKSB7CiAgIGNvbnN0IG1lYXN1cmVFbnRyeSA9IHBlcmZvcm1hbmNlLm1lYXN1cmUo IkEiLCAibWFyayIsIHVuZGVmaW5lZCk7CiAgIGNvbnN0IGVuZE1heCA9IHBlcmZvcm1hbmNlLm5v dygpOwogICBhc3NlcnRfZXF1YWxzKG1lYXN1cmVFbnRyeS5zdGFydFRpbWUsIG1hcmtFbnRyeS5z dGFydFRpbWUpOwotICBhc3NlcnRfZ3JlYXRlcl90aGFuX2VxdWFsKGVuZFRpbWUobWVhc3VyZUVu dHJ5KSwgZW5kTWluKTsKLSAgYXNzZXJ0X2dyZWF0ZXJfdGhhbl9lcXVhbChlbmRNYXgsIGVuZFRp bWUobWVhc3VyZUVudHJ5KSk7CisgIGFzc2VydF9hcHByb3hfZXF1YWxzKGVuZFRpbWUobWVhc3Vy ZUVudHJ5KSwgZW5kTWluLCAwLjEpOworICBhc3NlcnRfYXBwcm94X2VxdWFscyhlbmRNYXgsIGVu ZFRpbWUobWVhc3VyZUVudHJ5KSwgMC4xKTsKIH0sICJXaGVuIHRoZSBzdGFydCBtYXJrIGlzIGdp dmVuIGFuZCB0aGUgZW5kIGlzIHVucHJvdmlkZWQsIHRoZSBzdGFydCB0aW1lIG9mIHRoZSBtZWFz dXJlIGVudHJ5IHNob3VsZCBiZSB0aGUgc3RhcnQgbWFyaydzIHRpbWUsIHRoZSBlbmQgc2hvdWxk IGJlIG5vdy4iKTsKIAogdGVzdChmdW5jdGlvbigpIHsK 412129 2020-10-22 13:51:32 -0700 2020-10-23 11:03:17 -0700 Patch bug-217414-20201022225128.patch text/plain 1962 dpino U3VidmVyc2lvbiBSZXZpc2lvbjogMjY4ODg2CmRpZmYgLS1naXQgYS9MYXlvdXRUZXN0cy9pbXBv cnRlZC93M2MvQ2hhbmdlTG9nIGIvTGF5b3V0VGVzdHMvaW1wb3J0ZWQvdzNjL0NoYW5nZUxvZwpp bmRleCBjMTJiNDBjZmI1NTdmZDNmNjE0NDVmZTcxOThhNGNmNTFjMmI5OWUxLi5mODUxYTYwMDlj OGEwNjJmNDNjOTc1M2UyODJiNzZmNGY5YjNjZTJhIDEwMDY0NAotLS0gYS9MYXlvdXRUZXN0cy9p bXBvcnRlZC93M2MvQ2hhbmdlTG9nCisrKyBiL0xheW91dFRlc3RzL2ltcG9ydGVkL3czYy9DaGFu Z2VMb2cKQEAgLTEsMyArMSwxNiBAQAorMjAyMC0xMC0yMiAgRGllZ28gUGlubyBHYXJjaWEgIDxk cGlub0BpZ2FsaWEuY29tPgorCisgICAgICAgIFJFR1JFU1NJT04gKHIyNjc0MDIpOiBbIGlPUyBN b2phdmUrIFJlbGVhc2UgXSBpbXBvcnRlZC93M2Mvd2ViLXBsYXRmb3JtLXRlc3RzL3VzZXItdGlt aW5nL21lYXN1cmUtbDMuYW55Lmh0bWwgaXMgYSBmbGFreSBmYWlsdXJlCisgICAgICAgIGh0dHBz Oi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNnaT9pZD0yMTc0MTQKKworICAgICAgICBSZXZp ZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAgICAgICBTb21ldGltZXMgbGhzIG9mIGNvbXBh cmlzb24gaXMgaGlnaGVyIGJ5IGFuIGluZmluaXRpc2ltYWwgbWFyZ2luLgorICAgICAgICBSb3Vu ZCBudW1iZXIgdG8gdHdvLWRpZ2l0IHByZWNpc2lvbiB0byBwcmV2ZW50IHN1Y2ggc2NlbmFyaW9z LgorCisgICAgICAgICogd2ViLXBsYXRmb3JtLXRlc3RzL3VzZXItdGltaW5nL21lYXN1cmUtbDMu YW55LmpzOgorICAgICAgICAodGVzdCk6CisKIDIwMjAtMTAtMjIgIE5vYW0gUm9zZW50aGFsICA8 bm9hbUB3ZWJraXQub3JnPgogCiAgICAgICAgIFN1cHBvcnQgRVhJRiBpbWFnZSByZXNvbHV0aW9u IGluIENvcmVHcmFwaGljcyAobWFjT1MvaU9TKQpkaWZmIC0tZ2l0IGEvTGF5b3V0VGVzdHMvaW1w b3J0ZWQvdzNjL3dlYi1wbGF0Zm9ybS10ZXN0cy91c2VyLXRpbWluZy9tZWFzdXJlLWwzLmFueS5q cyBiL0xheW91dFRlc3RzL2ltcG9ydGVkL3czYy93ZWItcGxhdGZvcm0tdGVzdHMvdXNlci10aW1p bmcvbWVhc3VyZS1sMy5hbnkuanMKaW5kZXggMjRjMjdjNDgzNTE1ZWQ2NWUzODM3N2Q0ODA4Nzdk ZDRhZjlmZmRhYi4uYjRkNGM2NTA1MzMzOWYyZWUyMmQ1OWE4MDExMTU0OGEwMzMwY2RkMiAxMDA2 NDQKLS0tIGEvTGF5b3V0VGVzdHMvaW1wb3J0ZWQvdzNjL3dlYi1wbGF0Zm9ybS10ZXN0cy91c2Vy LXRpbWluZy9tZWFzdXJlLWwzLmFueS5qcworKysgYi9MYXlvdXRUZXN0cy9pbXBvcnRlZC93M2Mv d2ViLXBsYXRmb3JtLXRlc3RzL3VzZXItdGltaW5nL21lYXN1cmUtbDMuYW55LmpzCkBAIC0xNyw5 ICsxNyw5IEBAIHRlc3QoZnVuY3Rpb24oKSB7CiAgIHBlcmZvcm1hbmNlLmNsZWFyTWFya3MoKTsK ICAgcGVyZm9ybWFuY2UuY2xlYXJNZWFzdXJlcygpOwogICBjb25zdCBtYXJrRW50cnkgPSBwZXJm b3JtYW5jZS5tYXJrKCJtYXJrIiwge3N0YXJ0VGltZTogMTIzfSk7Ci0gIGNvbnN0IGVuZE1pbiA9 IHBlcmZvcm1hbmNlLm5vdygpOworICBjb25zdCBlbmRNaW4gPSBOdW1iZXIocGVyZm9ybWFuY2Uu bm93KCkudG9GaXhlZCgyKSk7CiAgIGNvbnN0IG1lYXN1cmVFbnRyeSA9IHBlcmZvcm1hbmNlLm1l YXN1cmUoIkEiLCAibWFyayIsIHVuZGVmaW5lZCk7Ci0gIGNvbnN0IGVuZE1heCA9IHBlcmZvcm1h bmNlLm5vdygpOworICBjb25zdCBlbmRNYXggPSBOdW1iZXIocGVyZm9ybWFuY2Uubm93KCkudG9G aXhlZCgyKSk7CiAgIGFzc2VydF9lcXVhbHMobWVhc3VyZUVudHJ5LnN0YXJ0VGltZSwgbWFya0Vu dHJ5LnN0YXJ0VGltZSk7CiAgIGFzc2VydF9ncmVhdGVyX3RoYW5fZXF1YWwoZW5kVGltZShtZWFz dXJlRW50cnkpLCBlbmRNaW4pOwogICBhc3NlcnRfZ3JlYXRlcl90aGFuX2VxdWFsKGVuZE1heCwg ZW5kVGltZShtZWFzdXJlRW50cnkpKTsK