Bug 171945 - [iOS Simulator] LayoutTest/perf/class-list-remove.html is flaky.
Summary: [iOS Simulator] LayoutTest/perf/class-list-remove.html is flaky.
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-05-10 13:58 PDT by Matt Lewis
Modified: 2017-08-30 11:54 PDT (History)
4 users (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Matt Lewis 2017-05-10 13:58:45 PDT 
The test perf/class-list-remove.html is a flaky timeout and failure.

History:
https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=perf%2Fclass-list-remove.html

Build:
https://build.webkit.org/builders/Apple%20iOS%2010%20Simulator%20Debug%20WK2%20(Tests)/builds/1240
https://build.webkit.org/results/Apple%20iOS%2010%20Simulator%20Debug%20WK2%20(Tests)/r216603%20(1240)/results.html

Diff of failure:
--- /Volumes/Data/slave/ios-simulator-10-debug-tests-wk2/build/layout-test-results/perf/class-list-remove-expected.txt
+++ /Volumes/Data/slave/ios-simulator-10-debug-tests-wk2/build/layout-test-results/perf/class-list-remove-actual.txt
@@ -1,4 +1,51 @@
 Tests that classList remove is linear.
 PASS
-PASS
+FAIL: got indeterminate result expected O(n)
 
+DEBUG LOG:
+run iteration. magnitude 2 jsObjectCountBefore 105588 jsObjectCountAfter 105298
+run iteration. magnitude 4 jsObjectCountBefore 105298 jsObjectCountAfter 105315
+run iteration. magnitude 8 jsObjectCountBefore 105315 jsObjectCountAfter 105322
+run iteration. magnitude 16 jsObjectCountBefore 105322 jsObjectCountAfter 105342
+run iteration. magnitude 32 jsObjectCountBefore 105342 jsObjectCountAfter 105357
+run iteration. magnitude 64 jsObjectCountBefore 105357 jsObjectCountAfter 105400
+run iteration. magnitude 128 jsObjectCountBefore 105400 jsObjectCountAfter 105467
+run iteration. magnitude 256 jsObjectCountBefore 105467 jsObjectCountAfter 105607
+magnitudes: 2,4,8,16,32,64,128,256
+iterations: 994,1030,1010,1013,981,889,587,596
+numPoints 8 slope 0.00008086324748213563 intercept 0.024469446382383093 rSquared 0.8225402542595746
+numPoints 8 slope 0.003779652272491818 intercept 0.01783512948658312 rSquared 0.6609118475489981
+numPoints 8 slope 0.11690718251801958 intercept -3.9109471008969945 rSquared 0.6820356738794592
+rSquared 0.8225402542595746 rSquaredXLog 0.6609118475489981 rSquaredXYLog 0.6820356738794592
+numTriesLeft: 2
+run iteration. magnitude 2 jsObjectCountBefore 105607 jsObjectCountAfter 105392
+run iteration. magnitude 4 jsObjectCountBefore 105392 jsObjectCountAfter 105402
+run iteration. magnitude 8 jsObjectCountBefore 105402 jsObjectCountAfter 105412
+run iteration. magnitude 16 jsObjectCountBefore 105412 jsObjectCountAfter 105432
+run iteration. magnitude 32 jsObjectCountBefore 105432 jsObjectCountAfter 105445
+run iteration. magnitude 64 jsObjectCountBefore 105445 jsObjectCountAfter 105490
+run iteration. magnitude 128 jsObjectCountBefore 105490 jsObjectCountAfter 105556
+run iteration. magnitude 256 jsObjectCountBefore 105556 jsObjectCountAfter 105696
+magnitudes: 2,4,8,16,32,64,128,256
+iterations: 960,997,1028,979,732,884,747,594
+numPoints 8 slope 0.00006347742910417926 intercept 0.025823377977511646 rSquared 0.8178572027461609
+numPoints 8 slope 0.0030018737046194474 intercept 0.02050673275651004 rSquared 0.6726802825287663
+numPoints 8 slope 0.09527449776697665 intercept -3.8253968259187703 rSquared 0.6903650147940906
+rSquared 0.8178572027461609 rSquaredXLog 0.6726802825287663 rSquaredXYLog 0.6903650147940906
+numTriesLeft: 1
+run iteration. magnitude 2 jsObjectCountBefore 105696 jsObjectCountAfter 105485
+run iteration. magnitude 4 jsObjectCountBefore 105485 jsObjectCountAfter 105495
+run iteration. magnitude 8 jsObjectCountBefore 105495 jsObjectCountAfter 105504
+run iteration. magnitude 16 jsObjectCountBefore 105504 jsObjectCountAfter 105525
+run iteration. magnitude 32 jsObjectCountBefore 105525 jsObjectCountAfter 105543
+run iteration. magnitude 64 jsObjectCountBefore 105543 jsObjectCountAfter 105587
+run iteration. magnitude 128 jsObjectCountBefore 105587 jsObjectCountAfter 105653
+run iteration. magnitude 256 jsObjectCountBefore 105653 jsObjectCountAfter 105793
+magnitudes: 2,4,8,16,32,64,128,256
+iterations: 943,798,1053,982,936,890,702,471
+numPoints 8 slope 0.00010212414422618029 intercept 0.024805807753716977 rSquared 0.896257803279789
+numPoints 8 slope 0.003807491538495301 intercept 0.01944003783602357 rSquared 0.4581830545747703
+numPoints 8 slope 0.10395717425324151 intercept -3.8206076174057735 rSquared 0.46319402420374683
+rSquared 0.896257803279789 rSquaredXLog 0.4581830545747703 rSquaredXYLog 0.46319402420374683
+
+
Comment 1 Matt Lewis 2017-05-10 14:03:32 PDT 
Marked test as flaky on iOS Simulator Debug
https://trac.webkit.org/changeset/216617/webkit/
Comment 2 Alicia Boya García 2017-08-30 08:56:13 PDT 
The test is flaky in GTK too. I'm moving the flakiness expectation to the global TestExpectations...
Comment 3 Matt Lewis 2017-08-30 11:54:09 PDT 
The expectations mentioned by Alicia were changed in:
https://trac.webkit.org/changeset/221382/webkit