Bug 153049 – [iOS Simulator] perf/array-binary-search.html is flaky

• Status: REOPENED
• Component: New Bugs

[Ryan Haddad]

[iOS Simulator] perf/array-binary-search.html failing

<https://build.webkit.org/results/Apple%20iOS%209%20Simulator%20Release%20WK2%20(Tests)/r194934%20(2257)/results.html>

--- /Volumes/Data/slave/ios-simulator-9-release-tests-wk2/build/layout-test-results/perf/array-binary-search-expected.txt
+++ /Volumes/Data/slave/ios-simulator-9-release-tests-wk2/build/layout-test-results/perf/array-binary-search-actual.txt
@@ -1,3 +1,50 @@
 Tests that binary search over an array is logarithmic.
-PASS
+FAIL: got >=O(n^2) expected O(log n)
 
+DEBUG LOG:
+run iteration. magnitude 2 jsObjectCountBefore 3655 jsObjectCountAfter 3540
+run iteration. magnitude 4 jsObjectCountBefore 3540 jsObjectCountAfter 3584
+run iteration. magnitude 8 jsObjectCountBefore 3584 jsObjectCountAfter 3596
+run iteration. magnitude 16 jsObjectCountBefore 3596 jsObjectCountAfter 3603
+run iteration. magnitude 32 jsObjectCountBefore 3603 jsObjectCountAfter 3612
+run iteration. magnitude 64 jsObjectCountBefore 3612 jsObjectCountAfter 3619
+run iteration. magnitude 128 jsObjectCountBefore 3619 jsObjectCountAfter 3626
+run iteration. magnitude 256 jsObjectCountBefore 3626 jsObjectCountAfter 3633
+magnitudes: 2,4,8,16,32,64,128,256
+iterations: 102711,64086,60253,45993,40249,35192,30793,19499
+numPoints 8 slope 0.0000034935292346429116 intercept 0.00040447404823461535 rSquared 0.92294096258322
+numPoints 8 slope 0.00017765513290951202 intercept 0.00007305134174606317 rSquared 0.8777800735838863
+numPoints 8 slope 0.2926986028378953 intercept -8.398446560235858 rSquared 0.956647199824382
+rSquared 0.92294096258322 rSquaredXLog 0.8777800735838863 rSquaredXYLog 0.956647199824382
+numTriesLeft: 2
+run iteration. magnitude 2 jsObjectCountBefore 3633 jsObjectCountAfter 3693
+run iteration. magnitude 4 jsObjectCountBefore 3693 jsObjectCountAfter 3700
+run iteration. magnitude 8 jsObjectCountBefore 3700 jsObjectCountAfter 3711
+run iteration. magnitude 16 jsObjectCountBefore 3711 jsObjectCountAfter 3718
+run iteration. magnitude 32 jsObjectCountBefore 3718 jsObjectCountAfter 3725
+run iteration. magnitude 64 jsObjectCountBefore 3725 jsObjectCountAfter 3727
+run iteration. magnitude 128 jsObjectCountBefore 3727 jsObjectCountAfter 3734
+run iteration. magnitude 256 jsObjectCountBefore 3734 jsObjectCountAfter 3741
+magnitudes: 2,4,8,16,32,64,128,256
+iterations: 103663,79335,61570,49160,40446,29983,27193,24267
+numPoints 8 slope 0.0000028626943159894496 intercept 0.0004265463303606754 rSquared 0.7527807950683167
+numPoints 8 slope 0.00017067266447015172 intercept 0.00007668735021237644 rSquared 0.984082608680832
+numPoints 8 slope 0.3069426284412647 intercept -8.474127703521189 rSquared 0.9841747406912329
+rSquared 0.7527807950683167 rSquaredXLog 0.984082608680832 rSquaredXYLog 0.9841747406912329
+numTriesLeft: 1
+run iteration. magnitude 2 jsObjectCountBefore 3741 jsObjectCountAfter 3786
+run iteration. magnitude 4 jsObjectCountBefore 3786 jsObjectCountAfter 3793
+run iteration. magnitude 8 jsObjectCountBefore 3793 jsObjectCountAfter 3799
+run iteration. magnitude 16 jsObjectCountBefore 3799 jsObjectCountAfter 3806
+run iteration. magnitude 32 jsObjectCountBefore 3806 jsObjectCountAfter 3813
+run iteration. magnitude 64 jsObjectCountBefore 3813 jsObjectCountAfter 3820
+run iteration. magnitude 128 jsObjectCountBefore 3820 jsObjectCountAfter 3828
+run iteration. magnitude 256 jsObjectCountBefore 3828 jsObjectCountAfter 3835
+magnitudes: 2,4,8,16,32,64,128,256
+iterations: 106327,75886,61968,43091,36786,26133,21191,24120
+numPoints 8 slope 0.000003086106968348591 intercept 0.0004783417341308045 rSquared 0.6073273097639971
+numPoints 8 slope 0.0001995748388574566 intercept 0.0000525747374716456 rSquared 0.9341088245898135
+numPoints 8 slope 0.33510161188908394 intercept -8.482943010702595 rSquared 0.9513112135944649
+rSquared 0.6073273097639971 rSquaredXLog 0.9341088245898135 rSquaredXYLog 0.9513112135944649
+
+

[Ryan Haddad]

Now that we have flakiness dashboard results for iOS simulator results, it looks like this test has only failed once:
<http://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=perf%2Farray-binary-search.html>

[Jiewen Tan]

Committed r199580: <http://trac.webkit.org/changeset/199580>

[Jiewen Tan]

(In reply to comment #2)
> Committed r199580: <http://trac.webkit.org/changeset/199580>

Mark it as flaky on ios-simulator-wk2.

Reopen to counter auto-closing.