Bug 290141
| Summary: | Regression (291871) : Speedometer3 | All devices | .6% Sp3, 1% TodoMVC-JavaScript-ES6-Webpack-Complex-DOM | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Stephanie Lewis <slewis> |
| Component: | CSS | Assignee: | Sam Weinig <sam> |
| Status: | NEW | ||
| Severity: | Normal | CC: | koivisto, m_dubet, sam, slewis, webkit-bug-importer, yulun_wu |
| Priority: | P1 | Keywords: | InRadar |
| Version: | Other | ||
| Hardware: | Mac (Apple Silicon) | ||
| OS: | All | ||
Stephanie Lewis
https://github.com/WebKit/WebKit/commit/c652c23c52e249db4208beb501ce900a24fe0502 is a 1% WebCompact performance on Speedometer3 on Mac.
| Attachments | ||
|---|---|---|
| Add attachment proposed patch, testcase, etc. |
Stephanie Lewis
s/WebCompact/WebComponent
Radar WebKit Bug Importer
<rdar://problem/147585376>
Sam Weinig
The real fix here will be to make the generated parsers more efficient, currently we branch more than some of the hand written ones. I am working on making a structure for this here -> https://github.com/WebKit/WebKit/pull/42846, but probably in the short term, would make sense to add back the hand written code (but leave the generator changes) for the properties changed in that commit:
font-variant-east-asian
font-variant-ligatures
font-variant-numeric
text-autospace
fill
stroke
speak-as
text-indent
text-transform
contain
hanging-punctuation
text-decoration-line
-webkit-text-decorations-in-effect
text-emphasis-position
text-emphasis-style
touch-action
font-width
@font-face font-variant-east-asian
@font-face font-variant-ligatures
@font-face font-variant-numeric
@font-face font-weight
Oof. I'd love to not roll them all back, so I am going to see if I can figure out which ones are actually used by the test.
Sam Weinig
@Stephanie, can you clarify / link to what specific subtest this was on? I have conflicting information from slack that says it is TodoMVC-JavaScript-ES6-Webpack-Complex-DOM that is regressed.
Yulun Wu
Hi Same! I just started as the speedo defense DRI. Our bisections are showing that TodoMVC-JavaScript-ES6-Webpack-Complex-DOM is regressed ~ 1% due to this PR.
Yulun Wu
Hi Sam* not same, apologies for the misspelling :(
Stephanie Lewis
stephanie@Stephanies-MacBook-Pro-2 ~ % /Volumes/Data/Source/OpenSource/Tools/Scripts/compare-results -a ~/Downloads/speedometer3.json ~/Downloads/speedometer3-3.json -b ~/Downloads/speedometer3-2.json ~/Downloads/speedometer3-4.json --detailed
--------------------------------------------------------------------------------------------------------------------------------------------
| subtest | ms | ms | b / a | pValue (significance using False Discovery Rate) |
--------------------------------------------------------------------------------------------------------------------------------------------
| Charts-chartjs-async |6.593000 |6.494750 |0.985098 | 0.423384 |
| Charts-chartjs-sync |18.577750 |18.489250 |0.995236 | 0.806244 |
| Charts-observable-plot-async |8.021250 |7.970000 |0.993611 | 0.473888 |
| Charts-observable-plot-sync |19.437000 |19.688500 |1.012939 | 0.650860 |
| Editor-CodeMirror-async |4.592750 |4.584500 |0.998204 | 0.926573 |
| Editor-CodeMirror-sync |11.382250 |11.453000 |1.006216 | 0.878044 |
| Editor-TipTap-async |3.460750 |3.411500 |0.985769 | 0.545998 |
| Editor-TipTap-sync |38.862250 |39.025000 |1.004188 | 0.744195 |
| NewsSite-Next-async |31.771500 |31.925250 |1.004839 | 0.253521 |
| NewsSite-Next-sync |10.360500 |10.404500 |1.004247 | 0.590057 |
| NewsSite-Nuxt-async |31.657500 |31.582250 |0.997623 | 0.851838 |
| NewsSite-Nuxt-sync |4.866250 |4.898500 |1.006627 | 0.275114 |
| Perf-Dashboard-async |3.949250 |3.959500 |1.002595 | 0.611318 |
| Perf-Dashboard-sync |22.148000 |22.387250 |1.010802 | 0.294758 |
| React-Stockcharts-SVG-async |11.430250 |11.421250 |0.999213 | 0.954645 |
| React-Stockcharts-SVG-sync |34.185000 |34.079500 |0.996914 | 0.830707 |
| TodoMVC-Angular-Complex-DOM-async |4.642750 |4.652250 |1.002046 | 0.918838 |
| TodoMVC-Angular-Complex-DOM-sync |15.729000 |15.711500 |0.998887 | 0.947908 |
| TodoMVC-Backbone-async |4.476750 |4.523000 |1.010331 | 0.247494 |
| TodoMVC-Backbone-sync |10.126750 |10.190500 |1.006295 | 0.706854 |
| TodoMVC-JavaScript-ES5-async |4.192250 |4.204750 |1.002982 | 0.981026 |
| TodoMVC-JavaScript-ES5-sync |16.520000 |16.582750 |1.003798 | 0.673711 |
| TodoMVC-JavaScript-ES6-Webpack-Complex-DOM-async |4.784500 |4.867250 |1.017295 | 0.134699 |
| TodoMVC-JavaScript-ES6-Webpack-Complex-DOM-sync |22.830500 |23.105250 |1.012034 | 0.040272 |
| TodoMVC-Lit-Complex-DOM-async |7.328000 |7.399750 |1.009791 | 0.313719 |
| TodoMVC-Lit-Complex-DOM-sync |1.880500 |1.913500 |1.017549 | 0.173102 |
| TodoMVC-Preact-Complex-DOM-async |5.991500 |6.106000 |1.019110 | 0.081469 |
| TodoMVC-Preact-Complex-DOM-sync |1.607750 |1.611000 |1.002021 | 0.854882 |
| TodoMVC-React-Complex-DOM-async |4.742000 |4.744250 |1.000474 | 0.972780 |
| TodoMVC-React-Complex-DOM-sync |14.969250 |14.954500 |0.999015 | 0.945754 |
| TodoMVC-React-Redux-async |3.139750 |3.155000 |1.004857 | 0.845193 |
| TodoMVC-React-Redux-sync |16.866750 |16.902750 |1.002134 | 0.897234 |
| TodoMVC-Svelte-Complex-DOM-async |5.074750 |5.165000 |1.017784 | 0.421736 |
| TodoMVC-Svelte-Complex-DOM-sync |1.515250 |1.541750 |1.017489 | 0.228285 |
| TodoMVC-Vue-async |10.177000 |10.189500 |1.001228 | 0.930241 |
| TodoMVC-Vue-sync |3.340750 |3.389000 |1.014443 | 0.455318 |
| TodoMVC-WebComponents-async |5.029750 |5.039750 |1.001988 | 0.986965 |
| TodoMVC-WebComponents-sync |4.772250 |4.814750 |1.008906 | 0.380375 |
| TodoMVC-jQuery-async |4.000250 |4.163250 |1.040747 | 0.377080 |
| TodoMVC-jQuery-sync |52.568500 |53.742750 |1.022338 | 0.011403 |
--------------------------------------------------------------------------------------------------------------------------------------------
a mean = 48.88008
b mean = 48.59062
pValue = 0.0483505594
(Bigger means are better.)
1.006 times worse
Results ARE significant
Stephanie Lewis
Let me know if you want more data. WebComponents has pretty high variance atm and showed up in the initial bisection but is not the test that is most regressed here I don't believe
Sam Weinig
* Put another notch in pleas to make the comments monospaced *
Sam Weinig
Pull request: https://github.com/WebKit/WebKit/pull/43514
Sam Weinig
Pull request: https://github.com/WebKit/WebKit/pull/44256
Sam Weinig
Pull request: https://github.com/WebKit/WebKit/pull/44283