Bug 237385

Summary: let vs var performance
Product: WebKit Reporter: jeremy.tellaa
Component: JavaScriptCoreAssignee: Nobody <webkit-unassigned>
Status: RESOLVED INVALID    
Severity: Normal CC: saam, ysuzuki
Priority: P2    
Version: Safari 15   
Hardware: All   
OS: All   

Description jeremy.tellaa 2022-03-02 11:33:28 PST
I have built a very basic POC, and I found that looping when the loop is initialized with `let` instead of `var` is about seven times slower:

```
<!doctype html>
<html lang="en">

<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
  <meta name="theme-color" content="#000000">
  <title>POC</title>
</head>

<body>
  <div id="root"></div>
  <script type="text/javascript">
  const nb = 1000 * 1000 * 1000;
  const t0 = performance.now();
  for (var i = 0; i < nb; i++) { };
  const t1 = performance.now();
  for (let j = 0; j < nb; j++) { };
  const t2 = performance.now();
  const resVar = Math.floor(t1 - t0);
  const resLet = Math.floor(t2 - t1);

  res = `Looping ${nb} times with 'var' took ${resVar}ms.<br>Looping ${nb} times with 'let' took ${resLet}ms.`
  document.getElementById('root').innerHTML = res;
  </script>
</body>

</html>
```
Comment 1 Alexey Proskuryakov 2022-03-02 16:43:13 PST
I cannot reproduce this on Apple Silicon macOS with Safari 15.3. What is the software and hardware that you are seeing this with?

Looping 1000000000 times with 'var' took 2440ms.
Looping 1000000000 times with 'let' took 942ms.

On retry:

Looping 1000000000 times with 'var' took 2466ms.
Looping 1000000000 times with 'let' took 320ms.
Comment 2 Yusuke Suzuki 2022-03-02 17:09:15 PST
Measurement needs to be isolated by a function. And need to ensure that both functions are not inlined. Since both are run in one function, thus, the first one gets penalty because the latter one could get optimizing JIT.
Comment 3 Yusuke Suzuki 2022-03-02 17:21:49 PST
Measuring these very small difference needs very careful crafting of microbenchmarks.

1. The first one is getting a penalty because we inline both and second one gets faster JIT from the beginning.
2. Isolating them into a function is not enough. If we would like to measure this level of small difference, we need to ensure that they are not inlined etc.

Here is the changed benchmark. It is executed via JSC shell since inlining control is available to JSC shell.

```
function varTest(nb) {
  for (var i = 0; i < nb; i++) { };
}
noInline(varTest);
function letTest(nb) {
  for (let j = 0; j < nb; j++) { };
}
noInline(letTest);

const nb = 1000 * 1000 * 1000;

for (var i = 0; i < 10; ++i) {
    varTest(nb);
    letTest(nb);
}

const t0 = Date.now();
varTest(nb);
const t1 = Date.now();
letTest(nb);
const t2 = Date.now();
const resVar = Math.floor(t1 - t0);
const resLet = Math.floor(t2 - t1);

print(`Looping ${nb} times with 'var' took ${resVar}ms.\nLooping ${nb} times with 'let' took ${resLet}ms.`);
```

Then the result is the same.

Looping 1000000000 times with 'var' took 241ms.
Looping 1000000000 times with 'let' took 237ms.

Closing this as it is the same.
Comment 4 jeremy.tellaa 2022-03-03 06:37:07 PST
I am reoping this. I think some optimisation is not made in one cases. I made two benches:

<script type="text/javascript">
  const f = () => { for (let i = 0; i < nb; i++) { } };
  
  const nb = 1000 * 1000 * 1000;
  const t0 = performance.now();
  f();
  const t1 = performance.now();
  const res = Math.floor(t1 - t0);

  const txt = `Looping ${nb} times with took ${res}ms.`
  document.getElementById('root').innerHTML = txt;
</script>

<script type="text/javascript">
  const f = () => { for (var i = 0; i < nb; i++) { } };
  
  const nb = 1000 * 1000 * 1000;
  const t0 = performance.now();
  f();
  const t1 = performance.now();
  const res = Math.floor(t1 - t0);

  const txt = `Looping ${nb} times with took ${res}ms.`
  document.getElementById('root').innerHTML = txt;
</script>

I ran those separately a bunch of times after clicking on "Empty the caches" every time between runs. This first one takes ~10 seconds while the second one ~3 seconds.