Bug 270262 – [WASM] BBQ JIT fails to propagate NaN through f32 min/max

• Status: RESOLVED
• Resolution: FIXED
• Component: WebAssembly

[Marcus Plutowski]

rdar://120540053

JS PoC:

```
// (module
//     (type (func (param f32) (result f32)))
//     (func (type 0) (local f32)
//         (local.set 1 (f32.min (f32.const 12345678) (f32.const nan)))
//         (local.get 1)
//     )
//   (export "poc" (func 0))
// )
const wasm_code = new Uint8Array([0, 97, 115, 109, 1, 0, 0, 0, 1, 6, 1, 96, 1, 125, 1, 125, 3, 2, 1, 0, 7, 7, 1, 3, 112, 111, 99, 0, 0, 10, 21, 1, 19, 1, 1, 125, 67, 78, 97, 60, 75, 67, 0, 0, 192, 127, 150, 33, 1, 32, 1, 11]);
try {
    const wasm_mod = new WebAssembly.Module(wasm_code);
    const wasm_instance = new WebAssembly.Instance(wasm_mod);
    let ret = undefined;
    for (let i = 0; i < 10000; i++) {
        ret = wasm_instance.exports.poc(1.1);
    }
    print(ret);
}
catch (e) {
}
```

When JIT'ed, it returns sometime either NaN or 12345678. See below:

```
$ jsc --jitPolicyScale=0.0 --useConcurrentJIT=true --useBBQJIT=true ./poc.js
NaN

$ jsc --jitPolicyScale=0.0 --useConcurrentJIT=true --useBBQJIT=true ./poc.js
12345678

$ jsc --jitPolicyScale=0.0 --useConcurrentJIT=true --useBBQJIT=true ./poc.js
NaN

$ jsc --jitPolicyScale=0.0 --useConcurrentJIT=true --useBBQJIT=true ./poc.js
NaN

$ jsc --jitPolicyScale=0.0 --useConcurrentJIT=true --useBBQJIT=true ./poc.js
12345678

$ jsc --jitPolicyScale=0.0 --useConcurrentJIT=true --useBBQJIT=true ./poc.js
NaN

$ jsc --jitPolicyScale=0.0 --useConcurrentJIT=true --useBBQJIT=true ./poc.js
12345678
```

When we disable the concurrent JIT it works as expected:

```
$ jsc --jitPolicyScale=0.0 --useConcurrentJIT=false --useBBQJIT=true ./poc.js
NaN
```

And with no JIT:

```
$ jsc --useWasmLLInt=true --useBBQJIT=false --useOMGJIT=false ./poc.js 
NaN
```

[Marcus Plutowski]

Pull request: https://github.com/WebKit/WebKit/pull/27460

[EWS]

Committed 277891@main (d449735fb880): <https://commits.webkit.org/277891@main>

Reviewed commits have been landed. Closing PR #27460 and removing active labels.