38446 2010-05-02 20:55:35 -0700 SunSpider: all four bitops benchmarks can be replaced with NOP 2011-07-02 14:09:50 -0700 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) PC OS X 10.5 RESOLVED FIXED P2 Normal --- 0 nnethercote mjs ap brendan mjs oliver oldest_to_newest 220004 0 nnethercote 2010-05-02 20:55:35 -0700 There are four bitops benchmarks in SunSpider: 3bit-bits-in-byte, bits-in-byte, bitwise-and, nsieve-bits. All of them compute things but never do anything with the computed values which means a sufficiently clever compiler can optimise them all down to a single NOP. I noticed this because I'm working on improving dead code elimination for Mozilla's TraceMonkey, which is now sufficiently clever to turn this (from 3bit-bits-in-byte): for(var y=0; y<256; y++) func(y); into an empty loop, because func() has no side effects and its result isn't used. While this provides me personally with a certain level of satisfaction it doesn't seem a desirable property for a benchmark. Note that some of these contain empty loops like this: for(var x=0; x<500; x++) which are presumably there to "spin up" the benchmark, but can also be eliminated entirely. It's possible that some of the other SunSpider benchmarks have the same undesirable property, I haven't checked; the bitops ones are certainly the smallest and so the most affected. They should be changed to record a value and print it out at the end, something like that. 226555 1 nnethercote 2010-05-16 15:32:02 -0700 I noticed recently that string-validate-input.js also has this property. Because that benchmark is dominated by string concatenation, a JS engine can over-optimize this benchmark by applying a lazy concatenation approach -- ie. build a tree of to-be-concatenated strings and then concatenate them when needed. Except in this string-validate-input.js the final string is never used so the actual final concatenation step is not needed. The final string should be used. Printing the whole thing out is a bad idea, it's 420,000 chars and so I/O costs would dominate. Maybe printing out every 1000th char would be ok? That provide a reasonable amount of "use" of the final string so that over-optimizing the benchmark would be difficult. Or computing a really simple hash of the string would be a more thorough use. bitops-bitwise-and.js provides an example of a really simple hash that could be used -- just '&' together every char. 431627 2 mjs 2011-07-02 13:06:21 -0700 (In reply to comment #1) > I noticed recently that string-validate-input.js also has this property. Because that benchmark is dominated by string concatenation, a JS engine can over-optimize this benchmark by applying a lazy concatenation approach -- ie. build a tree of to-be-concatenated strings and then concatenate them when needed. Except in this string-validate-input.js the final string is never used so the actual final concatenation step is not needed. > > The final string should be used. Printing the whole thing out is a bad idea, it's 420,000 chars and so I/O costs would dominate. Maybe printing out every 1000th char would be ok? That provide a reasonable amount of "use" of the final string so that over-optimizing the benchmark would be difficult. Or computing a really simple hash of the string would be a more thorough use. bitops-bitwise-and.js provides an example of a really simple hash that could be used -- just '&' together every char. I filed bug 63864 for this latter issue. 431629 3 99558 mjs 2011-07-02 13:34:41 -0700 Created attachment 99558 Patch 431636 4 mjs 2011-07-02 14:09:50 -0700 Committed r90319: <http://trac.webkit.org/changeset/90319> 99558 2011-07-02 13:34:41 -0700 2011-07-02 14:01:49 -0700 Patch bug-38446-20110702133439.patch text/plain 3556 mjs SW5kZXg6IFBlcmZvcm1hbmNlVGVzdHMvU3VuU3BpZGVyL0NoYW5nZUxvZwo9PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0t LSBQZXJmb3JtYW5jZVRlc3RzL1N1blNwaWRlci9DaGFuZ2VMb2cJKHJldmlzaW9uIDkwMzE2KQor KysgUGVyZm9ybWFuY2VUZXN0cy9TdW5TcGlkZXIvQ2hhbmdlTG9nCSh3b3JraW5nIGNvcHkpCkBA IC0xLDMgKzEsMTkgQEAKKzIwMTEtMDctMDIgIE1hY2llaiBTdGFjaG93aWFrICA8bWpzQGFwcGxl LmNvbT4KKworICAgICAgICBSZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAgICAgICBT dW5TcGlkZXI6IGFsbCBmb3VyIGJpdG9wcyBiZW5jaG1hcmtzIGNhbiBiZSByZXBsYWNlZCB3aXRo IE5PUAorICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9zaG93X2J1Zy5jZ2k/aWQ9Mzg0 NDYKKworICAgICAgICAqIHRlc3RzL3N1bnNwaWRlci0xLjAvYml0b3BzLTNiaXQtYml0cy1pbi1i eXRlLmpzOgorICAgICAgICAoVGltZUZ1bmMpOiBTYXZlIHRoZSByZXN1bHQuCisgICAgICAgICog dGVzdHMvc3Vuc3BpZGVyLTEuMC9iaXRvcHMtYml0cy1pbi1ieXRlLmpzOgorICAgICAgICAoVGlt ZUZ1bmMpOiBTYXZlIHRoZSByZXN1bHQuCisgICAgICAgICogdGVzdHMvc3Vuc3BpZGVyLTEuMC9i aXRvcHMtYml0d2lzZS1hbmQuanM6IE1vcmUgZXhwbGljaXRseQorICAgICAgICBzYXZlIHRoZSBy ZXN1bHRzLgorICAgICAgICAqIHRlc3RzL3N1bnNwaWRlci0xLjAvYml0b3BzLW5zaWV2ZS1iaXRz LmpzOgorICAgICAgICAoc2lldmUpOiBTYXZlIHRoZSByZXN1bHQuCisKIDIwMTEtMDYtMzAgIE1h Y2llaiBTdGFjaG93aWFrICA8bWpzQGFwcGxlLmNvbT4KIAogICAgICAgICBSZXZpZXdlZCBieSBB ZGFtIEJhcnRoLgpJbmRleDogUGVyZm9ybWFuY2VUZXN0cy9TdW5TcGlkZXIvdGVzdHMvc3Vuc3Bp ZGVyLTEuMC9iaXRvcHMtM2JpdC1iaXRzLWluLWJ5dGUuanMKPT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gUGVyZm9y bWFuY2VUZXN0cy9TdW5TcGlkZXIvdGVzdHMvc3Vuc3BpZGVyLTEuMC9iaXRvcHMtM2JpdC1iaXRz LWluLWJ5dGUuanMJKHJldmlzaW9uIDkwMjIxKQorKysgUGVyZm9ybWFuY2VUZXN0cy9TdW5TcGlk ZXIvdGVzdHMvc3Vuc3BpZGVyLTEuMC9iaXRvcHMtM2JpdC1iaXRzLWluLWJ5dGUuanMJKHdvcmtp bmcgY29weSkKQEAgLTEsNSArMSw3IEBACiAvLyBDb3B5cmlnaHQgKGMpIDIwMDQgYnkgQXJ0aHVy IExhbmdlcmVpcyAoYXJ0aHVyX2V4dCBhdCBkb21haW4geGZpbml0ZWdhbWVzLCB0bGQgY29tCiAK K3ZhciByZXN1bHQgPSAwOworCiAvLyAxIG9wID0gNiBBTkRzLCAzIFNIUnMsIDMgU0hMcywgNCBh c3NpZ25zLCAyIEFERHMKIC8vIE8oMSkKIGZ1bmN0aW9uIGZhc3QzYml0bG9va3VwKGIpIHsKQEAg LTI1LDggKzI3LDEwIEBAIGFkZHIzLHIzLHIxMAogCiBmdW5jdGlvbiBUaW1lRnVuYyhmdW5jKSB7 CiB2YXIgeCwgeSwgdDsKK3ZhciBzdW0gPSAwOwogZm9yKHZhciB4PTA7IHg8NTAwOyB4KyspCi1m b3IodmFyIHk9MDsgeTwyNTY7IHkrKykgZnVuYyh5KTsKK2Zvcih2YXIgeT0wOyB5PDI1NjsgeSsr KSBzdW0gKz0gZnVuYyh5KTsKK3JldHVybiBzdW07CiB9CiAKLVRpbWVGdW5jKGZhc3QzYml0bG9v a3VwKTsKK3N1bSA9IFRpbWVGdW5jKGZhc3QzYml0bG9va3VwKTsKSW5kZXg6IFBlcmZvcm1hbmNl VGVzdHMvU3VuU3BpZGVyL3Rlc3RzL3N1bnNwaWRlci0xLjAvYml0b3BzLWJpdHMtaW4tYnl0ZS5q cwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09Ci0tLSBQZXJmb3JtYW5jZVRlc3RzL1N1blNwaWRlci90ZXN0cy9zdW5zcGlk ZXItMS4wL2JpdG9wcy1iaXRzLWluLWJ5dGUuanMJKHJldmlzaW9uIDkwMjIxKQorKysgUGVyZm9y bWFuY2VUZXN0cy9TdW5TcGlkZXIvdGVzdHMvc3Vuc3BpZGVyLTEuMC9iaXRvcHMtYml0cy1pbi1i eXRlLmpzCSh3b3JraW5nIGNvcHkpCkBAIC0xLDYgKzEsOCBAQAogLy8gQ29weXJpZ2h0IChjKSAy MDA0IGJ5IEFydGh1ciBMYW5nZXJlaXMgKGFydGh1cl9leHQgYXQgZG9tYWluIHhmaW5pdGVnYW1l cywgdGxkIGNvbSkKIAogCit2YXIgcmVzdWx0ID0gMDsKKwogLy8gMSBvcCA9IDIgYXNzaWducywg MTYgY29tcGFyZS9icmFuY2hlcywgOCBBTkRzLCAoMC04KSBBRERzLCA4IFNITHMKIC8vIE8obikK IGZ1bmN0aW9uIGJpdHNpbmJ5dGUoYikgewpAQCAtMTQsOCArMTYsMTAgQEAgcmV0dXJuIGM7CiAK IGZ1bmN0aW9uIFRpbWVGdW5jKGZ1bmMpIHsKIHZhciB4LCB5LCB0OwordmFyIHN1bSA9IDA7CiBm b3IodmFyIHg9MDsgeDwzNTA7IHgrKykKLWZvcih2YXIgeT0wOyB5PDI1NjsgeSsrKSBmdW5jKHkp OworZm9yKHZhciB5PTA7IHk8MjU2OyB5KyspIHN1bSArPSBmdW5jKHkpOworcmV0dXJuIHN1bTsK IH0KIAotVGltZUZ1bmMoYml0c2luYnl0ZSk7CityZXN1bHQgPSBUaW1lRnVuYyhiaXRzaW5ieXRl KTsKSW5kZXg6IFBlcmZvcm1hbmNlVGVzdHMvU3VuU3BpZGVyL3Rlc3RzL3N1bnNwaWRlci0xLjAv Yml0b3BzLWJpdHdpc2UtYW5kLmpzCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIFBlcmZvcm1hbmNlVGVzdHMvU3Vu U3BpZGVyL3Rlc3RzL3N1bnNwaWRlci0xLjAvYml0b3BzLWJpdHdpc2UtYW5kLmpzCShyZXZpc2lv biA5MDIyMSkKKysrIFBlcmZvcm1hbmNlVGVzdHMvU3VuU3BpZGVyL3Rlc3RzL3N1bnNwaWRlci0x LjAvYml0b3BzLWJpdHdpc2UtYW5kLmpzCSh3b3JraW5nIGNvcHkpCkBAIC0yNiwzICsyNiw1IEBA CiBiaXR3aXNlQW5kVmFsdWUgPSA0Mjk0OTY3Mjk2OwogZm9yICh2YXIgaSA9IDA7IGkgPCA2MDAw MDA7IGkrKykKICAgICBiaXR3aXNlQW5kVmFsdWUgPSBiaXR3aXNlQW5kVmFsdWUgJiBpOworCit2 YXIgcmVzdWx0ID0gYnV0d2lzZUFuZFZhbHVlOwpJbmRleDogUGVyZm9ybWFuY2VUZXN0cy9TdW5T cGlkZXIvdGVzdHMvc3Vuc3BpZGVyLTEuMC9iaXRvcHMtbnNpZXZlLWJpdHMuanMKPT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PQotLS0gUGVyZm9ybWFuY2VUZXN0cy9TdW5TcGlkZXIvdGVzdHMvc3Vuc3BpZGVyLTEuMC9iaXRv cHMtbnNpZXZlLWJpdHMuanMJKHJldmlzaW9uIDkwMjIxKQorKysgUGVyZm9ybWFuY2VUZXN0cy9T dW5TcGlkZXIvdGVzdHMvc3Vuc3BpZGVyLTEuMC9iaXRvcHMtbnNpZXZlLWJpdHMuanMJKHdvcmtp bmcgY29weSkKQEAgLTI3LDYgKzI3LDcgQEAgZnVuY3Rpb24gc2lldmUoKSB7CiAgICAgICAgIHZh ciBpc1ByaW1lID0gbmV3IEFycmF5KCgxMDAwMDw8aSkrMzE+PjUpOwogICAgICAgICBwcmltZXMo aXNQcmltZSwgaSk7CiAgICAgfQorICAgIHJldHVybiBpc1ByaW1lOwogfQogCi1zaWV2ZSgpOwor dmFyIHJlc3VsdCA9IHNpZXZlKCk7Cg==