Bug 138118 - Assert that Array elements not copied when changing shape to ArrayStorage type are indeed holes
Summary: Assert that Array elements not copied when changing shape to ArrayStorage typ...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: JavaScriptCore (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Mark Lam
URL:
Keywords: InRadar
Depends on: 138138
Blocks:
  Show dependency treegraph
 
Reported: 2014-10-27 19:20 PDT by Mark Lam
Modified: 2014-12-22 15:09 PST (History)
10 users (show)

See Also:


Attachments
the patch. (4.36 KB, patch)
2014-10-27 22:31 PDT, Mark Lam
mhahnenb: review+
Details | Formatted Diff | Diff
updated patch (3.64 KB, patch)
2014-12-22 13:42 PST, Mark Lam
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Mark Lam 2014-10-27 19:20:55 PDT
When we convert a dense holey array into the ArrayStorage shape, the new holes in the new array gets garbage.

The following test is run with Heap::tryAllocateStorage() modified to scribble all over newly allocated buffers with the value 0xbaddda4a.  The test basically sets up a hole DoubleShape array, and then runs Array.unshift() on it to force it to change to the ArrayStorage shape.  Thereafter, the test dumps the values of the array again to look for the holes.

The test code:
=============
function doubleArray() {
    var arr = [];
    arr[0] = 0.1;
    arr[1] = 1.1;
    arr[2] = 2.1;
    arr[3] = 3.1;
    // arr[4] = hole;
    // arr[5] = hole;
    arr[6] = 6.1;
    arr[7] = 7.1;
    arr[8] = 8.1;
    return arr;
}

function test(name, arr, newElement) {
    print(name + " BEFORE unshift:");
    for (var i = 0; i < arr.length; i++)
        print("   arr[" + i + "] = " + arr[i]);

    arr.unshift(newElement);
    print(name + " AFTER unshift:");
    for (var i = 0; i < arr.length; i++)
        print("   arr[" + i + "] = " + arr[i]);
}
test("double array", doubleArray(), 100.5);

The output:
==========
double array BEFORE unshift:
   arr[0] = 0.1
   arr[1] = 1.1
   arr[2] = 2.1
   arr[3] = 3.1
   arr[4] = undefined
   arr[5] = undefined
   arr[6] = 6.1
   arr[7] = 7.1
   arr[8] = 8.1
   arr[9] = undefined
double array AFTER unshift:
   arr[0] = 100.5
   arr[1] = 0.1
   arr[2] = 1.1
   arr[3] = 2.1
   arr[4] = 3.1
   arr[5] = -3.7291244322514128e-25
   arr[6] = -3.7291244322514128e-25
   arr[7] = 6.1
   arr[8] = 7.1
   arr[9] = 8.1

Note that the resultant arr[5] and arr[6] which should be undefined (because of the holes) now contain junk.
Comment 1 Radar WebKit Bug Importer 2014-10-27 19:22:24 PDT
<rdar://problem/18791403>
Comment 2 Mark Lam 2014-10-27 22:31:50 PDT
Created attachment 240528 [details]
the patch.
Comment 3 Mark Hahnenberg 2014-10-28 07:56:36 PDT
Comment on attachment 240528 [details]
the patch.

R=me
Comment 4 Mark Hahnenberg 2014-10-28 07:59:01 PDT
Is there a test you could add that would reliably trigger the bug without having to make the tryAllocateStorage modification?
Comment 5 Mark Lam 2014-10-28 08:27:01 PDT
(In reply to comment #4)
> Is there a test you could add that would reliably trigger the bug without
> having to make the tryAllocateStorage modification?

Maybe, but it will require a lot of allocations and GCs to create allocatable memory filled with junk that we can allocate from.  Such a test will probably not be short running, and is flaky at best.

The test will also have to rely on operations like Array.unshift() to trigger the conversion to ArrayStorage shape.  Hence, the test is not general and will only be testing this specific case that has been fixed, and therefore have limited utility.  We’re also not likely to regress this specific piece of code.  Hence, I’m going to forego the test and land this without it for now.
Comment 6 Mark Lam 2014-10-28 08:30:29 PDT
Thanks for the review.  Landed in r175249: <http://trac.webkit.org/r175249>.
Comment 7 Mark Hahnenberg 2014-10-28 08:33:40 PDT
(In reply to comment #5)
> (In reply to comment #4)
> > Is there a test you could add that would reliably trigger the bug without
> > having to make the tryAllocateStorage modification?
> 
> Maybe, but it will require a lot of allocations and GCs to create
> allocatable memory filled with junk that we can allocate from.  Such a test
> will probably not be short running, and is flaky at best.

Is it even possible to create a storage allocation with uninitialized memory? I thought it was guaranteed that any CopiedSpace allocation was already pre-zeroed.

There's an invariant with array backing stores that requires they be zeroed when they're allocated. If it's possible to get random garbage in your array backing store, that sounds like a bug in the storage allocator.
Comment 8 Mark Lam 2014-10-28 08:36:16 PDT
(In reply to comment #7)
> There's an invariant with array backing stores that requires they be zeroed
> when they're allocated. If it's possible to get random garbage in your array
> backing store, that sounds like a bug in the storage allocator.

Interesting.  I’ll double check this in the code.
Comment 9 Geoffrey Garen 2014-10-28 10:41:28 PDT
Seems like you should be able to test this with a double array, since a zero-initialized double array will see zeros instead of holes.
Comment 10 Mark Lam 2014-10-28 10:44:14 PDT
(In reply to comment #9)
> Seems like you should be able to test this with a double array, since a
> zero-initialized double array will see zeros instead of holes.

That won't work.  The scenario here requires a conversion of the double array to an ArrayStorage array.  In the ArrayStorage array, 0s are holes.
Comment 11 WebKit Commit Bot 2014-10-28 10:48:24 PDT
Re-opened since this is blocked by bug 138138
Comment 12 Geoffrey Garen 2014-10-28 10:48:51 PDT
If zero is hole in ArrayStorage, and the GC zero-initializes ArrayStorage, then this patch is wrong and you should roll it out.

An alternative patch, which might be fine, would ASSERT that all indices that were holes in the input are holes in the output.
Comment 13 Mark Lam 2014-10-28 17:49:09 PDT
For the record, the patch was causing some JS test failures because the assertion I added in JSObject::convertUndecidedToArrayStorage() was firing.  Looks like the publicLength is not 0 even though JSObject::convertUndecidedToArrayStorage() does not copy any array elements.  I may have misunderstood how Undecided arrays work.  Will investigate further.
Comment 14 Filip Pizlo 2014-10-28 19:02:49 PDT
(In reply to comment #13)
> For the record, the patch was causing some JS test failures because the
> assertion I added in JSObject::convertUndecidedToArrayStorage() was firing. 
> Looks like the publicLength is not 0 even though
> JSObject::convertUndecidedToArrayStorage() does not copy any array elements.
> I may have misunderstood how Undecided arrays work.  Will investigate
> further.

Undecided can come into play when you do something like:

var a = []; // It's IsArray + Undecided because you haven't stored anything into it.
a.length = 42; // It's still IsArray + Undecided because you still haven't stored anything into it, and the publicLength is 42.
Comment 15 Mark Lam 2014-12-22 13:36:46 PST
I was wrong about heap memory not being zeroed out on allocation.  The backing store for Arrays are allocated from the CopiedSpace (aka storage space), and those allocations are zero filled.  Changing this patch to assert that the expected holes are holes.
Comment 16 Mark Lam 2014-12-22 13:42:40 PST
Created attachment 243639 [details]
updated patch
Comment 17 Michael Saboff 2014-12-22 14:31:34 PST
Comment on attachment 243639 [details]
updated patch

View in context: https://bugs.webkit.org/attachment.cgi?id=243639&action=review

> Source/JavaScriptCore/runtime/JSObject.cpp:853
> +        if (value == value) {

Shouldn't this always be true?  Seems like a bug.
Comment 18 Michael Saboff 2014-12-22 14:33:57 PST
Comment on attachment 243639 [details]
updated patch

View in context: https://bugs.webkit.org/attachment.cgi?id=243639&action=review

r=me

>> Source/JavaScriptCore/runtime/JSObject.cpp:853
>> +        if (value == value) {
> 
> Shouldn't this always be true?  Seems like a bug.

So this is a not a NaN check.
Comment 19 WebKit Commit Bot 2014-12-22 15:09:27 PST
Comment on attachment 243639 [details]
updated patch

Clearing flags on attachment: 243639

Committed r177657: <http://trac.webkit.org/changeset/177657>
Comment 20 WebKit Commit Bot 2014-12-22 15:09:33 PST
All reviewed patches have been landed.  Closing bug.