Bug 28117 – Native JSON.stringify does not omit functions

• Status: RESOLVED
• Resolution: FIXED
• Component: JavaScriptCore

[Luke Smith]

JSON does not define functions.  They should be treated in the same manner as undefined, omitted in objects and replaced by null in arrays.

Nightly r46919 stringifies functions as objects.

JSON.stringify({fn:function () {}}) should return '{}' but returns '{"fn":{}}' in the nightly.

[Mark Rowe (bdash)]

<rdar://problem/7129375>

[Oliver Hunt]

This is a bug in firefox.  Functions are an object, and therefore should be emitted, per section 15.12.3 of es262-5

[Oliver Hunt]

https://bugzilla.mozilla.org/show_bug.cgi?id=509339

[Luke Smith]

Per the final draft of the ECMA 5 spec located at http://www.ecma-international.org/publications/files/drafts/tc39-2009-025.pdf

Section 15.12.3 states in the definition of the Str abstract function
4. If value is null then return "null". 
5. If value is true then return "true". 
6. If value is false then return "false". 
7. If Type(value) is object then, 
a. If the [[Class]] internal property of value is "Number" then, 
i. Let value be ToNumber(value). 
b. Else if the [[Class]] internal property of value is "String" then, 
i. Let value be ToString(value). 
8. If Type(value) is string, then return the result of calling the abstract operation Quote with argument 
value. 
9. If Type(value) is number 
a. If value is finite then return ToString(value). 
b. else, return "null". 
10. If Type(value) is object, and IsCallable(value) is false 
a. If the [[Class]] internal property of value is "Array" then 
i. Return the result of calling the abstract operation JA with argument value. 
b. Return the result of calling the abstract operation JO with argument value. 
11. Return undefined.


Note 10. and IsCallable(value) is false

Also, every implementation of JSON including the reference implementation by Douglas Crockford treats functions as it would undefined.

From Crockford's http://json.org/json2.js
"Values that do not have JSON representations, such as undefined or
functions, will not be serialized. Such values in objects will be
dropped; in arrays they will be replaced with null. You can use
a replacer function to replace those with JSON values.
JSON.stringify(undefined) returns undefined."

From http://json.org/js.html
"Values that do not have a representation in JSON (such as functions and undefined) are excluded."

[Oliver Hunt]

Ah indeed.  That makes me sad.

[Oliver Hunt]

Created attachment 34637 [details]
Patch v1

[Oliver Hunt]

Committed r47086

[Darin Adler]

Comment on attachment 34637 [details]
Patch v1

Why not use inherits(&InternalFunction::info) instead of calling getCallData and then special-casing JSArray?

[Oliver Hunt]

(In reply to comment #8)
> (From update of attachment 34637 [details])
> Why not use inherits(&InternalFunction::info) instead of calling getCallData
> and then special-casing JSArray?

The uses the IsCallable internal function -- inheriting from InternalFunction is not a sufficient check, but now i've tested firefox with a regex, /a/, they clearly are ignoring IsCallable
*yay*

[Luke Smith]

What was the test against FF?  RegExp should return false to IsCallable.  Safari and webkit have been incorrectly reporting typeof /a/ === 'function' for several versions.

[Oliver Hunt]

(In reply to comment #10)
> What was the test against FF?  RegExp should return false to IsCallable. 
> Safari and webkit have been incorrectly reporting typeof /a/ === 'function' for
> several versions.

RegExps are callable -- eg. /b/("abc") -- the regex is callable, that means that the internal property IsCallable() must return true, which means that regexps should not be serialised.

[Luke Smith]

I was unaware of this feature until reading your comment.  This behavior is not in the spec.

section 9.11 specifies IsCallable
If the argument object has an [[Call]] internal method, then return true, otherwise return false.

The only references in the spec to setting the [[Call]] internal method are
13.2 Creating Function objects
13.2.3 The [[ThrowTypeError]] Function Object
15.3.4.5 Function.prototype.bind (thisArg [, arg1 [, arg2, …]])

Section 15.10.6, defining RegExp, states
The value of the [[Prototype]] internal property of the RegExp prototype object is the standard built-in 
Object prototype object (15.2.4)

I found that /a/("abc") is implemented (inconsistently) in Firefox 3.0, Firefox 3.5, Safari 3.2.1,  Safari 4, Opera 9.6.3, Opera 10 beta2, Chrome 3 beta, and the WebKit nightlies.  It is not implemented in IE6, IE7, IE8, or Chrome 2.

Mozilla identifies this feature as a JavaScript extension
https://developer.mozilla.org/en/Core_JavaScript_1.5_Reference/Global_Objects/RegExp#RegExp_instances

I infer that such is the case for the others, WebKit included.  The difference here being that the other browser vendors prevent the spec deviation from rippling through their implementation to (at least) typeof and recently to JSON.stringify.

Currently native JSON support is present in Firefox 3.5, IE8, Chrome 3 beta, and recent WebKit nightlies.  All three of the other currently implementing browsers agree both that typeof /a/ == 'object' and JSON.stringify(/a/) == '{}'.  There is value in interoperability.  If not typeof, please at least stringify RegExp instances as objects for the sake of convention.

[Oliver Hunt]

> Currently native JSON support is present in Firefox 3.5, IE8, Chrome 3 beta,
> and recent WebKit nightlies.  All three of the other currently implementing
> browsers agree both that typeof /a/ == 'object' and JSON.stringify(/a/) ==
> '{}'.  There is value in interoperability.  If not typeof, please at least
> stringify RegExp instances as objects for the sake of convention.

RegExps are callable -- breaking this would break compatibility with json2.js, and the bug is in mozilla for not correctly treating a callable object as callable.

We cannot reasonably serialise regexps as standard objects -- the bug is in firefox, not webkit.

[Luke Smith]

> breaking this would break compatibility with json2.js

Can you explain this?

[Raphael Speyer]

(In reply to comment #13)
> RegExps are callable -- breaking this would break compatibility with json2.js,
> and the bug is in mozilla for not correctly treating a callable object as
> callable.

I think there is an important distinction between a vendor extension to EcmaScript to allow an object to be treated as a function, and the meaning of IsCallable in the spec.

As Luke mentioned above, according to the spec, a IsCallable is false for RegExp's, and thus they should be serialised by JSON.stringify, in 15.12.3 step 10.

[Raphael Speyer]

(In reply to comment #15)

> As Luke mentioned above, according to the spec, a IsCallable is false for
> RegExp's, and thus they should be serialised by JSON.stringify, in 15.12.3 step
> 10.

That is, step 10 of the Str operation.

[Oliver Hunt]

(In reply to comment #16)
> (In reply to comment #15)
> 
> > As Luke mentioned above, according to the spec, a IsCallable is false for
> > RegExp's, and thus they should be serialised by JSON.stringify, in 15.12.3 step
> > 10.
> 
> That is, step 10 of the Str operation.

json2.js uses typeof holder === "function" to determine whether an object IsCallable, and regexp instances have the type "function".  This is correct behaviour, per ES5 (from ES262, v5, 11.4.3)
Object (native and doesn‘t implement  [[Call]])  -> "object" 
Object (native or host and implements  [[Call]]) -> "function" 

IsCallable is defined in section 9.11:
If the argument object has an [[Call]] internal method, then return true, otherwise 
return false. 

In other words, a RegExp instance has [[Call]] so typeof should be "function", and [[IsCallable]] is true.

You seem to believe that there is value in serialising a regexp object, when there is not -- a serialised regexp object is either undefined (in conforming implementations in which regexps are callable) or as the empty object {} (in conforming implementations in whichs regexps are not callable).

Firefox is not conforming in this regard.  Arguing will not accopmlish anything as it is pointless to add a slow check which will break behaviour relative to json2, just to handle the specific case of RegExps, when there are numerous other disagreements between different runtimes as to which objects are callable (NodeLists anyone?), especially given the serialisation of RegExps is unhelpful in either case.

If you feel really strongly about this, i suggest you go to http://bugs.mozilla.org and file a bug on the spidermonkey RegExp object incorrectly reporting typeof as "object" and claiming not to be callable.

[Raphael Speyer]

(In reply to comment #17)
> 
> json2.js uses typeof holder === "function" to determine whether an object
> IsCallable, 

That seems reasonable to me.

> and regexp instances have the type "function".  

I think this is the crux of the issue. I know that webkit gives typeof /a/ ===
'function', but I don't understand why. The spec does not say anywhere that a RegExp defines the internal property [[Call]].

> This is correct
> behaviour, per ES5 (from ES262, v5, 11.4.3)
> Object (native and doesn‘t implement  [[Call]])  -> "object" 
> Object (native or host and implements  [[Call]]) -> "function" 
> 

Exactly. So, wouldn't RegExp's fall into the first category? Yes, they can be treated as functions, but that's a particular vendor extension, it's not how RegExp instances are specified to behave.

> IsCallable is defined in section 9.11:
> If the argument object has an [[Call]] internal method, then return true,
> otherwise 
> return false. 

Right, and again, as per the spec, a RegExp does not have a [[Call]] internal method.

> 
> In other words, a RegExp instance has [[Call]] so typeof should be "function",
> and [[IsCallable]] is true.
> 

I think we forked back at /a/('a') implying that at a specification level, [[Call]] is defined for RegExp's.

> You seem to believe that there is value in serialising a regexp object, when
> there is not -- a serialised regexp object is either undefined (in conforming
> implementations in which regexps are callable) or as the empty object {} (in
> conforming implementations in whichs regexps are not callable).
> 

Granted, it's not going to be typical, and I can't actually think of a very good use case for it myself. However enumerable properties can be defined on RegExp's or their prototype, as can the toJSON method. I think this is more about conformance with the spec.

> Firefox is not conforming in this regard.  Arguing will not accopmlish anything
> as it is pointless to add a slow check which will break behaviour relative to
> json2, just to handle the specific case of RegExps, when there are numerous
> other disagreements between different runtimes as to which objects are callable
> (NodeLists anyone?), especially given the serialisation of RegExps is unhelpful
> in either case.
> 
> If you feel really strongly about this, i suggest you go to
> http://bugs.mozilla.org and file a bug on the spidermonkey RegExp object
> incorrectly reporting typeof as "object" and claiming not to be callable.

Well, as I said above, as far as I can tell, the spec says that RegExp is not callable, in the sense that the internal [[Call]] property is not defined, IsCallable will return false and thus at the language level, typeof should return 'object', and stringify should not ignore it.

[kangax]

(In reply to comment #18)
> (In reply to comment #17)
[...]
> > Firefox is not conforming in this regard.  Arguing will not accopmlish anything
> > as it is pointless to add a slow check which will break behaviour relative to
> > json2, just to handle the specific case of RegExps, when there are numerous
> > other disagreements between different runtimes as to which objects are callable
> > (NodeLists anyone?), especially given the serialisation of RegExps is unhelpful
> > in either case.
> > 
> > If you feel really strongly about this, i suggest you go to
> > http://bugs.mozilla.org and file a bug on the spidermonkey RegExp object
> > incorrectly reporting typeof as "object" and claiming not to be callable.
> 
> Well, as I said above, as far as I can tell, the spec says that RegExp is not
> callable, in the sense that the internal [[Call]] property is not defined,
> IsCallable will return false and thus at the language level, typeof should
> return 'object', and stringify should not ignore it.

Yes, at the "language level" `RegExp` objects don't have [[Call]], but don't forget that ES3 allows implementations to extend language and still be considered conforming. Quoting section 2:

| A conforming implementation of ECMAScript is permitted to provide 
| additional types, values, objects, properties, and functions beyond those
| described in this specification. In particular, a conforming implementation
| of ECMAScript is permitted to provide properties not described in this
| specification, and values for those properties, for objects that are
| described in this specification.

So here we are with a conforming implementation that adds [[Call]] to RegExp objects. Note that nowhere does ES3 explicitly disallow for `RegExp` objects to have [[Call]] as it does for, say, `Math` or "Global object".

[Raphael Speyer]

(In reply to comment #19)
> (In reply to comment #18)
> > (In reply to comment #17)
> [...]
> > 
> > Well, as I said above, as far as I can tell, the spec says that RegExp is not
> > callable, in the sense that the internal [[Call]] property is not defined,
> > IsCallable will return false and thus at the language level, typeof should
> > return 'object', and stringify should not ignore it.
> 
> Yes, at the "language level" `RegExp` objects don't have [[Call]], but don't
> forget that ES3 allows implementations to extend language and still be
> considered conforming. Quoting section 2:
> 
> | A conforming implementation of ECMAScript is permitted to provide 
> | additional types, values, objects, properties, and functions beyond those
> | described in this specification. In particular, a conforming implementation
> | of ECMAScript is permitted to provide properties not described in this
> | specification, and values for those properties, for objects that are
> | described in this specification.
> 
> So here we are with a conforming implementation that adds [[Call]] to RegExp
> objects. Note that nowhere does ES3 explicitly disallow for `RegExp` objects to
> have [[Call]] as it does for, say, `Math` or "Global object".

Good points. I guess to some extent it's a matter of interpretation here.

To me /a/('abc') just looks like syntactic sugar for /a/.exec('abc'). /a/ is certainly not a member of the "function" objects described in section 13 or 15.3 of the spec. The view that it's simply (unspecified) syntactic sugar, and doesn't change anything about the runtime behaviour seems to be Mozilla's approach.

On the other hand, WebKit seem to have taken it as both additional syntax and also a semantic change such that while a RegExp instance is not a function as in "/a/ instanceof Function", it does have an internal [[Call]] property, which is called by this new syntax. Perhaps that view is just as valid, but it does alter the behaviour of syntactically identical programs. It's hard to tell who's right because this behaviour was never specified to begin with.

I guess there's not much more for me I can say here. This sort of thing is best clarified, and specified (or explicitly not specified) by the TC39 commitee.

[Brendan Eich]

I cited these Mozilla bug reports to Ollie over IRC:

https://bugzilla.mozilla.org/show_bug.cgi?id=61911
https://bugzilla.mozilla.org/show_bug.cgi?id=289933

We fixed the higher-numbered one first, making typeof /a/ == "function". But then we retreated in 61911 due to complaints and (I seem to recall; it's hard to find evidence at the moment) real web compatibility problems.

The complaints weren't all from spec-purists who did not like the extension in SpiderMonkey that allows /a/(s) as shorthand for /a/.exec(s). I remember more than a few places where real code was flummoxed by typeof /a/ == "function". This confusing result broke code, whether or not such code expected (this would have been Mozilla-specific code, originally) to be able to call a regexp as shorthand for exec'ing it.

Anyway, we threw in the towel with the resolution of 61911. The only further retreat for us is to remove callability, but that may be hard. We may be stuck. We can't easily go back to typeof /a/ == "function", in any event.

/be

[Brendan Eich]

I've re-raised this issue (it has been beaten around before) on es-discuss:

https://mail.mozilla.org/pipermail/es-discuss/2009-August/009718.html

/be

[Oliver Hunt]

> > and regexp instances have the type "function".  
> 
> I think this is the crux of the issue. I know that webkit gives typeof /a/ ===
> 'function', but I don't understand why. The spec does not say anywhere that a
> RegExp defines the internal property [[Call]].
> 
> > This is correct
> > behaviour, per ES5 (from ES262, v5, 11.4.3)
> > Object (native and doesn‘t implement  [[Call]])  -> "object" 
> > Object (native or host and implements  [[Call]]) -> "function" 
> > 
> 
> Exactly. So, wouldn't RegExp's fall into the first category? Yes, they can be
> treated as functions, but that's a particular vendor extension, it's not how
> RegExp instances are specified to behave.
> 
> > IsCallable is defined in section 9.11:
> > If the argument object has an [[Call]] internal method, then return true,
> > otherwise 
> > return false. 
> 
> Right, and again, as per the spec, a RegExp does not have a [[Call]] internal
> method.
And you can take that up with mozilla who decided to give RegExp are

If an object can be called it is by definition callable.  To be called an
object must provide [[Call]] -- if you look at section 11.2.3 you will see that
a function call will throw an exception if IsCallable(func) returns false. 
Therefore by definition IsCallable must be true.

> 
> > 
> > In other words, a RegExp instance has [[Call]] so typeof should be "function",
> > and [[IsCallable]] is true.
> > 
> 
> I think we forked back at /a/('a') implying that at a specification level,
> [[Call]] is defined for RegExp's.
Mozilla extended the spec by saying RegExps were callable, thus forcing us to
implement this extension or be broken.  [[Call]] is defined on on RegExp as an
extension, but that does not make it not present.

> Well, as I said above, as far as I can tell, the spec says that RegExp is not
> callable, in the sense that the internal [[Call]] property is not defined,
> IsCallable will return false and thus at the language level, typeof should
> return 'object', and stringify should not ignore it.
[[Call]] is defined, IsCallable is true, typeof is function, and serialisation
should skip it.  If you want to argue this more, please provide a logical
explanation beyond "firefox violates spec here, so should you"

Once again, the fact that /a/("a") does not throw an exception demonstrates
that IsCallable(/a/) must be true in firefox

[Brendan Eich]

Ollie, could you point me to bugs or URLs showing where you had to follow the SpiderMonkey extension of callable regexps "or be broken"? Thanks,

/be