Testing with the W3 page <http://www.w3.org/2002/09/tests/keys-cancel2.html> I noticed that no keyup event was generated when turning caps lock on. But one was generated when turning it off.
I believe this is an issue in an underlying component of OS X, and I'm not aware of any workaround.
Also see https://bugzilla.mozilla.org/show_bug.cgi?id=712535.
In particular, the reported behavior of WebKit seems to indicate we have more bugs in this area:
I've tested this in Chrome and Safari, both of which fire a keyup event after toggling the caps-lock key.
On the first press of caps-lock the keydown event fires.
On the second press of caps-lock the keydown event fires again.
On the first press of caps-lock the keydown event should fire.
On the second press of caps-lock the keyup event should fire, Not keydown again.