Bug 10598 - JSMouseEvent needs to expose tablet pressure
Summary: JSMouseEvent needs to expose tablet pressure
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: 420+
Hardware: Mac OS X 10.4
: P4 Enhancement
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-08-28 00:35 PDT by Eric Seidel (no email)
Modified: 2009-12-08 20:05 PST (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Eric Seidel (no email) 2006-08-28 00:35:56 PDT
If we're ever going to see Illustrator for the web (which was part of my goal in working on SVG in the first place), we will need to expose tablet pressure to web applications.  This means exposing it as part of JSMouseEvent, perhaps initially as a WebKit extension.

Without tablet pressure, there is no way you could do anything remotely advanced in the way of brush tools or strokes.
Comment 1 Alp Toker 2007-12-28 18:53:49 PST
I guess we should study the XInput extension (http://www.gtk.org/~otaylor/xinput/howto/) and also Multi-Pointer X (http://wearables.unisa.edu.au/mpx/) to see what kind of data we want to expose. Other platforms may provide access to different information too.
Comment 2 Oliver Hunt 2007-12-29 19:56:44 PST
My 2c: For this to be really feasible the webapi/DOM groups need to work to define what these parameters should be.
Comment 3 Ilmari Heikkinen 2007-12-31 14:34:46 PST
Propose a vendor-specific (+ standardize based on the experiences from that) new event type for tablet events:

TabletEvent : MouseEvent {
  float rotation // 0 .. 360
  float tiltX // -1.0 .. 1.0
  float tiltY // -1.0 .. 1.0
  float pressure // 0.0 .. 1.0
  float tangentialPressure // -1.0 .. 1.0

  string type // tool type 'pen', 'eraser', 'cursor', 'unknown'

  uint64 uniqueID // unique ID for the tool 
                  // (e.g. two pens, one to draw with black, other with blue)
  
  uint deviceIndex // sequence number of the device on a multi-touch tablet
                   // (e.g. first finger uses 1, second finger uses 2)

  float accurateClientX
  float accurateClientY
  float accurateScreenX
  float accurateScreenY
}

Based on http://developer.apple.com/documentation/Cocoa/Reference/ApplicationKit/Classes/NSEvent_Class/Reference/Reference.html
Comment 4 Eric Seidel (no email) 2008-01-02 06:42:57 PST
A few comments:

1.  I'd use "preciseClientX", instead of "accurate", "float" or "double" would even be better, except I think those might get more and more precise over time.  We'd probably try to expose as accurate values as the device supported, which would be doubles on 64bit.

2.  How would one register for these?  Would normal places you would get mouse events give you these instead?  Probably not.  Could we implement these in webkit and test them using an addEventListener("-webkit-tabletmoved", foo, false); scheme?

3.  If these aren't part of normal mouse events, what do the names of the corresponding tablet registrations look like?  tabletMoved, tabletOver, tabletOut, etc?

4.  If these aren't part of normal mouse events, I think normal mouse events would be dispatched on a throttled basis, when tablet events were coming in.  Tablets send way more events than normal pointing devices do.
Comment 5 Ilmari Heikkinen 2008-01-03 20:06:24 PST
(In reply to comment #4)
> 1.  I'd use "preciseClientX", instead of "accurate", "float" or "double" would
> even be better, except I think those might get more and more precise over time.
>  We'd probably try to expose as accurate values as the device supported, which
> would be doubles on 64bit.

preciseClientX sounds good to me. The accuracy of the values is a bit non-issue, as a 45cm wide tablet can be addressed down to individual atoms with 32-bit numbers...

Also on naming, noticed that "type" is already used for the event type, so that would need to be "toolType" instead.

 
> 2.  How would one register for these?  Would normal places you would get mouse
> events give you these instead?  Probably not.  Could we implement these in
> webkit and test them using an addEventListener("-webkit-tabletmoved", foo,
> false); scheme?
> 
> 3.  If these aren't part of normal mouse events, what do the names of the
> corresponding tablet registrations look like?  tabletMoved, tabletOver,
> tabletOut, etc?

This is a difficult question, let's see..

Mouse events can be thought of as a subset of tablet events (toolType: 'cursor', deviceIndex: 0, all the rest 0, precise coords = normal coords). But piggy-backing tablet data on mouse events would bloat the mouse event API. And likely break things.

Having a single tablet event and doing the rest with mouse* would make it difficult to handle multi-touch (would have to keep track of device coords and compare mouse event coords to those to figure out which device triggered it.) It would be the simplest to implement. 

Implementing tabletover, tabletout, tabletdown, tabletup and tabletmove, and firing them along with the mouse events wouldn't break things, and would make application programming easier than having a single tablet event. Tablet events should be fired before mouse events to make it possible to cancel the following mouse event. Would go with this.

Need to have tabletenter and tabletleave, too, for "a new device has left/entered the tablet, deal?" Otherwise, difficult to figure out how many devices there are on the tablet.


> 4.  If these aren't part of normal mouse events, I think normal mouse events
> would be dispatched on a throttled basis, when tablet events were coming in. 
> Tablets send way more events than normal pointing devices do.

I don't know. A tablet does trigger regular mouse events already, do they need further throttling? I wouldn't throw away input events, if at all possible.
Comment 6 Ilmari Heikkinen 2009-12-03 18:22:17 PST
Mozilla bug with GTK2 implementation and details
https://bugzilla.mozilla.org/show_bug.cgi?id=531466
Comment 7 Eric Seidel (no email) 2009-12-08 10:36:56 PST
We should also look at whatever the iPhone has done for multi-touch events.
Comment 8 Ilmari Heikkinen 2009-12-08 18:13:08 PST
(In reply to comment #7)
> We should also look at whatever the iPhone has done for multi-touch events.

The iPhone multi-touch events could be boiled down to a unique id for every tracked touch, attached to mouse events. Rest seems to be implementable as JS application logic. Mousedown would start a touch track, mousemove would track the movement and mouseup would end it.

The way they currently work is that there's a Touch object for each tracked touch, and TouchEvents that carry a list of the Touch objects modified by the event. The touch events follow the mouse event naming (touchdown, touchmove, touchend), with an additional touchcancel event for canceled touches. The TouchEvents also carry some gesture level data, namely rotation and pinch-scale.

Additionally, there's a GestureEvent that has rotation and pinch-scale info as well. The use of it seems to be in easy combining of separate TouchEvents.

(Wrote the above based on a quick reading of Safari DOM Extensions Reference, corrections from more knowledgeable people welcome.)