Bug 62998 - Don't always select images during an image drag
Summary: Don't always select images during an image drag
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: UI Events (show other bugs)
Version: 528+ (Nightly build)
Hardware: PC Linux
: P2 Normal
Assignee: Daniel Cheng
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-06-20 10:13 PDT by Aparna Nandyal
Modified: 2011-10-20 15:24 PDT (History)
14 users (show)

See Also:


Attachments
Patch v01 for review (1.76 KB, patch)
2011-06-20 10:24 PDT, Aparna Nandyal
no flags Details | Formatted Diff | Diff
Quirky test case (316 bytes, text/html)
2011-10-17 02:12 PDT, Daniel Cheng
no flags Details
Patch (1.94 KB, patch)
2011-10-17 02:17 PDT, Daniel Cheng
no flags Details | Formatted Diff | Diff
Patch (6.51 KB, patch)
2011-10-17 03:07 PDT, Daniel Cheng
no flags Details | Formatted Diff | Diff
Patch (6.52 KB, patch)
2011-10-17 03:09 PDT, Daniel Cheng
no flags Details | Formatted Diff | Diff
Patch (7.37 KB, patch)
2011-10-17 16:15 PDT, Daniel Cheng
no flags Details | Formatted Diff | Diff
Patch (6.54 KB, patch)
2011-10-17 22:03 PDT, Daniel Cheng
no flags Details | Formatted Diff | Diff
Patch (5.67 KB, patch)
2011-10-17 22:05 PDT, Daniel Cheng
no flags Details | Formatted Diff | Diff
Patch (11.30 KB, patch)
2011-10-18 13:28 PDT, Daniel Cheng
no flags Details | Formatted Diff | Diff
Patch (11.34 KB, patch)
2011-10-18 16:44 PDT, Daniel Cheng
no flags Details | Formatted Diff | Diff
Patch (12.96 KB, patch)
2011-10-19 13:18 PDT, Daniel Cheng
tony: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Aparna Nandyal 2011-06-20 10:13:37 PDT
Precondition:
You can either use the test case in https://bugs.webkit.org/show_bug.cgi?id=61504 or go to any website that has an image (example - http://en.wikipedia.org/wiki/Sloth). 

Steps to reproduce the problem:
1. Open the browser, QtTestBrowser
2. Go to the above mentioned link
3. Select just the image (nothing else)
4. Open any folder in the desktop.
5. Drag it into that folder.

Expected result:
The dragged image should be in the folder

Actual result:
Unable to drag the image

Additional info:
If there is only an image on the web page as is the case with test case in https://bugs.webkit.org/show_bug.cgi?id=61504, after the first drag, the image gets selected automatically. So unless the image selection is removed, we cannot drag the image into a folder.

Regression info:
This bug is a regression caused by changeset http://trac.webkit.org/changeset/86472 and the related change.
Comment 1 Aparna Nandyal 2011-06-20 10:24:52 PDT
Created attachment 97821 [details]
Patch v01 for review

Existing test cases cover regular drag and drop of image. Cannot specifically auto test this case as it requires dropping into a folder on the system.
Comment 2 Daniel Cheng 2011-06-20 10:36:17 PDT
Is this not working on other non-Qt/non-Mac platforms, e.g. Chromium Linux?
Comment 3 Yael 2011-06-21 11:03:36 PDT
(In reply to comment #2)
> Is this not working on other non-Qt/non-Mac platforms, e.g. Chromium Linux?

I tested on Chromium build on Windows and Linux and this works in both.
Chromium's clipboard actually converts the image to markup and Qt's clipboard should probably do the same.
Comment 4 Daniel Cheng 2011-06-21 13:21:34 PDT
(In reply to comment #3)
> (In reply to comment #2)
> > Is this not working on other non-Qt/non-Mac platforms, e.g. Chromium Linux?
> 
> I tested on Chromium build on Windows and Linux and this works in both.
> Chromium's clipboard actually converts the image to markup and Qt's clipboard should probably do the same.

My own testing seems to indicate it doesn't work on the lastest Linux Chromium build. The problem is when an image is the only thing in a selection, we use the selection logic instead of the image drag logic. I think I'm actually OK with this, though if we want to special case a selection with only an image, that's probably OK.

In opinion, the real bug here is that starting a drag on an image selects it. So if you want to drag it somewhere, change your mind, and then want to drag it again, you won't be able to drag an image file to the desktop. Note that dragging an image in IE/Firefox don't cause the image to get selected, but it causes the image to get selected in any WebKit browser.
Comment 5 Tony Chang 2011-06-21 13:53:23 PDT
Perhaps selecting when dragging an image should be Mac specific behavior?  That's the behavior I get when I try to drag an image in a native Mac app like Stickies or TextEdit and it would be nice to preserve that behavior on OSX.
Comment 6 Alexey Proskuryakov 2011-06-21 13:57:26 PDT
(In reply to comment #5)
> Perhaps selecting when dragging an image should be Mac specific behavior?  That's the behavior I get when I try to drag an image in a native Mac app like Stickies or TextEdit and it would be nice to preserve that behavior on OSX.

I know that I really dislike this behavior in Safari. Why should an image become grayed out on the page after I drag it somewhere?

But if it's the platform standard, it probably cannot be changed.
Comment 7 Darin Adler 2011-06-21 17:21:18 PDT
(In reply to comment #6)
> I know that I really dislike this behavior in Safari.

I also dislike it in Safari.

> But if it's the platform standard, it probably cannot be changed.

Maybe the platform standard is for an image in editable text, and it may be acceptable to have a different behavior for images outside of that context.
Comment 8 Aparna Nandyal 2011-06-21 22:03:14 PDT
> I tested on Chromium build on Windows and Linux and this works in both.
> Chromium's clipboard actually converts the image to markup and Qt's clipboard should probably do the same.

My testing indicates that it doesnt work on Chromium Linux build with latest WebKit code. 

As indicated, dragging an image does not cause image to be selected in other browsers. Even if we make WebKit behave the same way, we will still have a problem when user selects the image manually and tries to drag it.

Code before changeset 86472 follows a different logic for selection drag and different logic for image drag.
The fix checks if the selection has only an image and follows code path of image drag logic there after instead of selection logic.
Comment 9 Daniel Cheng 2011-06-22 12:44:14 PDT
> As indicated, dragging an image does not cause image to be selected in other browsers. Even if we make WebKit behave the same way, we will still have a problem when user selects the image manually and tries to drag it.

It's worth noting that IE and Firefox both have this problem.

> Code before changeset 86472 follows a different logic for selection drag and different logic for image drag.

That logic was arguably wrong.

> The fix checks if the selection has only an image and follows code path of image drag logic there after instead of selection logic.

I'm not certain this is the appropriate fix though. If the user accidentally selects any whitespace around the image, then they get a different behavior (drag to desktop won't work). And if we strip whitespace, we might be breaking behavior that is actually desired such as dragging for rich editing.
Comment 10 Eric Seidel (no email) 2011-09-12 15:40:41 PDT
Comment on attachment 97821 [details]
Patch v01 for review

View in context: https://bugs.webkit.org/attachment.cgi?id=97821&action=review

> Source/WebCore/page/DragController.cpp:590
> +                        state.m_dragType =  static_cast<DragSourceAction>(state.m_dragType | DragSourceActionImage);
> +                    else
> +                        state.m_dragType =   DragSourceActionImage;

Please use only 1 space after =.

How do we test this?
Comment 11 Eric Seidel (no email) 2011-09-12 15:41:05 PDT
Oliver was Mr. drag and drop in a past life.
Comment 12 Daniel Cheng 2011-10-17 02:12:28 PDT
Created attachment 111227 [details]
Quirky test case

I was locally testing a change to not create a selection around the image if the image is not in a contenteditable area. This resulted in a rather interesting quirk though. Let's say we have two images, foo (in a non-editable area) and bar (in an editable area).

Drag bar enough to cause it to get selected.
Drag foo into the same content editable area as bar.
bar disappears and foo is copied to wherever the drop occurred, so we end up with only two copies of foo and zero copies of bar.

I would have expected the final result to be two foos and one bar. Interestingly enough, Firefox also exhibits this quirk. I guess whenever you drag something into an editable area it's treated as replacing the selection?
Comment 13 Daniel Cheng 2011-10-17 02:17:45 PDT
Created attachment 111228 [details]
Patch
Comment 14 Daniel Cheng 2011-10-17 02:22:36 PDT
Comment on attachment 111228 [details]
Patch

Needs a test.
Comment 15 Daniel Cheng 2011-10-17 03:07:45 PDT
Created attachment 111233 [details]
Patch
Comment 16 Daniel Cheng 2011-10-17 03:09:27 PDT
Created attachment 111234 [details]
Patch
Comment 17 WebKit Review Bot 2011-10-17 04:43:24 PDT
Comment on attachment 111234 [details]
Patch

Attachment 111234 [details] did not pass chromium-ews (chromium-xvfb):
Output: http://queues.webkit.org/results/10075960

New failing tests:
editing/pasteboard/drag-image-in-about-blank-frame.html
editing/pasteboard/4947130.html
editing/pasteboard/drag-image-to-contenteditable-in-iframe.html
fast/events/drag-selects-image.html
editing/selection/drag-to-contenteditable-iframe.html
http/tests/security/drag-drop-same-unique-origin.html
Comment 18 Ryosuke Niwa 2011-10-17 11:17:43 PDT
Comment on attachment 111234 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=111234&action=review

> Source/WebCore/page/DragController.cpp:619
> +#if OS(DARWIN)

It seems like this should be tied to editing behavior.
Comment 19 Daniel Cheng 2011-10-17 16:15:00 PDT
Created attachment 111341 [details]
Patch
Comment 20 Daniel Cheng 2011-10-17 16:16:34 PDT
Comment on attachment 111341 [details]
Patch

What's the proper way to rebaseline results for the other layout test failures? I'm pretty sure that the remaining failures are all pixel diffs because the image doesn't get selected anymore which is OK.
Comment 21 WebKit Review Bot 2011-10-17 21:49:38 PDT
Comment on attachment 111341 [details]
Patch

Attachment 111341 [details] did not pass chromium-ews (chromium-xvfb):
Output: http://queues.webkit.org/results/10126112

New failing tests:
editing/selection/drag-to-contenteditable-iframe.html
editing/pasteboard/drag-image-in-about-blank-frame.html
http/tests/security/drag-drop-same-unique-origin.html
editing/pasteboard/drag-image-to-contenteditable-in-iframe.html
editing/pasteboard/4947130.html
Comment 22 Daniel Cheng 2011-10-17 22:03:03 PDT
Created attachment 111379 [details]
Patch
Comment 23 Daniel Cheng 2011-10-17 22:05:54 PDT
Created attachment 111380 [details]
Patch
Comment 24 WebKit Review Bot 2011-10-17 22:42:39 PDT
Comment on attachment 111380 [details]
Patch

Attachment 111380 [details] did not pass chromium-ews (chromium-xvfb):
Output: http://queues.webkit.org/results/10126129

New failing tests:
editing/selection/drag-to-contenteditable-iframe.html
editing/pasteboard/drag-image-to-contenteditable-in-iframe.html
Comment 25 Daniel Cheng 2011-10-18 13:28:18 PDT
Created attachment 111494 [details]
Patch
Comment 26 Daniel Cheng 2011-10-18 16:44:14 PDT
Created attachment 111527 [details]
Patch
Comment 27 Daniel Cheng 2011-10-18 18:19:37 PDT
Tony, mind taking a look? The updated patch matches Firefox behavior:
- dragging an image that is not editable does not cause the image to be selected
- dragging an image in is an editable area causes the image to be selected
Comment 28 Ryosuke Niwa 2011-10-18 18:41:16 PDT
Comment on attachment 111527 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=111527&action=review

> Source/WebCore/page/DragController.cpp:617
>  static void prepareClipboardForImageDrag(Frame* src, Clipboard* clipboard, Element* node, const KURL& linkURL, const KURL& imageURL, const String& label)

I'd rename src to frame if not sourceFrame.

> Source/WebCore/page/DragController.cpp:619
> +    if (node->isContentEditable()) {

You should check for being richly editable instead.

> LayoutTests/fast/events/drag-selects-image-expected.txt:7
> +This tests that images are properly left selected or unselected when an image drag is started. On non-Mac platforms, neither image should be selected when dragging. On Mac, only the image in the editable area should be selected when an image drag is started.

So this beahvior difference can't be changed by editingBehavior?
Comment 29 Daniel Cheng 2011-10-18 18:56:39 PDT
(In reply to comment #28)
> (From update of attachment 111527 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=111527&action=review
> 
> > Source/WebCore/page/DragController.cpp:617
> >  static void prepareClipboardForImageDrag(Frame* src, Clipboard* clipboard, Element* node, const KURL& linkURL, const KURL& imageURL, const String& label)
> 
> I'd rename src to frame if not sourceFrame.
> 

I'd argue that "Frame" doesn't really add any information though. It's a really short function, and it's clear that it's a Frame*.

> > Source/WebCore/page/DragController.cpp:619
> > +    if (node->isContentEditable()) {
> 
> You should check for being richly editable instead.
> 

What's the difference? I looked around but the obvious places don't really seem to explain what the difference between the two is.

> > LayoutTests/fast/events/drag-selects-image-expected.txt:7
> > +This tests that images are properly left selected or unselected when an image drag is started. On non-Mac platforms, neither image should be selected when dragging. On Mac, only the image in the editable area should be selected when an image drag is started.
> 
> So this beahvior difference can't be changed by editingBehavior?

Oops. I'll make sure to update that comment in the next patch.
Comment 30 Ryosuke Niwa 2011-10-18 19:04:30 PDT
Comment on attachment 111527 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=111527&action=review

>>> Source/WebCore/page/DragController.cpp:617
>>>  static void prepareClipboardForImageDrag(Frame* src, Clipboard* clipboard, Element* node, const KURL& linkURL, const KURL& imageURL, const String& label)
>> 
>> I'd rename src to frame if not sourceFrame.
> 
> I'd argue that "Frame" doesn't really add any information though. It's a really short function, and it's clear that it's a Frame*.

Then source will be better. src is an abbreviation and we should avoid using it unless we're talking about src attribute and a couple other special cases.

>>> Source/WebCore/page/DragController.cpp:619
>>> +    if (node->isContentEditable()) {
>> 
>> You should check for being richly editable instead.
> 
> What's the difference? I looked around but the obvious places don't really seem to explain what the difference between the two is.

isContentEditable is plaintext only.
Comment 31 Daniel Cheng 2011-10-18 22:00:06 PDT
(In reply to comment #30)
> (From update of attachment 111527 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=111527&action=review
> 
> >>> Source/WebCore/page/DragController.cpp:617
> >>>  static void prepareClipboardForImageDrag(Frame* src, Clipboard* clipboard, Element* node, const KURL& linkURL, const KURL& imageURL, const String& label)
> >> 
> >> I'd rename src to frame if not sourceFrame.
> > 
> > I'd argue that "Frame" doesn't really add any information though. It's a really short function, and it's clear that it's a Frame*.
> 
> Then source will be better. src is an abbreviation and we should avoid using it unless we're talking about src attribute and a couple other special cases.
> 
> >>> Source/WebCore/page/DragController.cpp:619
> >>> +    if (node->isContentEditable()) {
> >> 
> >> You should check for being richly editable instead.
> > 
> > What's the difference? I looked around but the obvious places don't really seem to explain what the difference between the two is.
> 
> isContentEditable is plaintext only.

Hypothetically speaking, how would I end up with a contenteditable image in a plaintext-only editing area?
Comment 32 Ryosuke Niwa 2011-10-18 22:08:23 PDT
(In reply to comment #31)
> Hypothetically speaking, how would I end up with a contenteditable image in a plaintext-only editing area?

<div style="-webkit-user-modify: read-write-plaintext-only"><img src="~~"></div>

There's nothing that prevents authors from putting random stuff inside an element that's read-write-plaintext-only. I know it's pretty bad :(
Comment 33 Daniel Cheng 2011-10-18 22:31:06 PDT
(In reply to comment #32)
> (In reply to comment #31)
> > Hypothetically speaking, how would I end up with a contenteditable image in a plaintext-only editing area?
> 
> <div style="-webkit-user-modify: read-write-plaintext-only"><img src="~~"></div>
> 
> There's nothing that prevents authors from putting random stuff inside an element that's read-write-plaintext-only. I know it's pretty bad :(

How odd.
I notice isContentEditable() calls updateLayoutIgnorePendingStylesheets(). Do you know how important it is for me to call it here?
Comment 34 Ryosuke Niwa 2011-10-18 22:33:52 PDT
(In reply to comment #33)
> How odd.
> I notice isContentEditable() calls updateLayoutIgnorePendingStylesheets(). Do you know how important it is for me to call it here?

You should be using isContentEditable unless you're certain that the layout is up-to-date. We need style-recalc to figure out if an element is editable or not.
Comment 35 Tony Chang 2011-10-19 09:47:28 PDT
Comment on attachment 111527 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=111527&action=review

In general, this approach seems fine.  r- for Ryosuke's comments.

> LayoutTests/fast/events/drag-selects-image.html:16
> +        eventSender.leapForward(500);

Nit: Making this a shorter time (like 300ms) will be nicer for QT and GTK+. I think they implement leapForward with sleep().

> LayoutTests/fast/events/drag-selects-image.html:49
> +<p>This tests that images are properly left selected or unselected when an image drag is started. On
> +non-Mac platforms, neither image should be selected when dragging. On Mac, only the image in the

Is the Mac/non-Mac part still accurate?  It would also be nice to provide instructions on how to run the test manually.
Comment 36 Daniel Cheng 2011-10-19 13:18:53 PDT
Created attachment 111663 [details]
Patch
Comment 37 Tony Chang 2011-10-19 13:39:21 PDT
Comment on attachment 111663 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=111663&action=review

> LayoutTests/fast/events/drag-selects-image.html:16
> +        eventSender.leapForward(100);

It looks like the drag delay on Mac is .15s, but maybe that's only for text?
Comment 38 Daniel Cheng 2011-10-19 13:41:25 PDT
Comment on attachment 111663 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=111663&action=review

>> LayoutTests/fast/events/drag-selects-image.html:16
>> +        eventSender.leapForward(100);
> 
> It looks like the drag delay on Mac is .15s, but maybe that's only for text?

Right, I believe that's only for text.
Comment 39 Daniel Cheng 2011-10-19 14:37:06 PDT
Committed r97878: <http://trac.webkit.org/changeset/97878>
Comment 40 Simon Fraser (smfr) 2011-10-20 15:24:50 PDT
This may have broken a test on Mac: bug 70551