Bug 49141 - Use Blob URL instead of webkit-fake-url when pasting an image
Summary: Use Blob URL instead of webkit-fake-url when pasting an image
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: HTML Editing (show other bugs)
Version: 528+ (Nightly build)
Hardware: Macintosh OS X 10.6
: P2 Major
Assignee: Chris Dumez
URL:
Keywords:
Depends on: 163100
Blocks:
  Show dependency treegraph
 
Reported: 2010-11-07 06:39 PST by Chris J. Shull
Modified: 2017-04-04 08:24 PDT (History)
20 users (show)

See Also:


Attachments
WIP Patch (2.62 KB, patch)
2016-11-08 13:00 PST, Chris Dumez
no flags Details | Formatted Diff | Diff
Patch (8.68 KB, patch)
2016-11-08 13:43 PST, Chris Dumez
no flags Details | Formatted Diff | Diff
Patch (11.14 KB, patch)
2016-11-08 13:51 PST, Chris Dumez
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Chris J. Shull 2010-11-07 06:39:42 PST
When an element is set to be contentEditable, images can be pasted into the element. Currently the img src is set to a webkit-fake-url which makes the image data inaccessible via JS (I think because the same domain policy?).
However, this makes no sense in this case; if you are going to let people paste text into an element, and let JS access the text, then the same should be true for images. 
One way which might get around this is if the img's src was not a webkit-fake-url, but a data uri (I would guess, but I am not certain).

Example Use Case:
Taking a screenshot to clipboard, which can be pasted into an online bug reporter.
(Much easier than; screenshot to file, drag file onto file input.)
Comment 1 Orion Poplawski 2012-01-27 09:54:16 PST
I think we may be seeing this or something similar with the yui editor in Alfresco with OS X clients.  Pasting in images usually works, but at some point the image source will change to webkit-fake-url.
Comment 2 Vinod Seraphin 2014-03-08 18:03:57 PST
Firefox pastes data: uris (inline image data) which works really well, and Chrome (even when it was webkit based) supported establishing a paste event listener, having a 'file' type of clipboard data iteme on a paste, and using FileReader to allow accessing that data as a data: url value via readAsDataURL method.

Safari has fallen behind badly by not supporting some way paste clipboard image can be implemented.  Fixing this defect would greatly help.
Comment 3 Mike Sherov 2014-07-15 11:01:16 PDT
This is exacerbated by the fact that CSP directives would need to add webkit-fake-url: to all directives. Switching to a data-uri normalizes the behavior.
Comment 4 Mike West 2014-07-15 21:12:36 PDT
Both Blink and WebKit should have mechanisms for annotating certain schemes as bypassing CSP. For example, images at 'chrome-extension:' URLs will load even in the presence of `img-src 'none'`.

See `SchemeRegistry::schemeShouldBypassContentSecurityPolicy()` for the implementation, if that's the direction you'd like to go.
Comment 5 Aleksander Nowodziński 2015-05-05 04:42:24 PDT
This issue breaks pasting/upload feature in CKEditor (see more in https://dev.ckeditor.com/ticket/13029) and other WYSIWYG editors because there is no way to obtain image data pasted into the browser. 

It would be great to see this issue fixed after all these years, i.e. by using a base64-encoded resource representation instead of webkit-fake-url, like in other modern browsers.
Comment 6 Mark Schwarz 2015-09-16 13:57:26 PDT
This also affects users of Atlassian JIRA: https://confluence.atlassian.com/jira/attaching-a-screenshot-185729550.html.  

See Atlassian comment from Nov 5, 2014 at the link above.
Comment 7 Markandey Singh 2015-12-15 16:37:33 PST
I wonder when this bug will be fixed for safari. This is clearly one of the big limitation on safari where copy paste of images does not work. Would be nice if webkit supports similar implementation like Firefox where pasted image contains a data url instead for webkit-fake-url
Comment 8 Mateusz 2016-02-11 05:17:49 PST
Guys, this bug is 5 years old and Safari is the only browser which is not capable of pasting images. It affects a lot software like JIRA, Google Docs, Online Office,  CKEditor and many many more. Is there any plan to fix that?
Comment 9 Martin Edenhofer 2016-07-12 23:16:10 PDT
I confirm this in safari 9.1.1 (11601.6.17).
Comment 10 Mateusz 2016-10-05 07:15:15 PDT
The same applies to iOS10.0.2 - nothing changed and the bug still exists.
Comment 11 Chris Dumez 2016-10-05 09:07:40 PDT
Does anyone have a relatively simple test case?
Comment 12 Mateusz 2016-10-06 02:02:33 PDT
Sure, here are the steps:
1. open a page with images, e.g: http://www.bbc.com/news/science-environment-37511861 (or you can open google image search and type "landscape")
2. right click on any image (or hold on iPad) and pick "Copy picture"
3. open this fiddle: https://jsfiddle.net/madkdzre/ and paste the image into frame.
4.  inspect its url in dev tools.

Result: you'll see that url of the image looks like this "webkit-fake-url://FA85EB54-6F2A-4CC8-86A8-34D527A68789/blablabla.png" instead of url to pasted image. And it doesn't matter if you copy image from finder or from browser. It also doesn't matter if the image copied from browser was referred by url or dataURL string (like in case of google image search).

This is really annoying on desktops, but on iOS Safari is usually the only browser, and because of that bug you cannot paste images into any online editor. That's the biggest problem.
Comment 13 Chris Dumez 2016-10-06 13:42:49 PDT
(In reply to comment #12)
> Sure, here are the steps:
> 1. open a page with images, e.g:
> http://www.bbc.com/news/science-environment-37511861 (or you can open google
> image search and type "landscape")
> 2. right click on any image (or hold on iPad) and pick "Copy picture"
> 3. open this fiddle: https://jsfiddle.net/madkdzre/ and paste the image into
> frame.
> 4.  inspect its url in dev tools.
> 
> Result: you'll see that url of the image looks like this
> "webkit-fake-url://FA85EB54-6F2A-4CC8-86A8-34D527A68789/blablabla.png"
> instead of url to pasted image. And it doesn't matter if you copy image from
> finder or from browser. It also doesn't matter if the image copied from
> browser was referred by url or dataURL string (like in case of google image
> search).
> 
> This is really annoying on desktops, but on iOS Safari is usually the only
> browser, and because of that bug you cannot paste images into any online
> editor. That's the biggest problem.

Thanks for the test case, I can reproduce the issue.

Interestingly, it seems to work fine if I copy from Firefox or Preview.app and then paste in Safari. The issue only seems to happen if we copy from Safari.
Comment 14 Mateusz 2016-10-06 13:55:04 PDT
 
> Interestingly, it seems to work fine if I copy from Firefox or Preview.app
> and then paste in Safari. The issue only seems to happen if we copy from
> Safari.

On desktop that's true but on iOS all the images pasted into contenteditable are referenced by fake url (no matter where they are copied from). I've just tried to paste image copied from the following apps: Notes, Pages, Keynote and Gmail and all of them have been pasted as webkit-fake-url.
Comment 15 Chris Dumez 2016-10-06 14:56:17 PDT
On my Mac, it seems we use RTFD format to copy/paste the image by default. There is an internal setting (preferMIMETypeForImages) to stop using RTFD format for images and simply treat them as images. If I turn on the setting, the image seems to paste fine.

The bug must be related to our RTFD formatting / processing code.
Comment 16 Chris Dumez 2016-10-06 14:57:54 PDT
(In reply to comment #15)
> On my Mac, it seems we use RTFD format to copy/paste the image by default.
> There is an internal setting (preferMIMETypeForImages) to stop using RTFD
> format for images and simply treat them as images. If I turn on the setting,
> the image seems to paste fine.
> 
> The bug must be related to our RTFD formatting / processing code.

Also note that when not using RTFD, the pasted image looks like:
<img src="webkit-fake-url://1d5cfcc6-f668-47ca-a47d-6e118d68df50/image.tiff">

It is also using a webkit-fake-url and yet the image is displayed correctly.
Comment 17 Chris Dumez 2016-10-06 15:30:42 PDT
In EditorMac.mm:
Editor::createFragmentAndAddResources(NSAttributedString *string)

We:
1. Defer loading
2. Create a DocumentFragment as a list of resources from the DTFD Data
3. Register the resources with the DocumentLoader (in this case, there is one resource with a webkit-fake-url and its associated data)
4. Stop load deferring

However, with WebKit2, I see the following
Calling createFragment()
WebLoaderStrategy::scheduleLoad(webkit-fake-url://36D48F33-17C4-4958-8856-569F23B788CE/_91448609_rosetta_crash_land_624_v3.png)
DocumentLoader::scheduleArchiveLoad(webkit-fake-url://36D48F33-17C4-4958-8856-569F23B788CE/_91448609_rosetta_crash_land_624_v3.png)
Adding archive resource to loader, url: webkit-fake-url://36D48F33-17C4-4958-8856-569F23B788CE/_91448609_rosetta_crash_land_624_v3.png // Too late

So load deferring seems to fail we try to load webkit-fake-url://36D48F33-17C4-4958-8856-569F23B788CE/_91448609_rosetta_crash_land_624_v3.png *before* we had a chance registering it with the DocumentLoader.
Comment 18 Chris Dumez 2016-10-06 15:48:41 PDT
(In reply to comment #17)
> In EditorMac.mm:
> Editor::createFragmentAndAddResources(NSAttributedString *string)
> 
> We:
> 1. Defer loading
> 2. Create a DocumentFragment as a list of resources from the DTFD Data
> 3. Register the resources with the DocumentLoader (in this case, there is
> one resource with a webkit-fake-url and its associated data)
> 4. Stop load deferring
> 
> However, with WebKit2, I see the following
> Calling createFragment()
> WebLoaderStrategy::scheduleLoad(webkit-fake-url://36D48F33-17C4-4958-8856-
> 569F23B788CE/_91448609_rosetta_crash_land_624_v3.png)
> DocumentLoader::scheduleArchiveLoad(webkit-fake-url://36D48F33-17C4-4958-
> 8856-569F23B788CE/_91448609_rosetta_crash_land_624_v3.png)
> Adding archive resource to loader, url:
> webkit-fake-url://36D48F33-17C4-4958-8856-569F23B788CE/
> _91448609_rosetta_crash_land_624_v3.png // Too late
> 
> So load deferring seems to fail we try to load
> webkit-fake-url://36D48F33-17C4-4958-8856-569F23B788CE/
> _91448609_rosetta_crash_land_624_v3.png *before* we had a chance registering
> it with the DocumentLoader.

Call track that causes the load to start:
1   0x105125d03 WebKit::WebLoaderStrategy::loadResource(WebCore::Frame&, WebCore::CachedResource&, WebCore::ResourceRequest const&, WebCore::ResourceLoaderOptions const&)
2   0x1089b892e WebCore::CachedResource::load(WebCore::CachedResourceLoader&)
3   0x1089bde4c WebCore::CachedResourceLoader::requestResource(WebCore::CachedResource::Type, WebCore::CachedResourceRequest&&)
4   0x1089bcb79 WebCore::CachedResourceLoader::requestImage(WebCore::CachedResourceRequest&&)
5   0x108eb322b WebCore::ImageLoader::updateFromElement()
6   0x108dd7316 WebCore::HTMLImageElement::selectImageSource()
7   0x108c43ced WebCore::Element::attributeChanged(WebCore::QualifiedName const&, WTF::AtomicString const&, WTF::AtomicString const&, WebCore::Element::AttributeModificationReason)
8   0x108c44709 WebCore::Element::parserSetAttributes(WTF::Vector<WebCore::Attribute, 0ul, WTF::CrashOnOverflow, 16ul> const&)
9   0x108da3038 WebCore::HTMLConstructionSite::createHTMLElementOrFindCustomElementInterface(WebCore::AtomicHTMLToken&, WebCore::JSCustomElementInterface**)
10  0x108da338d WebCore::HTMLConstructionSite::insertSelfClosingHTMLElement(WebCore::AtomicHTMLToken&)
11  0x108e4de00 WebCore::HTMLTreeBuilder::processStartTagForInBody(WebCore::AtomicHTMLToken&)
12  0x108e4a79a WebCore::HTMLTreeBuilder::processStartTag(WebCore::AtomicHTMLToken&)
13  0x108e49a18 WebCore::HTMLTreeBuilder::constructTree(WebCore::AtomicHTMLToken&)
14  0x108db6356 WebCore::HTMLDocumentParser::constructTreeFromHTMLToken(WebCore::HTMLTokenizer::TokenPtr&)
15  0x108db617c WebCore::HTMLDocumentParser::pumpTokenizerLoop(WebCore::HTMLDocumentParser::SynchronousMode, bool, WebCore::PumpSession&)
16  0x108db5b83 WebCore::HTMLDocumentParser::pumpTokenizer(WebCore::HTMLDocumentParser::SynchronousMode)
17  0x108db6432 WebCore::HTMLDocumentParser::insert(WebCore::SegmentedString const&)
18  0x108db6e28 WebCore::HTMLDocumentParser::parseDocumentFragment(WTF::String const&, WebCore::DocumentFragment&, WebCore::Element&, WebCore::ParserContentPolicy)
19  0x1093e0a53 WebCore::createFragmentForInnerOuterHTML(WebCore::Element&, WTF::String const&, WebCore::ParserContentPolicy, int&)
20  0x108c494db WebCore::Element::setInnerHTML(WTF::String const&, int&)
21  0x105ae21c8 -[DOMElement setInnerHTML:]
22  0x7fffac179539 -[NSHTMLWriter readDocumentFragment:]
23  0x7fffac1795e5 -[NSHTMLWriter documentFragmentForDocument:]
24  0x7fffac17f795 -[NSAttributedString(NSAttributedStringUIFoundationAdditions) _documentFromRange:document:documentAttributes:subresources:]
25  0x105b6b898 _WebCreateFragment
26  0x108c346ce WebCore::Editor::createFragment(NSAttributedString*)
27  0x108c3c623 WebCore::Editor::createFragmentAndAddResources(NSAttributedString*)
Comment 19 Mateusz 2016-10-06 15:51:07 PDT
Please also note that displaying the pasted image is one problem and using it is another problem. I mean even if the pasted image displays correctly (I can see it even if it has webkit-fake-url) I can't use it from JS code - so e.g. when I paste it to Gmail online app it looks fine, but when I send the email it's gone. I've tried also to draw it on canvas and get as dataURL but of course that also fails because of security reasons (image comes from a different protocol than the web app).
Comment 20 Chris Dumez 2016-10-06 20:02:49 PDT
Ok, so I think I have a plan (in several steps):
1. Add HTML the pasteboard when copying an image in Safari, instead of RTFD. This matches Firefox and pasting this in Safari works great (Image shows and has a regular URL). Note that HTML has priority over images in WebKit when reading from the pasteboard.
2. When reading an image from the pasteboard, check if there is also a URL in the pasteboard and use it if there is, instead of creating a WebKit-fake-URL. This is testable by copying an image from Chrome to Safari. The pasted image would then have
a proper URL instead of a webkit-fake-url.
3. When reading an image from the pasteboard, and if there is no URL in the pasteboard, construct a Blob URL instead of a webkit-fake-url. Blob URLs have the benefit of being standard and convenient to use from JavaScript.
Comment 21 Ryosuke Niwa 2016-10-06 20:11:04 PDT
(In reply to comment #20)
> Ok, so I think I have a plan (in several steps):
> 1. Add HTML the pasteboard when copying an image in Safari, instead of RTFD.
> This matches Firefox and pasting this in Safari works great (Image shows and
> has a regular URL). Note that HTML has priority over images in WebKit when
> reading from the pasteboard.

I don't think we can stop placing RTFD content into the pasteboard because other apps that don't support webarchive may depend on it being available (for pasting the content copied in WebKit).
Comment 22 Chris Dumez 2016-10-06 20:17:38 PDT
(In reply to comment #20)
> Ok, so I think I have a plan (in several steps):
> 1. Add HTML the pasteboard when copying an image in Safari, instead of RTFD.
> This matches Firefox and pasting this in Safari works great (Image shows and
> has a regular URL). Note that HTML has priority over images in WebKit when
> reading from the pasteboard.
> 2. When reading an image from the pasteboard, check if there is also a URL
> in the pasteboard and use it if there is, instead of creating a
> WebKit-fake-URL. This is testable by copying an image from Chrome to Safari.
> The pasted image would then have
> a proper URL instead of a webkit-fake-url.
> 3. When reading an image from the pasteboard, and if there is no URL in the
> pasteboard, construct a Blob URL instead of a webkit-fake-url. Blob URLs
> have the benefit of being standard and convenient to use from JavaScript.

Ryosuke pointed out offline that not writing RTFD to the pasteboard anymore is risky compatibility-wise. Therefore, my fallback plan is to keep writing RTFD to the pasteboard but add HTML as well. HTML has priority over RTFD when reading the pasteboard in WebKit so this should be fine as well.
Comment 23 Mike Sherov 2016-10-07 03:16:05 PDT
If the plan is to use Blob URLs instead of data-uris (like Firefox), will there be a way to convert them to data uris like Chrome does? That's main use case here: pasting an image into a WYSIWYG editor and being able to save the image.
Comment 24 Aleksander Nowodziński 2016-10-07 04:27:03 PDT
(In reply to comment #23)
> If the plan is to use Blob URLs instead of data-uris (like Firefox), will
> there be a way to convert them to data uris like Chrome does? That's main
> use case here: pasting an image into a WYSIWYG editor and being able to save
> the image.

So +1. 

The point is that whatever the output in DOM (webkit-fake-url or data-uri in the <img> tag), an easy way to obtain the data-uri in the paste event is needed, so the content that has been pasted in the WYSIWYG editor can be i.e. uploaded to the server. 

At this moment, the webkit-fake-url gives no way to the developers to obtain the actual image and do anything with it. Users paste/drag&drop tons of content into WYSIWYG (contenteditable) editors like that.
Comment 25 Chris Dumez 2016-10-07 08:48:52 PDT
(In reply to comment #23)
> If the plan is to use Blob URLs instead of data-uris (like Firefox), will
> there be a way to convert them to data uris like Chrome does? That's main
> use case here: pasting an image into a WYSIWYG editor and being able to save
> the image.

What is this API that Chrome has to convert to data URIs and that WebKit does not have?
Comment 26 Chris Dumez 2016-10-07 10:30:14 PDT
(In reply to comment #23)
> If the plan is to use Blob URLs instead of data-uris (like Firefox), will
> there be a way to convert them to data uris like Chrome does? That's main
> use case here: pasting an image into a WYSIWYG editor and being able to save
> the image.

I have confirmed that Firefox is indeed using Data URLs rather than Blob URLs when pasting images from Preview.app.
Comment 27 Chris Dumez 2016-10-07 14:40:49 PDT
(In reply to comment #25)
> (In reply to comment #23)
> > If the plan is to use Blob URLs instead of data-uris (like Firefox), will
> > there be a way to convert them to data uris like Chrome does? That's main
> > use case here: pasting an image into a WYSIWYG editor and being able to save
> > the image.
> 
> What is this API that Chrome has to convert to data URIs and that WebKit
> does not have?

It seems like WebKit would easily allow you to convert a Blob into a Data URL using FileReader.readAsDataURL().
Comment 28 Chris Dumez 2016-11-08 13:00:17 PST
Created attachment 294178 [details]
WIP Patch
Comment 29 Chris Dumez 2016-11-08 13:43:11 PST
Created attachment 294180 [details]
Patch
Comment 30 Chris Dumez 2016-11-08 13:51:28 PST
Created attachment 294181 [details]
Patch
Comment 31 Darin Adler 2016-11-08 14:37:58 PST
Comment on attachment 294181 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=294181&action=review

> Source/WebCore/editing/mac/EditorMac.mm:618
> +    Vector<uint8_t> data;
> +    data.append(buffer->data(), buffer->size());
> +    auto blob = Blob::create(WTFMove(data), type);

Can we write it this way?

    auto blob = Blob::create(Vector<uint8_t> { buffer->data(), buffer->size() }, type);
Comment 32 Chris Dumez 2016-11-08 16:15:12 PST
(In reply to comment #31)
> Comment on attachment 294181 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=294181&action=review
> 
> > Source/WebCore/editing/mac/EditorMac.mm:618
> > +    Vector<uint8_t> data;
> > +    data.append(buffer->data(), buffer->size());
> > +    auto blob = Blob::create(WTFMove(data), type);
> 
> Can we write it this way?
> 
>     auto blob = Blob::create(Vector<uint8_t> { buffer->data(),
> buffer->size() }, type);

Vector has no such constructor AFAICT.
Comment 33 WebKit Commit Bot 2016-11-09 11:17:22 PST
Comment on attachment 294181 [details]
Patch

Clearing flags on attachment: 294181

Committed r208451: <http://trac.webkit.org/changeset/208451>
Comment 34 WebKit Commit Bot 2016-11-09 11:17:30 PST
All reviewed patches have been landed.  Closing bug.
Comment 35 Ebrahim Byagowi 2017-04-04 07:11:36 PDT
Similar still available issue on copy paste on other places on the web: Bug 170449