Bug 49141 – Use Blob URL instead of webkit-fake-url when pasting an image

• Status: RESOLVED
• Resolution: FIXED
• Component: HTML Editing

[Chris J. Shull]

When an element is set to be contentEditable, images can be pasted into the element. Currently the img src is set to a webkit-fake-url which makes the image data inaccessible via JS (I think because the same domain policy?).
However, this makes no sense in this case; if you are going to let people paste text into an element, and let JS access the text, then the same should be true for images. 
One way which might get around this is if the img's src was not a webkit-fake-url, but a data uri (I would guess, but I am not certain).

Example Use Case:
Taking a screenshot to clipboard, which can be pasted into an online bug reporter.
(Much easier than; screenshot to file, drag file onto file input.)

[Orion Poplawski]

I think we may be seeing this or something similar with the yui editor in Alfresco with OS X clients.  Pasting in images usually works, but at some point the image source will change to webkit-fake-url.

[Vinod Seraphin]

Firefox pastes data: uris (inline image data) which works really well, and Chrome (even when it was webkit based) supported establishing a paste event listener, having a 'file' type of clipboard data iteme on a paste, and using FileReader to allow accessing that data as a data: url value via readAsDataURL method.

Safari has fallen behind badly by not supporting some way paste clipboard image can be implemented.  Fixing this defect would greatly help.

[Mike Sherov]

This is exacerbated by the fact that CSP directives would need to add webkit-fake-url: to all directives. Switching to a data-uri normalizes the behavior.

[Mike West]

Both Blink and WebKit should have mechanisms for annotating certain schemes as bypassing CSP. For example, images at 'chrome-extension:' URLs will load even in the presence of `img-src 'none'`.

See `SchemeRegistry::schemeShouldBypassContentSecurityPolicy()` for the implementation, if that's the direction you'd like to go.

[Aleksander Nowodziński]

This issue breaks pasting/upload feature in CKEditor (see more in https://dev.ckeditor.com/ticket/13029) and other WYSIWYG editors because there is no way to obtain image data pasted into the browser. 

It would be great to see this issue fixed after all these years, i.e. by using a base64-encoded resource representation instead of webkit-fake-url, like in other modern browsers.

[Mark Schwarz]

This also affects users of Atlassian JIRA: https://confluence.atlassian.com/jira/attaching-a-screenshot-185729550.html.  

See Atlassian comment from Nov 5, 2014 at the link above.

[Markandey Singh]

I wonder when this bug will be fixed for safari. This is clearly one of the big limitation on safari where copy paste of images does not work. Would be nice if webkit supports similar implementation like Firefox where pasted image contains a data url instead for webkit-fake-url

[Mateusz]

Guys, this bug is 5 years old and Safari is the only browser which is not capable of pasting images. It affects a lot software like JIRA, Google Docs, Online Office,  CKEditor and many many more. Is there any plan to fix that?

[Martin Edenhofer]

I confirm this in safari 9.1.1 (11601.6.17).

[Mateusz]

The same applies to iOS10.0.2 - nothing changed and the bug still exists.

[Chris Dumez]

Does anyone have a relatively simple test case?

[Mateusz]

Sure, here are the steps:
1. open a page with images, e.g: http://www.bbc.com/news/science-environment-37511861 (or you can open google image search and type "landscape")
2. right click on any image (or hold on iPad) and pick "Copy picture"
3. open this fiddle: https://jsfiddle.net/madkdzre/ and paste the image into frame.
4.  inspect its url in dev tools.

Result: you'll see that url of the image looks like this "webkit-fake-url://FA85EB54-6F2A-4CC8-86A8-34D527A68789/blablabla.png" instead of url to pasted image. And it doesn't matter if you copy image from finder or from browser. It also doesn't matter if the image copied from browser was referred by url or dataURL string (like in case of google image search).

This is really annoying on desktops, but on iOS Safari is usually the only browser, and because of that bug you cannot paste images into any online editor. That's the biggest problem.

[Chris Dumez]

(In reply to comment #12)
> Sure, here are the steps:
> 1. open a page with images, e.g:
> http://www.bbc.com/news/science-environment-37511861 (or you can open google
> image search and type "landscape")
> 2. right click on any image (or hold on iPad) and pick "Copy picture"
> 3. open this fiddle: https://jsfiddle.net/madkdzre/ and paste the image into
> frame.
> 4.  inspect its url in dev tools.
> 
> Result: you'll see that url of the image looks like this
> "webkit-fake-url://FA85EB54-6F2A-4CC8-86A8-34D527A68789/blablabla.png"
> instead of url to pasted image. And it doesn't matter if you copy image from
> finder or from browser. It also doesn't matter if the image copied from
> browser was referred by url or dataURL string (like in case of google image
> search).
> 
> This is really annoying on desktops, but on iOS Safari is usually the only
> browser, and because of that bug you cannot paste images into any online
> editor. That's the biggest problem.

Thanks for the test case, I can reproduce the issue.

Interestingly, it seems to work fine if I copy from Firefox or Preview.app and then paste in Safari. The issue only seems to happen if we copy from Safari.

[Mateusz]

 
> Interestingly, it seems to work fine if I copy from Firefox or Preview.app
> and then paste in Safari. The issue only seems to happen if we copy from
> Safari.

On desktop that's true but on iOS all the images pasted into contenteditable are referenced by fake url (no matter where they are copied from). I've just tried to paste image copied from the following apps: Notes, Pages, Keynote and Gmail and all of them have been pasted as webkit-fake-url.

[Chris Dumez]

On my Mac, it seems we use RTFD format to copy/paste the image by default. There is an internal setting (preferMIMETypeForImages) to stop using RTFD format for images and simply treat them as images. If I turn on the setting, the image seems to paste fine.

The bug must be related to our RTFD formatting / processing code.

[Chris Dumez]

(In reply to comment #15)
> On my Mac, it seems we use RTFD format to copy/paste the image by default.
> There is an internal setting (preferMIMETypeForImages) to stop using RTFD
> format for images and simply treat them as images. If I turn on the setting,
> the image seems to paste fine.
> 
> The bug must be related to our RTFD formatting / processing code.

Also note that when not using RTFD, the pasted image looks like:
<img src="webkit-fake-url://1d5cfcc6-f668-47ca-a47d-6e118d68df50/image.tiff">

It is also using a webkit-fake-url and yet the image is displayed correctly.

[Chris Dumez]

In EditorMac.mm:
Editor::createFragmentAndAddResources(NSAttributedString *string)

We:
1. Defer loading
2. Create a DocumentFragment as a list of resources from the DTFD Data
3. Register the resources with the DocumentLoader (in this case, there is one resource with a webkit-fake-url and its associated data)
4. Stop load deferring

However, with WebKit2, I see the following
Calling createFragment()
WebLoaderStrategy::scheduleLoad(webkit-fake-url://36D48F33-17C4-4958-8856-569F23B788CE/_91448609_rosetta_crash_land_624_v3.png)
DocumentLoader::scheduleArchiveLoad(webkit-fake-url://36D48F33-17C4-4958-8856-569F23B788CE/_91448609_rosetta_crash_land_624_v3.png)
Adding archive resource to loader, url: webkit-fake-url://36D48F33-17C4-4958-8856-569F23B788CE/_91448609_rosetta_crash_land_624_v3.png // Too late

So load deferring seems to fail we try to load webkit-fake-url://36D48F33-17C4-4958-8856-569F23B788CE/_91448609_rosetta_crash_land_624_v3.png *before* we had a chance registering it with the DocumentLoader.

[Chris Dumez]

(In reply to comment #17)
> In EditorMac.mm:
> Editor::createFragmentAndAddResources(NSAttributedString *string)
> 
> We:
> 1. Defer loading
> 2. Create a DocumentFragment as a list of resources from the DTFD Data
> 3. Register the resources with the DocumentLoader (in this case, there is
> one resource with a webkit-fake-url and its associated data)
> 4. Stop load deferring
> 
> However, with WebKit2, I see the following
> Calling createFragment()
> WebLoaderStrategy::scheduleLoad(webkit-fake-url://36D48F33-17C4-4958-8856-
> 569F23B788CE/_91448609_rosetta_crash_land_624_v3.png)
> DocumentLoader::scheduleArchiveLoad(webkit-fake-url://36D48F33-17C4-4958-
> 8856-569F23B788CE/_91448609_rosetta_crash_land_624_v3.png)
> Adding archive resource to loader, url:
> webkit-fake-url://36D48F33-17C4-4958-8856-569F23B788CE/
> _91448609_rosetta_crash_land_624_v3.png // Too late
> 
> So load deferring seems to fail we try to load
> webkit-fake-url://36D48F33-17C4-4958-8856-569F23B788CE/
> _91448609_rosetta_crash_land_624_v3.png *before* we had a chance registering
> it with the DocumentLoader.

Call track that causes the load to start:
1   0x105125d03 WebKit::WebLoaderStrategy::loadResource(WebCore::Frame&, WebCore::CachedResource&, WebCore::ResourceRequest const&, WebCore::ResourceLoaderOptions const&)
2   0x1089b892e WebCore::CachedResource::load(WebCore::CachedResourceLoader&)
3   0x1089bde4c WebCore::CachedResourceLoader::requestResource(WebCore::CachedResource::Type, WebCore::CachedResourceRequest&&)
4   0x1089bcb79 WebCore::CachedResourceLoader::requestImage(WebCore::CachedResourceRequest&&)
5   0x108eb322b WebCore::ImageLoader::updateFromElement()
6   0x108dd7316 WebCore::HTMLImageElement::selectImageSource()
7   0x108c43ced WebCore::Element::attributeChanged(WebCore::QualifiedName const&, WTF::AtomicString const&, WTF::AtomicString const&, WebCore::Element::AttributeModificationReason)
8   0x108c44709 WebCore::Element::parserSetAttributes(WTF::Vector<WebCore::Attribute, 0ul, WTF::CrashOnOverflow, 16ul> const&)
9   0x108da3038 WebCore::HTMLConstructionSite::createHTMLElementOrFindCustomElementInterface(WebCore::AtomicHTMLToken&, WebCore::JSCustomElementInterface**)
10  0x108da338d WebCore::HTMLConstructionSite::insertSelfClosingHTMLElement(WebCore::AtomicHTMLToken&)
11  0x108e4de00 WebCore::HTMLTreeBuilder::processStartTagForInBody(WebCore::AtomicHTMLToken&)
12  0x108e4a79a WebCore::HTMLTreeBuilder::processStartTag(WebCore::AtomicHTMLToken&)
13  0x108e49a18 WebCore::HTMLTreeBuilder::constructTree(WebCore::AtomicHTMLToken&)
14  0x108db6356 WebCore::HTMLDocumentParser::constructTreeFromHTMLToken(WebCore::HTMLTokenizer::TokenPtr&)
15  0x108db617c WebCore::HTMLDocumentParser::pumpTokenizerLoop(WebCore::HTMLDocumentParser::SynchronousMode, bool, WebCore::PumpSession&)
16  0x108db5b83 WebCore::HTMLDocumentParser::pumpTokenizer(WebCore::HTMLDocumentParser::SynchronousMode)
17  0x108db6432 WebCore::HTMLDocumentParser::insert(WebCore::SegmentedString const&)
18  0x108db6e28 WebCore::HTMLDocumentParser::parseDocumentFragment(WTF::String const&, WebCore::DocumentFragment&, WebCore::Element&, WebCore::ParserContentPolicy)
19  0x1093e0a53 WebCore::createFragmentForInnerOuterHTML(WebCore::Element&, WTF::String const&, WebCore::ParserContentPolicy, int&)
20  0x108c494db WebCore::Element::setInnerHTML(WTF::String const&, int&)
21  0x105ae21c8 -[DOMElement setInnerHTML:]
22  0x7fffac179539 -[NSHTMLWriter readDocumentFragment:]
23  0x7fffac1795e5 -[NSHTMLWriter documentFragmentForDocument:]
24  0x7fffac17f795 -[NSAttributedString(NSAttributedStringUIFoundationAdditions) _documentFromRange:document:documentAttributes:subresources:]
25  0x105b6b898 _WebCreateFragment
26  0x108c346ce WebCore::Editor::createFragment(NSAttributedString*)
27  0x108c3c623 WebCore::Editor::createFragmentAndAddResources(NSAttributedString*)

[Mateusz]

Please also note that displaying the pasted image is one problem and using it is another problem. I mean even if the pasted image displays correctly (I can see it even if it has webkit-fake-url) I can't use it from JS code - so e.g. when I paste it to Gmail online app it looks fine, but when I send the email it's gone. I've tried also to draw it on canvas and get as dataURL but of course that also fails because of security reasons (image comes from a different protocol than the web app).

[Chris Dumez]

Ok, so I think I have a plan (in several steps):
1. Add HTML the pasteboard when copying an image in Safari, instead of RTFD. This matches Firefox and pasting this in Safari works great (Image shows and has a regular URL). Note that HTML has priority over images in WebKit when reading from the pasteboard.
2. When reading an image from the pasteboard, check if there is also a URL in the pasteboard and use it if there is, instead of creating a WebKit-fake-URL. This is testable by copying an image from Chrome to Safari. The pasted image would then have
a proper URL instead of a webkit-fake-url.
3. When reading an image from the pasteboard, and if there is no URL in the pasteboard, construct a Blob URL instead of a webkit-fake-url. Blob URLs have the benefit of being standard and convenient to use from JavaScript.

[Ryosuke Niwa]

(In reply to comment #20)
> Ok, so I think I have a plan (in several steps):
> 1. Add HTML the pasteboard when copying an image in Safari, instead of RTFD.
> This matches Firefox and pasting this in Safari works great (Image shows and
> has a regular URL). Note that HTML has priority over images in WebKit when
> reading from the pasteboard.

I don't think we can stop placing RTFD content into the pasteboard because other apps that don't support webarchive may depend on it being available (for pasting the content copied in WebKit).

[Chris Dumez]

(In reply to comment #20)
> Ok, so I think I have a plan (in several steps):
> 1. Add HTML the pasteboard when copying an image in Safari, instead of RTFD.
> This matches Firefox and pasting this in Safari works great (Image shows and
> has a regular URL). Note that HTML has priority over images in WebKit when
> reading from the pasteboard.
> 2. When reading an image from the pasteboard, check if there is also a URL
> in the pasteboard and use it if there is, instead of creating a
> WebKit-fake-URL. This is testable by copying an image from Chrome to Safari.
> The pasted image would then have
> a proper URL instead of a webkit-fake-url.
> 3. When reading an image from the pasteboard, and if there is no URL in the
> pasteboard, construct a Blob URL instead of a webkit-fake-url. Blob URLs
> have the benefit of being standard and convenient to use from JavaScript.

Ryosuke pointed out offline that not writing RTFD to the pasteboard anymore is risky compatibility-wise. Therefore, my fallback plan is to keep writing RTFD to the pasteboard but add HTML as well. HTML has priority over RTFD when reading the pasteboard in WebKit so this should be fine as well.

[Mike Sherov]

If the plan is to use Blob URLs instead of data-uris (like Firefox), will there be a way to convert them to data uris like Chrome does? That's main use case here: pasting an image into a WYSIWYG editor and being able to save the image.

[Aleksander Nowodziński]

(In reply to comment #23)
> If the plan is to use Blob URLs instead of data-uris (like Firefox), will
> there be a way to convert them to data uris like Chrome does? That's main
> use case here: pasting an image into a WYSIWYG editor and being able to save
> the image.

So +1. 

The point is that whatever the output in DOM (webkit-fake-url or data-uri in the <img> tag), an easy way to obtain the data-uri in the paste event is needed, so the content that has been pasted in the WYSIWYG editor can be i.e. uploaded to the server. 

At this moment, the webkit-fake-url gives no way to the developers to obtain the actual image and do anything with it. Users paste/drag&drop tons of content into WYSIWYG (contenteditable) editors like that.

[Chris Dumez]

(In reply to comment #23)
> If the plan is to use Blob URLs instead of data-uris (like Firefox), will
> there be a way to convert them to data uris like Chrome does? That's main
> use case here: pasting an image into a WYSIWYG editor and being able to save
> the image.

What is this API that Chrome has to convert to data URIs and that WebKit does not have?

[Chris Dumez]

(In reply to comment #23)
> If the plan is to use Blob URLs instead of data-uris (like Firefox), will
> there be a way to convert them to data uris like Chrome does? That's main
> use case here: pasting an image into a WYSIWYG editor and being able to save
> the image.

I have confirmed that Firefox is indeed using Data URLs rather than Blob URLs when pasting images from Preview.app.

[Chris Dumez]

(In reply to comment #25)
> (In reply to comment #23)
> > If the plan is to use Blob URLs instead of data-uris (like Firefox), will
> > there be a way to convert them to data uris like Chrome does? That's main
> > use case here: pasting an image into a WYSIWYG editor and being able to save
> > the image.
> 
> What is this API that Chrome has to convert to data URIs and that WebKit
> does not have?

It seems like WebKit would easily allow you to convert a Blob into a Data URL using FileReader.readAsDataURL().

[Chris Dumez]

Created attachment 294178 [details]
WIP Patch

[Chris Dumez]

Created attachment 294180 [details]
Patch

[Chris Dumez]

Created attachment 294181 [details]
Patch

[Darin Adler]

Comment on attachment 294181 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=294181&action=review

> Source/WebCore/editing/mac/EditorMac.mm:618
> +    Vector<uint8_t> data;
> +    data.append(buffer->data(), buffer->size());
> +    auto blob = Blob::create(WTFMove(data), type);

Can we write it this way?

    auto blob = Blob::create(Vector<uint8_t> { buffer->data(), buffer->size() }, type);

[Chris Dumez]

(In reply to comment #31)
> Comment on attachment 294181 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=294181&action=review
> 
> > Source/WebCore/editing/mac/EditorMac.mm:618
> > +    Vector<uint8_t> data;
> > +    data.append(buffer->data(), buffer->size());
> > +    auto blob = Blob::create(WTFMove(data), type);
> 
> Can we write it this way?
> 
>     auto blob = Blob::create(Vector<uint8_t> { buffer->data(),
> buffer->size() }, type);

Vector has no such constructor AFAICT.

[WebKit Commit Bot]

Comment on attachment 294181 [details]
Patch

Clearing flags on attachment: 294181

Committed r208451: <http://trac.webkit.org/changeset/208451>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Ebrahim Byagowi]

Similar still available issue on copy paste on other places on the web: Bug 170449