Bug 176252

Summary: WebKit should claim that it can show responses for a broader range of JSON MIMETypes
Product: WebKit Reporter: Joseph Pecoraro <joepeck>
Component: WebKit APIAssignee: Joseph Pecoraro <joepeck>
Status: RESOLVED FIXED    
Severity: Normal CC: achristensen, ap, beidson, buildbot, commit-queue, ggaren, joepeck, rniwa, webkit-bug-importer, youennf
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
[PATCH] Proposed Fix
none
[PATCH] Proposed Fix
ap: review-, buildbot: commit-queue-
Archive of layout-test-results from ews105 for mac-elcapitan-wk2
none
Archive of layout-test-results from ews101 for mac-elcapitan
none
Archive of layout-test-results from ews124 for ios-simulator-wk2
none
Archive of layout-test-results from ews116 for mac-elcapitan
none
[PATCH] Proposed Fix
none
[PATCH] Proposed Fix
none
[PATCH] Proposed Fix none

Description Joseph Pecoraro 2017-09-01 16:01:27 PDT
WebKit claims to be able to support some JSON types ("application/json") but not other types ("application/vnd.api+json") even though it can and actually does handle it. Lets broaden that to the full list of resources WebKit decides to handle as text.

Steps to Reproduce:
1. Visit a URL that loads JSON with MIMEType "application/json"
  => WebKit shows the content as text
2. Visit a URL that loads JSON with MIMEType "application/vnd.api+json"
  => WebKit ignores the content by default, Safari decides to download the response

Notes:
WebKit knows how to display responses matching a whitelist of different MIME types.

This whitelist is mostly summarized by the following: (WebKit extends this list to PDFs and plugins)

>     bool MIMETypeRegistry::canShowMIMEType(const String& mimeType)
>     {
>         if (isSupportedImageMIMEType(mimeType) || isSupportedNonImageMIMEType(mimeType) || isSupportedMediaMIMEType(mimeType))
>             return true;
>     
>         if (mimeType.startsWith("text/", false))
>             return !isUnsupportedTextMIMEType(mimeType);
>     
>         return false;
>     }
 
Included in this whitelist is "application/json" but not other MIME types that we recognize as JSON (namely isSupportedJSONMIMEType())

When WebKit lets clients decide what to do with a response it includes an indication of whether or not WebKit claims to be able to handle the response or not:

>    /*! Contains information about a navigation response, used for making policy decisions. */
>    @interface WKNavigationResponse : NSObject
>    
>    /*! @abstract A Boolean value indicating whether WebKit can display the response's MIME type natively.
>     @discussion Allowing a navigation response with a MIME type that can't be shown will cause the navigation to fail. */
>    @property (nonatomic, readonly) BOOL canShowMIMEType;
>    
>    @end

Since this is based off of the strict whitelist above, canShowMIMEType may be NO even if WebKit can ultimately recognize and show the response as text. There is a list non-"text/" prefixed MIME Types in isSupportedJavaScriptMIMEType and isSupportedJSONMIMEType that WebKit will be able to show as text, but it claims it does not know how to show. So WebKit clients like Safari allow "application/json" but not some other JSON types.
Comment 1 Joseph Pecoraro 2017-09-01 16:01:37 PDT
<rdar://problem/34212885>
Comment 2 Joseph Pecoraro 2017-09-01 16:08:57 PDT
Modifying the whitelist of MIME Types WebKit claims to be able to show will affect WebKit clients.
- Default behavior would have been to Ignore the load and would now be Use.
- WebKit client's will receive a difference value for the canShowMimeType indication (WKNavigationResponse)
- Safari behavior would likely have been to Download a `!canShowMimeType` indicated response and may now change to Use.

In this case, since WebKit already claimed to support "application/json" I think it is safe to extend this to a broader list of JSON mime types. 

Does anyone actually like that Safari / WebKit sometimes ignores / downloads a JSON response instead of showing it immediately? We've had requests, specifically for "application/vnd.api+json" to show it immediately as text instead of downloading it.
Comment 3 Joseph Pecoraro 2017-09-01 16:09:14 PDT
Created attachment 319661 [details]
[PATCH] Proposed Fix
Comment 4 Joseph Pecoraro 2017-09-01 16:43:30 PDT
Created attachment 319669 [details]
[PATCH] Proposed Fix
Comment 5 Build Bot 2017-09-01 17:33:52 PDT
Comment on attachment 319669 [details]
[PATCH] Proposed Fix

Attachment 319669 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/4429962

New failing tests:
imported/w3c/web-platform-tests/resource-timing/rt-initiatorType-element.html
Comment 6 Build Bot 2017-09-01 17:33:53 PDT
Created attachment 319678 [details]
Archive of layout-test-results from ews105 for mac-elcapitan-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews105  Port: mac-elcapitan-wk2  Platform: Mac OS X 10.11.6
Comment 7 Build Bot 2017-09-01 17:34:11 PDT
Comment on attachment 319669 [details]
[PATCH] Proposed Fix

Attachment 319669 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/4429965

New failing tests:
imported/w3c/web-platform-tests/html/webappapis/scripting/events/compile-event-handler-settings-objects.html
imported/w3c/web-platform-tests/html/semantics/forms/historical.html
imported/w3c/web-platform-tests/html/browsers/browsing-the-web/unloading-documents/unload/003.html
imported/w3c/web-platform-tests/resource-timing/rt-initiatorType-element.html
imported/w3c/web-platform-tests/html/browsers/history/the-location-interface/location-pathname-setter-question-mark.html
imported/w3c/web-platform-tests/html/browsers/the-window-object/garbage-collection-and-browsing-contexts/discard_iframe_history_1.html
http/tests/inspector/network/resource-mime-type.html
Comment 8 Build Bot 2017-09-01 17:34:13 PDT
Created attachment 319679 [details]
Archive of layout-test-results from ews101 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews101  Port: mac-elcapitan  Platform: Mac OS X 10.11.6
Comment 9 Build Bot 2017-09-01 17:59:30 PDT
Comment on attachment 319669 [details]
[PATCH] Proposed Fix

Attachment 319669 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/4429989

New failing tests:
imported/w3c/web-platform-tests/resource-timing/rt-initiatorType-element.html
Comment 10 Build Bot 2017-09-01 17:59:31 PDT
Created attachment 319681 [details]
Archive of layout-test-results from ews124 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews124  Port: ios-simulator-wk2  Platform: Mac OS X 10.12.5
Comment 11 Build Bot 2017-09-01 18:05:45 PDT
Comment on attachment 319669 [details]
[PATCH] Proposed Fix

Attachment 319669 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/4430042

New failing tests:
imported/w3c/web-platform-tests/html/webappapis/scripting/events/compile-event-handler-settings-objects.html
imported/w3c/web-platform-tests/html/semantics/forms/historical.html
imported/w3c/web-platform-tests/html/browsers/browsing-the-web/unloading-documents/unload/003.html
imported/w3c/web-platform-tests/resource-timing/rt-initiatorType-element.html
imported/w3c/web-platform-tests/html/browsers/history/the-location-interface/location-pathname-setter-question-mark.html
imported/w3c/web-platform-tests/html/browsers/the-window-object/garbage-collection-and-browsing-contexts/discard_iframe_history_1.html
http/tests/inspector/network/resource-mime-type.html
Comment 12 Build Bot 2017-09-01 18:05:47 PDT
Created attachment 319683 [details]
Archive of layout-test-results from ews116 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews116  Port: mac-elcapitan  Platform: Mac OS X 10.11.6
Comment 13 Alexey Proskuryakov 2017-09-06 12:31:15 PDT
Comment on attachment 319669 [details]
[PATCH] Proposed Fix

Marking r- for the red EWS.
Comment 14 Joseph Pecoraro 2017-09-06 13:38:22 PDT
Created attachment 320057 [details]
[PATCH] Proposed Fix
Comment 15 Joseph Pecoraro 2017-09-06 13:40:00 PDT
Created attachment 320058 [details]
[PATCH] Proposed Fix
Comment 16 Joseph Pecoraro 2017-09-07 15:27:43 PDT
Comment on attachment 320058 [details]
[PATCH] Proposed Fix

View in context: https://bugs.webkit.org/attachment.cgi?id=320058&action=review

> Tools/ChangeLog:3
> +        WebKit should claim that it can show responses a broader range of JSON MIMETypes

Typo: "responses *for* a broader range"
Comment 17 Alexey Proskuryakov 2017-09-07 15:57:55 PDT
Comment on attachment 320058 [details]
[PATCH] Proposed Fix

View in context: https://bugs.webkit.org/attachment.cgi?id=320058&action=review

> Tools/ChangeLog:13
> +        * TestWebKitAPI/Tests/WebCore/MIMETypeRegistry.cpp: Added.

Discussing with Joseph on IRC, this change impacts behavior that's observable via API (SPI?). I think that it should be tested using API, and not by exposing an internal function.
Comment 18 Joseph Pecoraro 2017-09-07 19:36:17 PDT
Created attachment 320227 [details]
[PATCH] Proposed Fix

Now with API tests for the change in -[WKNavigationResponse canShowMIMEType].
Comment 19 WebKit Commit Bot 2017-09-07 23:21:33 PDT
Comment on attachment 320227 [details]
[PATCH] Proposed Fix

Clearing flags on attachment: 320227

Committed r221778: <http://trac.webkit.org/changeset/221778>
Comment 20 WebKit Commit Bot 2017-09-07 23:21:35 PDT
All reviewed patches have been landed.  Closing bug.