Bug 27267 – HTTP Accept header gives preference of XML over HTML

• Component: WebCore Misc.
• Status: RESOLVED FIXED

[Kris Jordan]

For developers wanting to use the HTTP protocol to implement RESTful content negotiation where resources can be represented as HTML or XML WebKit unhelpfully prefers XML over HTML. Marciej has acknowledged this error: 

"Most WebKit-based browsers (and Safari in particular) would probably do a better job rendering HTML than XHTML or generic XML, if only because the code paths are much better tested. So the Accept header is somewhat in error."

Here is a demo URL that will be opened in XML in WebKit and HTML in Firefox 3+: 
http://recessframework.org/demo/content-negotiation/tweet

The accept header appears to be defined on line 200 of WebCore/loader/FrameLoader.cpp

I recommend two alternatives:

Copy Firefox 3.5's: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
or
Simplify: text/html,*/*

For more background information please see:
http://newmediacampaigns.com/page/browser-rest-http-accept-headers
http://newmediacampaigns.com/page/webkit-team-admits-accept-header-error

Thanks!

[Mark Rowe (bdash)]

This could be considered to be a duplicate of bug 12296.

[Maciej Stachowiak]

bug 12296 made a specific proposal for how to change Accept, but I think following the lead of Firefox 3.5 would be better than the suggested header in that bug. So not quite a duplicate.

[Matt Bishop]

FWIW, here are some other default ACCEPT headers:

IE8: image/gif, image/jpeg, image/pjpeg, image/pjpeg, application/x-shockwave-flash, application/xaml+xml, application/vnd.ms-xpsdocument, application/x-ms-xbap, application/x-ms-application, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, application/x-silverlight, */*

Opera: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1

Chrome: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5

[Sven-S. Porst]

I had troubles with Safari on author URLs at the arXiv preprint server:

e.g. http://arxiv.org/a/bahns_d_1

While this works perfectly in Firefox (returning and displaying HTML content, presumably because it has text/html as its first accept header), it gives a poor experience in Safari because the server takes the application/xml Accept header as a hint that we'd really prefer prefer XML/Atom format content and then redirects to the Atom feed. That, in turn, causes Safari to not display the page at all on my machine but to  launch my separate RSS reader and open it there. I.e. it results in rather annoying behaviour.

Just in case you needed another example where the difference in Accept headers causes real-world differences in the results seen by the user.

[Alexey Proskuryakov]

*** Bug 41914 has been marked as a duplicate of this bug. ***

This bug has been around quite a while and has last been updated half a year ago.

Since I as a backend developer ran into the same problem lately and had to do some seriously ugly hacks, that change the accept header, reciefed from a webkit request to something more desent on my server...

Has there been any progress in this problem.

Are there any plans to implement a better accept-header in webkit in the near future?

Regards,
Yustin

[Alexey Proskuryakov]

I think that there is consensus about making the Accept string match Firefox again (the current one matches an older version). Someone just needs to do the work: <http://www.webkit.org/coding/contributing.html>.

[Kris Jordan]

I'll take a stab here shortly.

(In reply to comment #7)
> I think that there is consensus about making the Accept string match Firefox again (the current one matches an older version). Someone just needs to do the work: <http://www.webkit.org/coding/contributing.html>.

[Kris Jordan]

Created attachment 82505 [details]
Patch to fix the bug with solution described in bug thread.

The default accept header now mirror's FireFox'. The meaningful change is that 'text/html' is now preferred over 'application/xml'.

[James Leigh]

Thanks for taking this on Kris. The file LayoutTests/http/tests/misc/xhtml-expected.txt will need to be changed as well. This test calls xhtml.php, which echos the accept header back to the client. It needs to match the file above to pass the test.

[Alexey Proskuryakov]

The link I gave above (<http://www.webkit.org/coding/contributing.html>) mentions running layout tests, as well as some other necessary steps.

[Kris Jordan]

Created attachment 82508 [details]
Patch with Tests

The expected result of the xhtml test is now included in the patch.

So there is a patch now. good. :)

Any chance of getting this patch into an official release anytime soon?

The bug is still in the 'NEW' Status.

[Alexey Proskuryakov]

Please mark the patch for review by clicking Details link to the right of it. Otherwise, looks fine at a first glance.

[Kris Jordan]

Marked for review (I believe). Let me know if I did not do that correctly.

[Alexey Proskuryakov]

It should be r?, not r+.

[Kris Jordan]

That makes sense- sorry, first time through. Updated accordingly.

[Alexey Proskuryakov]

Comment on attachment 82508 [details]
Patch with Tests

I don't understand the change in XHTMLMP case. Why downgrade application/xml, but not application/xhtml+xml?

I suggest leaving XHTMLMP unchanged, for people who care about it (if any) to consider consequences of changing the Accept string.

[James Leigh]

My understanding is that XHTMLMP is not used in any products, so this is pure theoretical and I only post this as an FYI. According to http://www.passani.it/gap/#MIME_TYPE application/xml is not a recommended content-type for mobile. It also says application/xhtml+xml is a better choice than text/html. The current patch (id= 82508) is consistent with best practices as of April 2010.

[Kris Jordan]

(In reply to comment #18)
> (From update of attachment 82508 [details])
> I don't understand the change in XHTMLMP case. Why downgrade application/xml, but not application/xhtml+xml?

Two motivations:

1) The heart of this issue is making sure application/xml is downgraded below text/html. How text/html and application/xhtml+xml are arranged is, relatively, inconsequential. I did not want to change any more than was necessary to resolve the fundamental issue of this ticket.

2) (This is *much* more nit picking and *much* less important.) 'application/xhtml+xml;profile='http://www.wapforum.org/xhtml' is not equivalent to 'application/xhtml'. It implies the server should only serve application/xhtml+xml if it can do so with that specific WAP profile. Technically, by the HTTP spec, the XHTMLMP accept header prioritizes a generic 'application/xhtml' below 'application/xml' because it falls under the expansion of */*. In practice, servers often ignore the profile parameter.

> I suggest leaving XHTMLMP unchanged, for people who care about it (if any) to consider consequences of changing the Accept string.

I don't know who or what uses the XHTMLMP case, and I don't have a problem leaving it alone, but I do believe it is a win to get HTML > XML in any case. Let me know if you need me to resubmit sans the XHTMLMP change for this to continue making progress.

[Alexey Proskuryakov]

Comment on attachment 82508 [details]
Patch with Tests

OK.

[WebKit Commit Bot]

Comment on attachment 82508 [details]
Patch with Tests

Clearing flags on attachment: 82508

Committed r80776: <http://trac.webkit.org/changeset/80776>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Kris Jordan]

Thanks for reviewing and getting it in the commit queue Alexey! Great to see this ticket resolved.