Bug 41448 - HTMLTokenizer should ASSERT that it never emits a null character
Summary: HTMLTokenizer should ASSERT that it never emits a null character
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: 528+ (Nightly build)
Hardware: Other OS X 10.5
: P2 Normal
Assignee: Eric Seidel (no email)
Depends on:
Blocks: 41123
  Show dependency treegraph
Reported: 2010-07-01 01:11 PDT by Eric Seidel (no email)
Modified: 2010-07-01 20:15 PDT (History)
2 users (show)

See Also:

Patch (3.26 KB, patch)
2010-07-01 01:14 PDT, Eric Seidel (no email)
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Eric Seidel (no email) 2010-07-01 01:11:55 PDT
HTMLTokenizer should ASSERT that it never emits a null character
Comment 1 Eric Seidel (no email) 2010-07-01 01:13:09 PDT
These ASSERTs would have prevented the problems seen in bug 41436 (which were fixed by bug 41439).
Comment 2 Eric Seidel (no email) 2010-07-01 01:14:59 PDT
Created attachment 60214 [details]
Comment 3 Alexey Proskuryakov 2010-07-01 12:10:51 PDT
I still think it's a bad bad idea in HTML5 to replace nulls with FFFD's.
Comment 4 Eric Seidel (no email) 2010-07-01 13:33:59 PDT
The ASSERTs here are mostly to guard against us ever emitting \0 which we've repurposed to mean EndOfFile.  You should definitely take your complaints up with Hixie.  If we do allow emitting \0 at some point then we'll need to use a different character to mean EOF.

Also, one side-effect of allowing \0 in the rendered content is that diff treats files with \0 in them as binary (which bit us in bug 41436).
Comment 5 Adam Barth 2010-07-01 19:58:18 PDT
Comment on attachment 60214 [details]

@ap: I don't mean to ignore your comments.  I certainly find them valuable, but that's an spec-level issue, not something that should block us ASSERTing that the implementation is doing what we intend it to do at the moment.
Comment 6 WebKit Commit Bot 2010-07-01 20:15:41 PDT
Comment on attachment 60214 [details]

Clearing flags on attachment: 60214

Committed r62352: <http://trac.webkit.org/changeset/62352>
Comment 7 WebKit Commit Bot 2010-07-01 20:15:46 PDT
All reviewed patches have been landed.  Closing bug.