Bug 276431 – [CSS] Tokenizer should distinguish newline from whitespace

• Component: WebCore Misc.
• Status: RESOLVED FIXED

[Matthieu Dubet]

https://drafts.csswg.org/css-syntax/#tokenizer-definitions

newline
    U+000A LINE FEED. Note that U+000D CARRIAGE RETURN and U+000C FORM FEED are not included in this definition, as they are converted to U+000A LINE FEED during preprocessing. 
whitespace
    A newline, U+0009 CHARACTER TABULATION, or U+0020 SPACE.


But currently we just consider newline as a whitespace character.

CSSParserToken CSSTokenizer::whiteSpace(UChar /*cc*/)
{
    auto startOffset = m_input.offset();
    m_input.advanceUntilNonWhitespace();
    // FIXME: This does not preserve whitespace type (like tabs).
    auto whitespaceCount = 1 + (m_input.offset() - startOffset);
    return CSSParserToken(whitespaceCount);
}
template<typename CharacterType> constexpr bool isASCIIWhitespace(CharacterType character)
{
    return character == ' ' || character == '\n' || character == '\t' || character == '\r' || character == '\f';
}

[Radar WebKit Bug Importer]

<rdar://problem/131474098>

[zak ridouh]

Pull request: https://github.com/WebKit/WebKit/pull/31139

[EWS]

Committed 282084@main (de28c5277420): <https://commits.webkit.org/282084@main>

Reviewed commits have been landed. Closing PR #31139 and removing active labels.