Bug 108778 – Add support for mathvariants that cannot be emulated via CSS

• Component: MathML
• Status: RESOLVED FIXED

[Martin Robinson]

Some mathvariant choices cannot be emulated via CSS, so they must be implemented via character replacement. These include double-struck, Fraktur, bold Fraktur, script,  and bold script.

[Martin Robinson]

Created attachment 186277 [details]
Patch

[Martin Robinson]

A few notes to reviewers about this patch: First, this is all quite new to me, so I wouldn't be surprised if the patch is wildly wrong. Apologies in advance and thanks for any guidance you can give me!

The use of DOM replacement here isn't ideal, since it's a one way transformation. If JavaScript changes the mathvariant attribute, then the change won't be reflected properly. It seems that the way to handle this case properly is to create a RenderMathMLText and have it reflect the transformed text.

To be completely honest, I took this approach initially as it's a similar situation to CSS text transformations. The issue is that in the case of MathML mathvariant, the transformation is from characters in the BMP to out of the BMP. In practical terms this means that the UTF-16 encoded text value of the RenderText will be a different length than the DOM string (non-BMP characters can take two UTF-16 code units). This causes havoc with the Range and TextIterator classes, leading, for instance, to corrupted paste data.

The issues with Range and TextIterator are probably solvable issues (and probably should be solved), but as I was looking at it, I noticed a startling large amount of yak hair accumulating on the floor. I went instead with this simpler approach that was less correct. 

The other thing to notice about this patch is that I did not do mappings for transformations that are now handled via CSS. I hope to do that in a future patch. I see mapping even CSS-emulatable mathvariants as a good thing, because it means that you can copy and paste while maintaining some semantic meaning. The MathML spec stresses that mathvariant carries semantics.

Finally, judging by https://bugzilla.mozilla.org/show_bug.cgi?id=449396 it seems that the mathvariant  attribute shouldn't be inherited. This is reflected in my patch. It doesn't look like we support mstyle yet.

[Build Bot]

Comment on attachment 186277 [details]
Patch

Attachment 186277 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://queues.webkit.org/results/16355890

[Martin Robinson]

Created attachment 186282 [details]
Try to fix the Mac build

[Martin Robinson]

Comment on attachment 186282 [details]
Try to fix the Mac build

Clearing the review flag. Daniel and I talked about this patch and my approach is indeed problematic.

[Frédéric Wang (:fredw)]

To follow-up yesterday discussion and for the record:

- I have the impression that the attribute is still used a lot. I asked to the LaTeXML folks to directly use the Unicode code point but MathJax still relies on mathvariant. So that's why this is blocking the MathJax tracking bug :)

- As Martin said above, mathvariant is available on <mstyle> too. Although I think most of mstyle should be removed (http://lists.w3.org/Archives/Public/www-math/2012Sep/0002.html) I believe mstyle@mathvariant is actually one of the attribute used in real content.

- Like Neil and David said, some characters are invariant to italic change for example U+210E (PLANCK CONSTANT) and the italic style should not be applied to them. Similarly, style="font-weight: bold;" should in theory not apply to e.g. fraktur letters.

- MathJax and Gecko "implements" this via CSS using different fonts. So instead of remapping the Unicode code point, they change the style (italic, bold, font-face). That could be a temporary solution. See http://mxr.mozilla.org/mozilla-central/source/layout/mathml/mathml.css#124

As Dave said, I plan to implement that in Gecko using a private -moz-mathvariant CSS property. Basically, the mathvariant attribute on mstyle and token elements will be mapped to this property and then the text rendering code will use that property to choose the right transform to perform. So I agree with you that this is related to the italic mi issue and could be handled the same way.

[Frédéric Wang (:fredw)]

(In reply to comment #6)
> As Dave said, I plan to implement that in Gecko using a private -moz-mathvariant CSS property. Basically, the mathvariant attribute on mstyle and token elements will be mapped to this property and then the text rendering code will use that property to choose the right transform to perform. So I agree with you that this is related to the italic mi issue and could be handled the same way.

FYI, James Kitchener has taken over my work and is currently implementing that via -moz-mathvariant CSS property and text transformations:
https://bugzilla.mozilla.org/show_bug.cgi?id=114365

[Frédéric Wang (:fredw)]

I was about to try to work on this bug. My plan was:

- add a CSS property mathvariant that is inherited and has initial value "auto".

- rules in mathml.css should be replaced by direct mapping in Source/WebCore/mathml/ for math/mstyle and token elements.

- when mathvariant is "auto", RenderMathMLToken::updateStyle should switch to "italic" for single char <mi> (at the moment it updates the font-style, which is not quite correct and does not work well with inherited mathvariant values)

- add a TextMathTransformer class that would be called in RenderText to do a similar job as applyTextTransform. The RenderStyle will be used to decide the mathvariant.

Now I noticed Martin's comment about the issue with change of string length, so that will need more work in Range and TextIterator...

BTW, Gecko's implementation has more transforms and mathvariant values, see https://hg.mozilla.org/mozilla-central/file/923f1411f42f/layout/generic/MathMLTextRunFactory.cpp. The case of italic/bold/bold-italic is not handled yet (https://bugzilla.mozilla.org/show_bug.cgi?id=930504).

I think this bug becomes important to migrate to MATH fonts, since they don't have separate font files for italic/bold style but only a single font with the mathvariants in the Plane 1.

[Frédéric Wang (:fredw)]

Created attachment 235110 [details]
font-style=italic in WebKit VS italic math alphanum in Gecko (Latin Modern Math)

[Frédéric Wang (:fredw)]

Thinking again about how to fix this bug, I believe we could just add a special case in the RenderMathMLToken class to paint mathvariant-transformed characters with a non-standard code path (like what we do for RenderMathMLOperator to draw stretchy operators). For example, it seems we could rely on the TextPainter class for that purpose. Of course, just like RenderMathMLOperator this could cause other issues like (e.g. for selection) but will allow to get correct rendering.

In my opinion, the priority is to support the "automatic mathvariant italic" for single-char <mi> because the current text-style=italic CSS gives poor rendering (cf attachment 235110 [details]).

[Frédéric Wang (:fredw)]

(In reply to comment #10)
> In my opinion, the priority is to support the "automatic mathvariant italic"
> for single-char <mi> because the current text-style=italic CSS gives poor
> rendering (cf attachment 235110 [details]).

This one is now done in Alex's MathMLLayout branch.

[Frédéric Wang (:fredw)]

Created attachment 274292 [details]
Patch

[Frédéric Wang (:fredw)]

Created attachment 277380 [details]
Patch

[Frédéric Wang (:fredw)]

Created attachment 279117 [details]
Patch

Updating test expectation for iOS.

[Frédéric Wang (:fredw)]

Created attachment 282054 [details]
Patch

[Frédéric Wang (:fredw)]

Created attachment 283115 [details]
Patch

[Build Bot]

Comment on attachment 283115 [details]
Patch

Attachment 283115 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/1645514

New failing tests:
imported/mathml-in-html5/mathml/relations/css-styling/mathvariant-transforms-1.html

[Build Bot]

Created attachment 283120 [details]
Archive of layout-test-results from ews100 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews100  Port: mac-yosemite  Platform: Mac OS X 10.10.5

[Build Bot]

Comment on attachment 283115 [details]
Patch

Attachment 283115 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/1645517

New failing tests:
imported/mathml-in-html5/mathml/relations/css-styling/mathvariant-transforms-1.html

[Build Bot]

Created attachment 283121 [details]
Archive of layout-test-results from ews105 for mac-yosemite-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews105  Port: mac-yosemite-wk2  Platform: Mac OS X 10.10.5

[Frédéric Wang (:fredw)]

Created attachment 283130 [details]
Patch

Fixing path to the woff font in the test...

[Frédéric Wang (:fredw)]

Created attachment 283132 [details]
Patch

Fix error in debug build

[Brent Fulgham]

Comment on attachment 283132 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=283132&action=review

I hd a few minor nits with the change, but overall looks good and does not seem to introduce any regressions. r=me.

> Source/WebCore/rendering/mathml/MathMLStyle.cpp:192
> +    // The mathvariant attribute on the <math>, <mstyle> or token elements override the default behavior.

// The mathvariant attribute .... OVERRIDES the default behavior.

<nitpick>

> Source/WebCore/rendering/mathml/MathMLStyle.h:46
> +    // The special value none means that no explicit mathvariant value has been specified.

// The special value 'None' ...

> Source/WebCore/rendering/mathml/MathMLStyle.h:47
> +    // Note that the values are important for the computation perform in the mathVariant function of RenderMathMLToken, do not change them!

// Note that the NUMERICAL values are important for the computation PERFORMED in the mathVariant function of RenderMathMLToken, do not change them!

> Source/WebCore/rendering/mathml/MathMLStyle.h:83
> +    MathVariant m_mathVariant = None;

These would be better as C++11 style:

bool m_displayStyle { false };
MathVariant m_mathVariant { None };

> Source/WebCore/rendering/mathml/RenderMathMLToken.cpp:63
> +// replacement represents the mapped mathvariant Unicode character.

I suggest that 'key' and 'replacement' be quoted to make it clear that you are referring to variables in the struct below (not just generic concepts of key and replacement.)

> Source/WebCore/rendering/mathml/RenderMathMLToken.cpp:68
> +static inline UChar32 ExtractKey(const MathVariantMapping* entry) { return entry->key; }

This crashes if 'entry' is null. If 'entry' is guaranteed to always have a value, could this be a const&?

> Source/WebCore/rendering/mathml/RenderMathMLToken.cpp:70
> +MathVariantMappingSearch(uint32_t key, const MathVariantMapping* table, size_t tableLength)

Weird newline after UChar32 here.

> Source/WebCore/rendering/mathml/RenderMathMLToken.cpp:72
> +    if (const MathVariantMapping* entry = tryBinarySearch<const MathVariantMapping, UChar32>(table, tableLength, key, ExtractKey))

This would read better as:

If (const auto* entry = ...

> Source/WebCore/rendering/mathml/RenderMathMLToken.cpp:289
> +static UChar32 mathVariant(UChar32& codePoint, MathMLStyle::MathVariant mathvariant)

It doesn't appear that codePoint is ever mutated in this function. I suggest that it be made a const&.

[Frédéric Wang (:fredw)]

(In reply to comment #23)
> > Source/WebCore/rendering/mathml/RenderMathMLToken.cpp:68
> > +static inline UChar32 ExtractKey(const MathVariantMapping* entry) { return entry->key; }
> 
> This crashes if 'entry' is null. If 'entry' is guaranteed to always have a
> value, could this be a const&?

As I see in binarySearchImpl from WTF/wtf/StdLibExtras.h, the API assumes that the parameter is a pointer. It is also always guaranteed to be non-null.

[Frédéric Wang (:fredw)]

Committed r203072: <http://trac.webkit.org/changeset/203072>