Bug 165601 – [GTK] WebKitWebProcess at 100% CPU loading hyphenation dictionaries

• Status: RESOLVED
• Resolution: FIXED
• Component: WebKitGTK

[Milan Crha]

Created attachment 296514 [details]
em00.html

Moving this from a downstream bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1402586

Opening the attached em00.html in a MiniBrowser causes WebKitWebProcess use 100% CPU for several seconds. The file is quite small, thus it should not. The downstream bug report contains more details and some backtraces as well. Just for easier searching I'm pasting here a part of one. A side effect of this is that the WebKitWebProcess has blocked the main thread, thus any requests to web extensions are simply blocked until the work is done.

Thread 1 (Thread 0x7f85f6233fc0 (LWP 5320)):
#0  0x00007f85e7e53eb1 in _int_malloc () at /lib64/libc.so.6
#1  0x00007f85e7e55f74 in malloc () at /lib64/libc.so.6
#2  0x00007f85eb79106c in hnj_malloc (size=5) at hnjalloc.c:46
        p = <optimized out>
#3  0x00007f85eb79110c in hnj_strdup (s=s@entry=0x7ffd4f3942a0 "ekin") at hyphen.c:71
        newstr = <optimized out>
        l = <optimized out>
#4  0x00007f85eb7911cf in hnj_hash_insert (hashtab=hashtab@entry=0x5634f05ac110, key=key@entry=0x7ffd4f3942a0 "ekin", val=2022) at hyphen.c:160
        e = 0x5634f0428f60
#5  0x00007f85eb7912c9 in hnj_get_state (hashtab=hashtab@entry=0x5634f05ac110, string=string@entry=0x7ffd4f3942a0 "ekin", dict=<optimized out>, dict=<optimized out>) at hyphen.c:189
#6  0x00007f85eb7919bf in hnj_get_state (dict=0x5634f01fc060, dict=0x5634f01fc060, string=0x7ffd4f3942a0 "ekin", hashtab=0x5634f05ac110) at hyphen.c:288
        state_num = -1
        i = <optimized out>
        j = <optimized out>
        word = "ekin", '\000' <repeats 12 times>, "\340c)\360\064V\000\000\300\323\030\350\205\177\000\000\n\000\000\000\000\000\000\000\006\332\344\347\205\177\000\000\340c)\360\064V\000\000\351{\354\347\205\177\000\000\000\000\000\000\000\000\000\000@\310\344\347\205\177\000\000\340c)\360\064V\000\000\300\323\030\350\205\177\000\000\n\000\000"
        pattern = "00040\000\000\000\000\000\000\347\205\177", '\000' <repeats 18 times>, "\004D9O\375\177\000\000$\001\344\347\205\177\000\000\000\000\000\000\000\000\000\000/\266(\360\064V\000\000\334;y\353\001", '\000' <repeats 11 times>, "(\000\000\000\000\000\000\000\340c)\360\064V\000\000\000\000\000"
        repl = <optimized out>
        replindex = 0 '\000'
        replcut = 0 '\000'
        state_num = 0
        last_state = <optimized out>
        ch = <optimized out>
        found = -1
#7  0x00007f85eb7919bf in hnj_hyphen_load_line (buf=<optimized out>, buf@entry=0x7ffd4f394400 "eki4n\n", dict=0x5634f01fc060, hashtab=hashtab@entry=0x5634f05ac110) at hyphen.c:352
        i = <optimized out>
        j = <optimized out>
        word = "ekin", '\000' <repeats 12 times>, "\340c)\360\064V\000\000\300\323\030\350\205\177\000\000\n\000\000\000\000\000\000\000\006\332\344\347\205\177\000\000\340c)\360\064V\000\000\351{\354\347\205\177\000\000\000\000\000\000\000\000\000\000@\310\344\347\205\177\000\000\340c)\360\064V\000\000\300\323\030\350\205\177\000\000\n\000\000"
        pattern = "00040\000\000\000\000\000\000\347\205\177", '\000' <repeats 18 times>, "\004D9O\375\177\000\000$\001\344\347\205\177\000\000\000\000\000\000\000\000\000\000/\266(\360\064V\000\000\334;y\353\001", '\000' <repeats 11 times>, "(\000\000\000\000\000\000\000\340c)\360\064V\000\000\000\000\000"
        repl = <optimized out>
        replindex = 0 '\000'
        replcut = 0 '\000'
        state_num = 0
        last_state = <optimized out>
        ch = <optimized out>
        found = -1
#8  0x00007f85eb791ec1 in hnj_hyphen_load_file (f=f@entry=0x5634f02963e0) at hyphen.c:444
        dict = {0x5634f01fc060, 0x7f85f60fd640 <WTF::TinyLRUCachePolicy<WTF::AtomicString, WTF::RefPtr<WebCore::HyphenationDictionary> >::cache()::cache>}
        hashtab = 0x5634f05ac110
        buf = "eki4n\n\000\000\n\000\000\000\000\000 3\n\000\002\366\205\177\000\000\373\321\344\347\205\177\000\000\020\314\344\347\205\177", '\000' <repeats 26 times>, "\001\000\000\000\000\000\000\000\340c)\360\064V\000\000P\214\252ޅ\177\000\000\006<y\353\205\177\000\000\001\000\000"
        nextlevel = 0
        i = <optimized out>
        k = 0
        e = <optimized out>
        state_num = 0
#9  0x00007f85eb791f42 in hnj_hyphen_load (fn=<optimized out>) at hyphen.c:383
        result = <optimized out>
        f = 0x5634f02963e0
#10 0x00007f85f500b6a6 in WebCore::HyphenationDictionary::HyphenationDictionary(WTF::CString const&) (dictPath=..., this=0x7f85dea827e0) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/platform/text/hyphen/HyphenationLibHyphen.cpp:157
        valueForNull = {m_storage = {__data = "\000\000\000\000\000\000\000", __align = {<No data fields>}}}
        dictionary = <optimized out>
        replacements = 0x0
        positions = 0x7f85deaa8770
        removedCharacterCounts = 0x7f85deaa8c40
        dictionaryPath = @0x7f85deaed1d0: {m_impl = {static isRefPtr = <optimized out>, m_ptr = 0x7f85deaa8770}}
        __for_begin = 0x7f85deaed1d0
        utf8StringCopy = {m_buffer = {static isRefPtr = <optimized out>, m_ptr = 0x7f85dea82760}}
        leadingSpaceBytes = <optimized out>
        leadingSpaceCharacters = 0
        hyphenArray = {<WTF::VectorBuffer<char, 0ul>> = {<WTF::VectorBufferBase<char>> = {m_buffer = 0x7f85deb046d0 "020412000", m_capacity = <optimized out>, m_size = <optimized out>}, <No data fields>}, <No data fields>}
        lowercaseLocaleIdentifier = <optimized out>
#11 0x00007f85f500b6a6 in WebCore::HyphenationDictionary::create(WTF::CString const&) (dictPath=...) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/platform/text/hyphen/HyphenationLibHyphen.cpp:147
        valueForNull = {m_storage = {__data = "\000\000\000\000\000\000\000", __align = {<No data fields>}}}
        dictionary = <optimized out>
        replacements = 0x0
        positions = 0x7f85deaa8770
        removedCharacterCounts = 0x7f85deaa8c40
        dictionaryPath = @0x7f85deaed1d0: {m_impl = {static isRefPtr = <optimized out>, m_ptr = 0x7f85deaa8770}}
        __for_begin = 0x7f85deaed1d0
        utf8StringCopy = {m_buffer = {static isRefPtr = <optimized out>, m_ptr = 0x7f85dea82760}}
        leadingSpaceBytes = <optimized out>
        leadingSpaceCharacters = 0
        hyphenArray = {<WTF::VectorBuffer<char, 0ul>> = {<WTF::VectorBufferBase<char>> = {m_buffer = 0x7f85deb046d0 "020412000", m_capacity = <optimized out>, m_size = <optimized out>}, <No data fields>}, <No data fields>}
        lowercaseLocaleIdentifier = <optimized out>
#12 0x00007f85f500b6a6 in WTF::TinyLRUCachePolicy<WTF::AtomicString, WTF::RefPtr<WebCore::HyphenationDictionary> >::createValueForKey(WTF::AtomicString const&) (dictionaryPath=...) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/platform/text/hyphen/HyphenationLibHyphen.cpp:195
        valueForNull = {m_storage = {__data = "\000\000\000\000\000\000\000", __align = {<No data fields>}}}
        dictionary = <optimized out>
        replacements = 0x0
        positions = 0x7f85deaa8770
        removedCharacterCounts = 0x7f85deaa8c40
        dictionaryPath = @0x7f85deaed1d0: {m_impl = {static isRefPtr = <optimized out>, m_ptr = 0x7f85deaa8770}}
        __for_begin = 0x7f85deaed1d0
        utf8StringCopy = {m_buffer = {static isRefPtr = <optimized out>, m_ptr = 0x7f85dea82760}}
        leadingSpaceBytes = <optimized out>
        leadingSpaceCharacters = 0
        hyphenArray = {<WTF::VectorBuffer<char, 0ul>> = {<WTF::VectorBufferBase<char>> = {m_buffer = 0x7f85deb046d0 "020412000", m_capacity = <optimized out>, m_size = <optimized out>}, <No data fields>}, <No data fields>}
        lowercaseLocaleIdentifier = <optimized out>
#13 0x00007f85f500b6a6 in WTF::TinyLRUCache<WTF::AtomicString, WTF::RefPtr<WebCore::HyphenationDictionary>, 4ul, WTF::TinyLRUCachePolicy<WTF::AtomicString, WTF::RefPtr<WebCore::HyphenationDictionary> > >::get(WTF::AtomicString const&) (key=..., this=<optimized out>) at /usr/src/debug/webkitgtk-2.14.2/Source/WTF/wtf/TinyLRUCache.h:69
        valueForNull = {m_storage = {__data = "\000\000\000\000\000\000\000", __align = {<No data fields>}}}
        dictionary = <optimized out>
        replacements = 0x0
        positions = 0x7f85deaa8770
        removedCharacterCounts = 0x7f85deaa8c40
        dictionaryPath = @0x7f85deaed1d0: {m_impl = {static isRefPtr = <optimized out>, m_ptr = 0x7f85deaa8770}}
        __for_begin = 0x7f85deaed1d0
        utf8StringCopy = {m_buffer = {static isRefPtr = <optimized out>, m_ptr = 0x7f85dea82760}}
        leadingSpaceBytes = <optimized out>
        leadingSpaceCharacters = 0
        hyphenArray = {<WTF::VectorBuffer<char, 0ul>> = {<WTF::VectorBufferBase<char>> = {m_buffer = 0x7f85deb046d0 "020412000", m_capacity = <optimized out>, m_size = <optimized out>}, <No data fields>}, <No data fields>}
        lowercaseLocaleIdentifier = <optimized out>
#14 0x00007f85f500b6a6 in WebCore::lastHyphenLocation(WTF::StringView, unsigned long, WTF::AtomicString const&) (string=..., beforeIndex=<optimized out>, localeIdentifier=...) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/platform/text/hyphen/HyphenationLibHyphen.cpp:244
        dictionary = <optimized out>
        replacements = 0x0
        positions = 0x7f85deaa8770
        removedCharacterCounts = 0x7f85deaa8c40
        dictionaryPath = @0x7f85deaed1d0: {m_impl = {static isRefPtr = <optimized out>, m_ptr = 0x7f85deaa8770}}
        __for_begin = 0x7f85deaed1d0
        utf8StringCopy = {m_buffer = {static isRefPtr = <optimized out>, m_ptr = 0x7f85dea82760}}
        leadingSpaceBytes = <optimized out>
        leadingSpaceCharacters = 0
        hyphenArray = {<WTF::VectorBuffer<char, 0ul>> = {<WTF::VectorBufferBase<char>> = {m_buffer = 0x7f85deb046d0 "020412000", m_capacity = <optimized out>, m_size = <optimized out>}, <No data fields>}, <No data fields>}
        lowercaseLocaleIdentifier = <optimized out>
#15 0x00007f85f4d0d27a in WebCore::maxWordFragmentWidth (glyphOverflow=..., fallbackFonts=..., suffixStart=<synthetic pointer>: <optimized out>, minimumSuffixLength=2, minimumPrefixLength=2, word=..., font=..., style=..., renderer=...) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/rendering/RenderText.cpp:744
        hyphenLocations = {<WTF::VectorBuffer<int, 8ul>> = {<WTF::VectorBufferBase<int>> = {m_buffer = 0x7ffd4f394920, m_capacity = 8, m_size = 1}, m_inlineBuffer = {{__data = "\005\000\000", __align = {<No data fields>}}, {__data = "\000\000\000", __align = {<No data fields>}}, {__data = "Q\b\317", <incomplete sequence \364>, __align = {<No data fields>}}, {__data = "\205\177\000", __align = {<No data fields>}}, {__data = "(C\251", <incomplete sequence \336>, __align = {<No data fields>}}, {__data = "\205\177\000", __align = {<No data fields>}}, {__data = "\001\000\000", __align = {<No data fields>}}, {__data = "\000\000\000", __align = {<No data fields>}}}}, <No data fields>}
        hyphenLocation = 5
        minimumFragmentWidthToConsider = <optimized out>
        maxFragmentWidth = <optimized out>
        suffixStart = 0
        maxFragmentWidth = <optimized out>
        currMinWidth = 0
        w = 72
        wordTrailingSpaceWidth = <optimized out>
        isSpace = <optimized out>
        c = 101
        previousCharacterIsSpace = <optimized out>
        wordLen = 9
        isNewline = <optimized out>
        hasBreak = false
        betweenWords = <optimized out>
        j = 15
        i = 6
        currMaxWidth = 47
        font = @0x7f85deb02918: {static s_codePath = WebCore::FontCascade::Auto, m_fontDescription = {<WebCore::FontDescription> = {m_featureSettings = {m_list = {<WTF::VectorBuffer<WebCore::FontFeature, 0ul>> = {<WTF::VectorBufferBase<WebCore::FontFeature>> = {m_buffer = 0x0, m_capacity = 0, m_size = 0}, <No data fields>}, <No data fields>}}, m_locale = {m_string = {m_impl = {static isRefPtr = <optimized out>, m_ptr = 0x7f85deb25b70}}}, m_computedSize = 16, m_orientation = 0, m_nonCJKGlyphOrientation = 0, m_widthVariant = 0, m_italic = 0, m_weight = 6, m_renderingMode = 0, m_textRendering = 0, m_script = 25, m_fontSynthesis = 3, m_variantCommonLigatures = 0, m_variantDiscretionaryLigatures = 0, m_variantHistoricalLigatures = 0, m_variantContextualAlternates = 0, m_variantPosition = 0, m_variantCaps = 0, m_variantNumericFigure = 0, m_variantNumericSpacing = 0, m_variantNumericFraction = 0, m_variantNumericOrdinal = 0, m_variantNumericSlashedZero = 0, m_variantAlternates = 0, m_variantEastAsianVariant = 0, m_variantEastAsianWidth = 0, m_variantEastAsianRuby = 0}, m_families = {m_data = 0x7f85deaa0fa8}, m_specifiedSize = 16, m_isAbsoluteSize = 1, m_kerning = 0, m_keywordSize = 0, m_fontSmoothing = 0, m_isSpecifiedFont = 1}, m_fonts = {static isRefPtr = <optimized out>, m_ptr = 0x7f85deaed798}, m_weakPtrFactory = {m_ref = {static isRef = <optimized out>, m_ptr = 0x7f85deb04470}}, m_letterSpacing = 0, m_wordSpacing = 0, m_useBackslashAsYenSymbol = false, m_enableKerning = 0, m_requiresShaping = 0}
        breakIterator = {static priorContextCapacity = 2, m_stringView = {m_characters = 0x7f85deb5599c, m_length = 15, m_is8Bit = true}, m_locale = {m_string = {m_impl = {static isRefPtr = <optimized out>, m_ptr = 0x7f85deb25b70}}}, m_iterator = 0x0, m_cachedPriorContext = 0x0, m_mode = WTF::LineBreakIteratorModeUAX14, m_cachedPriorContextLength = 0, m_priorContext = {0, 0}, m_isCJK = false}
        needsWordSpacing = false
        ignoringSpaces = false
        isSpace = <optimized out>
        firstWord = false
        firstLine = true
        nextBreakable = {m_isEngaged = true, m_value = {__data = "\017\000\000", __align = {<No data fields>}}}
        lastWordBoundary = 6
        wordTrailingSpace = {m_state = WebCore::WordTrailingSpace::WordTrailingSpaceState::Computed, m_width = {m_isEngaged = false, m_value = {__data = "\000\000\000", __align = {<No data fields>}}}, m_style = <optimized out>, m_textLayout = 0x0}
        maxWordWidth = 43
        minimumPrefixLength = 2
        minimumSuffixLength = 2
        firstGlyphLeftOverflow = {m_isEngaged = true, m_value = {__data = "\000\000\000", __align = {<No data fields>}}}
        breakNBSP = false
        breakAll = false
        keepAllWords = false
#16 0x00007f85f4d0d27a in WebCore::RenderText::computePreferredLogicalWidths(float, WTF::HashSet<WebCore::Font const*, WTF::PtrHash<WebCore::Font const*>, WTF::HashTraits<WebCore::Font const*> >&, WebCore::GlyphOverflow&) (this=this@entry=0x7f85dea95840, leadWidth=176, fallbackFonts=..., glyphOverflow=...) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/rendering/RenderText.cpp:913
        suffixStart = 0
        maxFragmentWidth = <optimized out>
        currMinWidth = 0
        w = 72
        wordTrailingSpaceWidth = <optimized out>
        isSpace = <optimized out>
        c = 101
        previousCharacterIsSpace = <optimized out>
        wordLen = 9
        isNewline = <optimized out>
        hasBreak = false
        betweenWords = <optimized out>
        j = 15
        i = 6
        currMaxWidth = 47
        font = @0x7f85deb02918: {static s_codePath = WebCore::FontCascade::Auto, m_fontDescription = {<WebCore::FontDescription> = {m_featureSettings = {m_list = {<WTF::VectorBuffer<WebCore::FontFeature, 0ul>> = {<WTF::VectorBufferBase<WebCore::FontFeature>> = {m_buffer = 0x0, m_capacity = 0, m_size = 0}, <No data fields>}, <No data fields>}}, m_locale = {m_string = {m_impl = {static isRefPtr = <optimized out>, m_ptr = 0x7f85deb25b70}}}, m_computedSize = 16, m_orientation = 0, m_nonCJKGlyphOrientation = 0, m_widthVariant = 0, m_italic = 0, m_weight = 6, m_renderingMode = 0, m_textRendering = 0, m_script = 25, m_fontSynthesis = 3, m_variantCommonLigatures = 0, m_variantDiscretionaryLigatures = 0, m_variantHistoricalLigatures = 0, m_variantContextualAlternates = 0, m_variantPosition = 0, m_variantCaps = 0, m_variantNumericFigure = 0, m_variantNumericSpacing = 0, m_variantNumericFraction = 0, m_variantNumericOrdinal = 0, m_variantNumericSlashedZero = 0, m_variantAlternates = 0, m_variantEastAsianVariant = 0, m_variantEastAsianWidth = 0, m_variantEastAsianRuby = 0}, m_families = {m_data = 0x7f85deaa0fa8}, m_specifiedSize = 16, m_isAbsoluteSize = 1, m_kerning = 0, m_keywordSize = 0, m_fontSmoothing = 0, m_isSpecifiedFont = 1}, m_fonts = {static isRefPtr = <optimized out>, m_ptr = 0x7f85deaed798}, m_weakPtrFactory = {m_ref = {static isRef = <optimized out>, m_ptr = 0x7f85deb04470}}, m_letterSpacing = 0, m_wordSpacing = 0, m_useBackslashAsYenSymbol = false, m_enableKerning = 0, m_requiresShaping = 0}
        breakIterator = {static priorContextCapacity = 2, m_stringView = {m_characters = 0x7f85deb5599c, m_length = 15, m_is8Bit = true}, m_locale = {m_string = {m_impl = {static isRefPtr = <optimized out>, m_ptr = 0x7f85deb25b70}}}, m_iterator = 0x0, m_cachedPriorContext = 0x0, m_mode = WTF::LineBreakIteratorModeUAX14, m_cachedPriorContextLength = 0, m_priorContext = {0, 0}, m_isCJK = false}
        needsWordSpacing = false
        ignoringSpaces = false
        isSpace = <optimized out>
        firstWord = false
        firstLine = true
        nextBreakable = {m_isEngaged = true, m_value = {__data = "\017\000\000", __align = {<No data fields>}}}
        lastWordBoundary = 6
        wordTrailingSpace = {m_state = WebCore::WordTrailingSpace::WordTrailingSpaceState::Computed, m_width = {m_isEngaged = false, m_value = {__data = "\000\000\000", __align = {<No data fields>}}}, m_style = <optimized out>, m_textLayout = 0x0}
        maxWordWidth = 43
        minimumPrefixLength = 2
        minimumSuffixLength = 2
        firstGlyphLeftOverflow = {m_isEngaged = true, m_value = {__data = "\000\000\000", __align = {<No data fields>}}}
        breakNBSP = false
        breakAll = false
        keepAllWords = false
#17 0x00007f85f4d112e2 in WebCore::RenderText::computePreferredLogicalWidths(float) (this=0x7f85dea95840, leadWidth=<optimized out>) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/rendering/RenderText.cpp:723
        fallbackFonts = {m_impl = {static m_maxLoad = <optimized out>, static m_minLoad = <optimized out>, m_table = 0x0, m_tableSize = 0, m_tableSizeMask = 0, m_keyCount = 0, m_deletedCount = 0}}
        glyphOverflow = {left = 0, right = 0, top = 0, bottom = 0, computeBounds = false}
#18 0x00007f85f4d0b1fc in WebCore::RenderText::trimmedPrefWidths(float, float&, bool&, float&, bool&, bool&, bool&, float&, float&, float&, float&, bool&) (this=this@entry=0x7f85dea95840, leadWidth=leadWidth@entry=176, beginMinW=@0x7ffd4f394c98: 18, beginWS=@0x7ffd4f394bce: false, endMinW=@0x7ffd4f394c9c: 29, endWS=@0x7ffd4f394bcf: false, hasBreakableChar=@0x7ffd4f394bcc: true, hasBreak=@0x7ffd4f394bcd: false, beginMaxW=@0x7ffd4f394ca0: 0, endMaxW=@0x7ffd4f394cd0: 0, minW=@0x7ffd4f394c90: 0, maxW=@0x7ffd4f394c94: 0, stripFrontSpaces=@0x7ffd4f394bcb: false) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/rendering/RenderText.cpp:606
#19 0x00007f85f4bb63a5 in WebCore::RenderBlockFlow::computeInlinePreferredLogicalWidths(WebCore::LayoutUnit&, WebCore::LayoutUnit&) const (this=this@entry=0x7f85deaccd68, minLogicalWidth=..., maxLogicalWidth=...) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/rendering/RenderBlockFlow.cpp:4334
        hasBreakableChar = true
        hasBreak = false
        beginMin = 18
        endMin = 29
        strippingBeginWS = false
        beginWS = false
        ti = <optimized out>
        endWS = false
        beginMax = 0
        endMax = 0
        childMin = 0
        childMax = 0
        child = 0x7f85dea95840
        autoWrap = true
        isAnonymousInlineBlock = false
        inlineMax = 176
        inlineMin = 29
        containingBlock = <optimized out>
        cw = <optimized out>
        stripFrontSpaces = false
        trailingSpaceChild = 0x0
        allowImagesToBreak = true
        oldAutoWrap = true
        childIterator = {parent = @0x7f85deaccd68, current = 0x7f85dea95840, endOfInline = false, initial = false}
        addedTextIndent = true
        hasRemainingNegativeTextIndent = false
        textIndent = {m_value = 0}
        prevFloat = <optimized out>
        isPrevChildInlineFlow = true
        shouldBreakLineAfterText = true
        canHangPunctuationAtStart = false
        canHangPunctuationAtEnd = false
        lastText = 0x7f85dea957e8
        addedStartPunctuationHang = false
#20 0x00007f85f4bb8302 in WebCore::RenderBlockFlow::computeIntrinsicLogicalWidths(WebCore::LayoutUnit&, WebCore::LayoutUnit&) const (this=0x7f85deaccd68, minLogicalWidth=..., maxLogicalWidth=...) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/rendering/RenderBlockFlow.cpp:358
        scrollbarWidth = <optimized out>
#21 0x00007f85f4b91cb4 in WebCore::RenderBlock::computePreferredLogicalWidths() (this=0x7f85deaccd68) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/rendering/RenderBlock.cpp:2773
        borderAndPadding = <optimized out>
#22 0x00007f85f4bcc331 in WebCore::RenderBox::minPreferredLogicalWidth() const (this=0x7f85deaccd68) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/rendering/RenderBox.cpp:1061
#23 0x00007f85f4b98fd4 in WebCore::RenderBlock::computeBlockPreferredLogicalWidths(WebCore::LayoutUnit&, WebCore::LayoutUnit&) const (this=0x7f85dea8f118, minLogicalWidth=..., maxLogicalWidth=...) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/rendering/RenderBlock.cpp:2841
        endMarginLength = {{m_intValue = 0, m_floatValue = 0, m_calculationValueHandle = 0}, m_hasQuirk = false, m_type = 3 '\003', m_isFloat = false}
        margin = <optimized out>
        marginEnd = {m_value = 0}
        w = {m_value = 0}
        startMarginLength = {{m_intValue = 0, m_floatValue = 0, m_calculationValueHandle = 0}, m_hasQuirk = false, m_type = 3 '\003', m_isFloat = false}
        marginStart = {m_value = 0}
        childMinPreferredLogicalWidth = {m_value = 0}
        childMaxPreferredLogicalWidth = {m_value = 0}
        nowrap = false
        child = 0x7f85deaccd68
        containingBlock = 0x7f85dea91320
        floatLeftWidth = {m_value = 0}
        floatRightWidth = {m_value = 0}
#24 0x00007f85f4bb83bd in WebCore::RenderBlockFlow::computeIntrinsicLogicalWidths(WebCore::LayoutUnit&, WebCore::LayoutUnit&) const (this=0x7f85dea8f118, minLogicalWidth=..., maxLogicalWidth=...) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/rendering/RenderBlockFlow.cpp:360
        scrollbarWidth = <optimized out>
#25 0x00007f85f4b91cb4 in WebCore::RenderBlock::computePreferredLogicalWidths() (this=this@entry=0x7f85dea8f118) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/rendering/RenderBlock.cpp:2773
        borderAndPadding = <optimized out>
#26 0x00007f85f4cf8cfc in WebCore::RenderTableCell::computePreferredLogicalWidths() (this=0x7f85dea8f118) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/rendering/RenderTableCell.cpp:185
        w = {{m_intValue = -558466400, m_floatValue = -6.57449899e+18, m_calculationValueHandle = 3736500896}, m_hasQuirk = 133, m_type = 127 '\177', m_isFloat = false}
#27 0x00007f85f4bcc331 in WebCore::RenderBox::minPreferredLogicalWidth() const (this=0x7f85dea8f118) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/rendering/RenderBox.cpp:1061
#28 0x00007f85f4b5b76c in WebCore::AutoTableLayout::recalcColumn(unsigned int) (this=this@entry=0x7f85deaf4d90, effCol=effCol@entry=0) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/rendering/AutoTableLayout.cpp:78
        cCellMaxWidth = <optimized out>
        cellLogicalWidth = {{m_intValue = 0, m_floatValue = 0, m_calculationValueHandle = 0}, m_hasQuirk = false, m_type = 0 '\000', m_isFloat = false}
        current = {cells = {<WTF::VectorBuffer<WebCore::RenderTableCell*, 1ul>> = {<WTF::VectorBufferBase<WebCore::RenderTableCell*>> = {m_buffer = 0x7ffd4f3950e0, m_capacity = 1, m_size = 1}, m_inlineBuffer = {{__data = "\030\361\250ޅ\177\000", __align = {<No data fields>}}}}, <No data fields>}, inColSpan = false}
        i = 0
        fixedContributor = 0x0
        maxContributor = 0x0
#29 0x00007f85f4b5c00d in WebCore::AutoTableLayout::fullRecalc() (this=this@entry=0x7f85deaf4d90) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/rendering/AutoTableLayout.cpp:182
        i = 0
        groupLogicalWidth = {{m_intValue = 0, m_floatValue = 0, m_calculationValueHandle = 0}, m_hasQuirk = false, m_type = 0 '\000', m_isFloat = false}
        currentColumn = <optimized out>
#30 0x00007f85f4b5c5e2 in WebCore::AutoTableLayout::computeIntrinsicLogicalWidths(WebCore::LayoutUnit&, WebCore::LayoutUnit&) (this=0x7f85deaf4d90, minWidth=..., maxWidth=...) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/rendering/AutoTableLayout.cpp:235
        spanMaxLogicalWidth = <optimized out>
        maxPercent = <optimized out>
        maxNonPercent = <optimized out>
        epsilon = <optimized out>
        remainingPercent = <optimized out>
#31 0x00007f85f4cebcfa in WebCore::RenderTable::computePreferredLogicalWidths() (this=0x7f85dea91320) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/rendering/RenderTable.cpp:811
        bordersPaddingAndSpacing = <optimized out>
#32 0x00007f85f4bcc331 in WebCore::RenderBox::minPreferredLogicalWidth() const (this=0x7f85dea91320) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/rendering/RenderBox.cpp:1061
#33 0x00007f85f4b98fd4 in WebCore::RenderBlock::computeBlockPreferredLogicalWidths(WebCore::LayoutUnit&, WebCore::LayoutUnit&) const (this=0x7f85dea8f000, minLogicalWidth=..., maxLogicalWidth=...) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/rendering/RenderBlock.cpp:2841
        endMarginLength = {{m_intValue = 0, m_floatValue = 0, m_calculationValueHandle = 0}, m_hasQuirk = false, m_type = 3 '\003', m_isFloat = false}
        margin = <optimized out>
        marginEnd = {m_value = 0}
        w = {m_value = 0}
        startMarginLength = {{m_intValue = 0, m_floatValue = 0, m_calculationValueHandle = 0}, m_hasQuirk = false, m_type = 3 '\003', m_isFloat = false}
        marginStart = {m_value = 0}
        childMinPreferredLogicalWidth = {m_value = 0}
        childMaxPreferredLogicalWidth = {m_value = 0}
        nowrap = false
        child = 0x7f85dea91320
        containingBlock = 0x7f85dea91190
        floatLeftWidth = {m_value = 0}
        floatRightWidth = {m_value = 0}
#34 0x00007f85f4bb83bd in WebCore::RenderBlockFlow::computeIntrinsicLogicalWidths(WebCore::LayoutUnit&, WebCore::LayoutUnit&) const (this=0x7f85dea8f000, minLogicalWidth=..., maxLogicalWidth=...) at /usr/src/debug/webkitgtk-2.14.2/Source/WebCore/rendering/RenderBlockFlow.cpp:360
        scrollbarWidth = <optimized out>
.
.
.
.
#124 0x00007f85f35ef2b9 in WTF::RunLoop::<lambda(gpointer)>::_FUN(gpointer) () at /usr/src/debug/webkitgtk-2.14.2/Source/WTF/wtf/glib/RunLoopGLib.cpp:68
#125 0x00007f85ecc8de42 in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
#126 0x00007f85ecc8e1c0 in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0
#127 0x00007f85ecc8e4e2 in g_main_loop_run () at /lib64/libglib-2.0.so.0
#128 0x00007f85f35efb70 in WTF::RunLoop::run() () at /usr/src/debug/webkitgtk-2.14.2/Source/WTF/wtf/glib/RunLoopGLib.cpp:94
        runLoop = @0x7f85debf7000: {<WTF::FunctionDispatcher> = {<WTF::ThreadSafeRefCounted<WTF::FunctionDispatcher>> = {<WTF::ThreadSafeRefCountedBase> = {m_refCount = {<std::__atomic_base<int>> = {static _S_alignment = 4, _M_i = 1}, <No data fields>}}, <No data fields>}, _vptr.FunctionDispatcher = 0x7f85f37aeba0 <vtable for WTF::RunLoop+16>}, m_functionQueueLock = {m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 512, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 17 times>, "\002", '\000' <repeats 21 times>, __align = 0}}, m_functionQueue = {m_start = 6, m_end = 7, m_buffer = {<WTF::VectorBufferBase<WTF::Function<void()> >> = {m_buffer = 0x7f85debcf0d8, m_capacity = 27, m_size = 0}, <No data fields>}}, m_mainContext = {m_ptr = 0x5634efe0e970}, m_mainLoops = {<WTF::VectorBuffer<WTF::GRefPtr<_GMainLoop>, 0ul>> = {<WTF::VectorBufferBase<WTF::GRefPtr<_GMainLoop> >> = {m_buffer = 0x7f85debfa180, m_capacity = 16, m_size = 1}, <No data fields>}, <No data fields>}, m_source = {m_ptr = 0x5634efe3e0d0}}
        nestedMainLoop = <optimized out>
#129 0x00007f85f41c28a9 in WebKit::ChildProcessMain<WebKit::WebProcess, WebKit::WebProcessMain>(int, char**) (argc=<optimized out>, argv=0x7ffd4f3981b8) at /usr/src/debug/webkitgtk-2.14.2/Source/WebKit2/Shared/unix/ChildProcessMain.h:61
        childMain = {<WebKit::ChildProcessMainBase> = {_vptr.ChildProcessMainBase = 0x7f85f5f4fbe0 <vtable for WebKit::WebProcessMain+16>, m_parameters = {uiProcessName = {m_impl = {static isRefPtr = <optimized out>, m_ptr = 0x0}}, clientIdentifier = {m_impl = {static isRefPtr = <optimized out>, m_ptr = 0x0}}, connectionIdentifier = 122, extraInitializationData = {m_impl = {static m_maxLoad = <optimized out>, static m_minLoad = <optimized out>, m_table = 0x0, m_tableSize = 0, m_tableSizeMask = 0, m_keyCount = 0, m_deletedCount = 0}}}}, <No data fields>}
#130 0x00007f85e7df0401 in __libc_start_main () at /lib64/libc.so.6
#131 0x00005634ef2ccc5a in _start ()

[Michael Catanzaro]

Note in particular that, based on the backtraces in the downstream report, it's clear that WebKit is creating multiple HyphenationDictionary objects in some unexpectedly-long loop.

[Michael Catanzaro]

This issue is reproducible on https://www.python.org/dev/peps/pep-0484/. WebKit spends 5-7 seconds at a time locked up in the hyphenation code. There is an assertion failure:

ASSERTION FAILED: xPos + prefixWidth <= availableWidth
../../Source/WebCore/rendering/line/BreakingContext.h(724) : void WebCore::tryHyphenating(WebCore::RenderText&, const WebCore::FontCascade&, const WTF::AtomicString&, unsigned int, int, int, int, unsigned int, unsigned int, float, int, bool, bool, int, WebCore::InlineIterator&, WTF::Optional<unsigned int>, bool&)
1   0x7f4cbe3b4c28 /home/mcatanzaro/src/jhbuild/install/lib/libjavascriptcoregtk-4.0.so.18(WTFCrash+0x1e) [0x7f4cbe3b4c28]
2   0x7f4cc5fb9fc5 /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore14tryHyphenatingERNS_10RenderTextERKNS_11FontCascadeERKN3WTF12AtomicStringEjiiijjfibbiRNS_14InlineIteratorENS5_8OptionalIjEERb+0x4a6) [0x7f4cc5fb9fc5]
3   0x7f4cc5fba5a3 /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZZN7WebCore15BreakingContext10handleTextERN3WTF6VectorINS_15WordMeasurementELm64ENS1_15CrashOnOverflowELm16EEERbRjENKUlRNS_14InlineIteratorEE0_clESA_+0x191) [0x7f4cc5fba5a3]
4   0x7f4cc5fc0a5f /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZNSt17_Function_handlerIFvRN7WebCore14InlineIteratorEEZNS0_15BreakingContext10handleTextERN3WTF6VectorINS0_15WordMeasurementELm64ENS5_15CrashOnOverflowELm16EEERbRjEUlS2_E0_E9_M_invokeERKSt9_Any_dataS2_+0x37) [0x7f4cc5fc0a5f]
5   0x7f4cc5fbed05 /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZNKSt8functionIFvRN7WebCore14InlineIteratorEEEclES2_+0x49) [0x7f4cc5fbed05]
6   0x7f4cc5fb7afc /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore15BreakingContext21InlineIteratorHistory4pushESt8functionIFvRNS_14InlineIteratorEEE+0xba) [0x7f4cc5fb7afc]
7   0x7f4cc5fbc188 /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore15BreakingContext10handleTextERN3WTF6VectorINS_15WordMeasurementELm64ENS1_15CrashOnOverflowELm16EEERbRj+0x19de) [0x7f4cc5fbc188]
8   0x7f4cc5fb5ce7 /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore11LineBreaker13nextLineBreakERNS_23BidiResolverWithIsolateINS_14InlineIteratorENS_7BidiRunENS_15BidiIsolatedRunEEERNS_8LineInfoERNS_15LineLayoutStateERNS_14RenderTextInfoEPNS_14FloatingObjectEjRN3WTF6VectorINS_15WordMeasurementELm64ENSF_15CrashOnOverflowELm16EEE+0x35b) [0x7f4cc5fb5ce7]
9   0x7f4cc5da050a /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore15RenderBlockFlow26layoutRunsAndFloatsInRangeERNS_15LineLayoutStateERNS_23BidiResolverWithIsolateINS_14InlineIteratorENS_7BidiRunENS_15BidiIsolatedRunEEERKS4_RKNS_10BidiStatusEj+0x358) [0x7f4cc5da050a]
10  0x7f4cc5da0109 /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore15RenderBlockFlow19layoutRunsAndFloatsERNS_15LineLayoutStateEb+0x44b) [0x7f4cc5da0109]
11  0x7f4cc5da2881 /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore15RenderBlockFlow15layoutLineBoxesEbRNS_10LayoutUnitES2_+0x6b1) [0x7f4cc5da2881]
12  0x7f4cc5d802c1 /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore15RenderBlockFlow20layoutInlineChildrenEbRNS_10LayoutUnitES2_+0xb7) [0x7f4cc5d802c1]
13  0x7f4cc5d7f648 /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore15RenderBlockFlow11layoutBlockEbNS_10LayoutUnitE+0x3b6) [0x7f4cc5d7f648]
14  0x7f4cc5d4d032 /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore11RenderBlock6layoutEv+0x7c) [0x7f4cc5d4d032]
15  0x7f4cc5d80682 /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore15RenderBlockFlow16layoutBlockChildERNS_9RenderBoxERNS0_10MarginInfoERNS_10LayoutUnitES6_+0x3be) [0x7f4cc5d80682]
16  0x7f4cc5d801db /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore15RenderBlockFlow19layoutBlockChildrenEbRNS_10LayoutUnitE+0x247) [0x7f4cc5d801db]
17  0x7f4cc5d7f66c /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore15RenderBlockFlow11layoutBlockEbNS_10LayoutUnitE+0x3da) [0x7f4cc5d7f66c]
18  0x7f4cc5d4d032 /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore11RenderBlock6layoutEv+0x7c) [0x7f4cc5d4d032]
19  0x7f4cc5d80682 /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore15RenderBlockFlow16layoutBlockChildERNS_9RenderBoxERNS0_10MarginInfoERNS_10LayoutUnitES6_+0x3be) [0x7f4cc5d80682]
20  0x7f4cc5d801db /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore15RenderBlockFlow19layoutBlockChildrenEbRNS_10LayoutUnitE+0x247) [0x7f4cc5d801db]
21  0x7f4cc5d7f66c /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore15RenderBlockFlow11layoutBlockEbNS_10LayoutUnitE+0x3da) [0x7f4cc5d7f66c]
22  0x7f4cc5d4d032 /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore11RenderBlock6layoutEv+0x7c) [0x7f4cc5d4d032]
23  0x7f4cc5d80682 /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore15RenderBlockFlow16layoutBlockChildERNS_9RenderBoxERNS0_10MarginInfoERNS_10LayoutUnitES6_+0x3be) [0x7f4cc5d80682]
24  0x7f4cc5d801db /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore15RenderBlockFlow19layoutBlockChildrenEbRNS_10LayoutUnitE+0x247) [0x7f4cc5d801db]
25  0x7f4cc5d7f66c /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore15RenderBlockFlow11layoutBlockEbNS_10LayoutUnitE+0x3da) [0x7f4cc5d7f66c]
26  0x7f4cc5d4d032 /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore11RenderBlock6layoutEv+0x7c) [0x7f4cc5d4d032]
27  0x7f4cc5d1a223 /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore13RenderElement14layoutIfNeededEv+0x35) [0x7f4cc5d1a223]
28  0x7f4cc5d87e23 /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore15RenderBlockFlow20insertFloatingObjectERNS_9RenderBoxE+0x219) [0x7f4cc5d87e23]
29  0x7f4cc5d8019e /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore15RenderBlockFlow19layoutBlockChildrenEbRNS_10LayoutUnitE+0x20a) [0x7f4cc5d8019e]
30  0x7f4cc5d7f66c /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore15RenderBlockFlow11layoutBlockEbNS_10LayoutUnitE+0x3da) [0x7f4cc5d7f66c]
31  0x7f4cc5d4d032 /home/mcatanzaro/src/jhbuild/install/lib/libwebkit2gtk-4.0.so.37(_ZN7WebCore11RenderBlock6layoutEv+0x7c) [0x7f4cc5d4d032]

[Michael Catanzaro]

Could it be a bug in the FreeType backend? I guess textWidth must be returning something wrong. Possibly from FontCascade::width?

[Christian Stadelmann]

See also: https://bugzilla.redhat.com/show_bug.cgi?id=1405791

[Zan Dobersek]

I'm using ToT on Debian stretch.


What locale are you using? Specifically, what locale is being passed (via `localeIdentifier`) to lastHyphenLocation?

In relation to that locale, how many dictionaries for that locale are stored at /usr/share/hyphen or /usr/local/share/hyphen?


In my case, I'm using the 'en' locale, meaning that related locales will be mapped via the availableLocales() HashMap to three dictionary files: hyph_en_GB.dic, hyph_en_AU.dic and hyph_en_ZA.dic, as provided by the hyphen-en-gb package.

We're storing these in a TinyLRUCache with a default capacity of 4, meaning that they will fit in just fine.

If I overwrite the locale to be 'es', the Spanish locale, then related locales will be mapped via the availableLocales() HashMap to 21 dictionary files. In lastHyphenLocation(), we iterate over these locales and retrieve a HyphenDictionary object from the TinyLRUCache for each one, until we find a locale that works for us. But because of using the TinyLRUCache with the default capacity of 4, the dictionary objects stored in the TinyLRUCache keep getting on evicted and being replaced with new ones.

With the 'en' locale, loading the Python PEP page, only three HyphenDictionary objects are created, with the underlying hyph_en_*.dic file opened and processed.

With the 'es' locale, loading the same page, 1723 HyphenDictionary objects are created, loading each hyph_es_*.dic file 82 times. It still doesn't spin the CPU usage of the WebProcess to 100% on my system, but it's obviously a problem.


While the TinyLRUCache capacity could be bumped, it should be noted that at least in Debian packages a lot of these locale variations for one specific locale under /usr/share/hyphen are simply symbolic links to that one master dict file. For instance, there's 21 different Spanish locales under /usr/share/hyphen, from Bolivian to Venezuelan, but it's 20 files just linking to the master hyphen_es_ES.dic file.

Same for the English locales -- hyph_en_AU.dic and hyph_en_ZA.dic link to hyph_en_GB.dic.

So we should maybe also look into detecting symlinks when storing these filepaths in the availableLocales() HashMap.

[Milan Crha]

I have in /usr/share/hyphen/ 23 hyphen_en_??.dic files, all but one sym-linking hyph_en_US.dic. Checking for symlinks sounds like a good idea to me.

[Christian Stadelmann]

(In reply to comment #5)
> What locale are you using? Specifically, what locale is being passed (via
> `localeIdentifier`) to lastHyphenLocation?

My system locale is the German (de / de_DE) locale. I don't know about those variables.

> In relation to that locale, how many dictionaries for that locale are stored
> at /usr/share/hyphen or /usr/local/share/hyphen?

/usr/share/hyphen: 29 locales, 6 are German (5 symlinks, one hyph_de_DE.dic file), 23 are English (22 symlinks, one hyph_en_US.dic file).

/usr/locale/share/hyphen: doesn't exist

> In my case, I'm using the 'en' locale, meaning that related locales will be
> mapped via the availableLocales() HashMap to three dictionary files:
> hyph_en_GB.dic, hyph_en_AU.dic and hyph_en_ZA.dic, as provided by the
> hyphen-en-gb package.
> 
> We're storing these in a TinyLRUCache with a default capacity of 4, meaning
> that they will fit in just fine.

No idea about that and the rest of your post.
hyph_en_US.dic is 106.1 kiB, hyph_de_DE.dic is 50.1 kiB.

[Michael Catanzaro]

Zan, can the problem you identified result in this assertion failure? I guess the assertion is a separate issue that just coincidentally is related to hyphenation and occurs on the same page?

(In reply to comment #5)
> With the 'en' locale, loading the Python PEP page, only three
> HyphenDictionary objects are created, with the underlying hyph_en_*.dic file
> opened and processed.

My locale is en_US.UTF-8. But I have 23 hyph_en_*.dic dictionaries installed, of which all except hyph_en_US.dic itself are symlinks to hyph_en_US.dic.

> With the 'es' locale, loading the same page, 1723 HyphenDictionary objects
> are created, loading each hyph_es_*.dic file 82 times. It still doesn't spin
> the CPU usage of the WebProcess to 100% on my system, but it's obviously a
> problem.

Thanks for debugging. :)

> While the TinyLRUCache capacity could be bumped, it should be noted that at
> least in Debian packages a lot of these locale variations for one specific
> locale under /usr/share/hyphen are simply symbolic links to that one master
> dict file. For instance, there's 21 different Spanish locales under
> /usr/share/hyphen, from Bolivian to Venezuelan, but it's 20 files just
> linking to the master hyphen_es_ES.dic file.
> 
> Same for the English locales -- hyph_en_AU.dic and hyph_en_ZA.dic link to
> hyph_en_GB.dic.
> 
> So we should maybe also look into detecting symlinks when storing these
> filepaths in the availableLocales() HashMap.

I guess we really ought to do both. A class named TinyLRUCache seems appropriate here, but I would never have assumed the capacity was as low as 4....

[Zan Dobersek]

(In reply to comment #8)
> Zan, can the problem you identified result in this assertion failure? I
> guess the assertion is a separate issue that just coincidentally is related
> to hyphenation and occurs on the same page?
> 

I don't know for sure, but I don't think it is.

[Zan Dobersek]

Created attachment 298478 [details]
WIP patch

[Michael Catanzaro]

Comment on attachment 298478 [details]
WIP patch

View in context: https://bugs.webkit.org/attachment.cgi?id=298478&action=review

It looks sane, thanks.

> Source/WebCore/platform/glib/FileSystemGlib.cpp:411
> +#if PLATFORM(GTK)

Since there's nothing GTK-specific here, I would omit the guards. This function works for any GLib port.

> Source/WebCore/platform/glib/FileSystemGlib.cpp:423
> +    if (!g_strcmp0(dirname.get(), ".")) {

What about ".."?

It would probably be easier to just use realpath(). Is there some disadvantage to doing that?

> Source/WebCore/platform/text/hyphen/HyphenationLibHyphen.cpp:181
> +    static TinyLRUCache<AtomicString, RefPtr<WebCore::HyphenationDictionary>, 8>& cache()

Maybe we should just use HashTable for this.

[WebKit Commit Bot]

Attachment 298478 [details] did not pass style-queue:


ERROR: Source/WebCore/ChangeLog:8:  You should remove the 'No new tests' and either add and list tests, or explain why no new tests were possible.  [changelog/nonewtests] [5]
Total errors found: 1 in 4 files


If any of these errors are false positives, please file a bug against check-webkit-style.

[Carlos Garcia Campos]

(In reply to comment #11)
> Comment on attachment 298478 [details]
> WIP patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=298478&action=review
> 
> It looks sane, thanks.
> 
> > Source/WebCore/platform/glib/FileSystemGlib.cpp:411
> > +#if PLATFORM(GTK)
> 
> Since there's nothing GTK-specific here, I would omit the guards. This
> function works for any GLib port.

Yes, we don't need any GTK guards in this file, only in the header. This is a bit confusing, in the case of FileSystem, this used to be FileSystemGtk, because it was only used by the GTK port, but we renamed it to glib because there's nothing specific to GTK+, but it's still only used by GTK+ port, other glib based ports use their own FileSystemFoo.

> > Source/WebCore/platform/glib/FileSystemGlib.cpp:423
> > +    if (!g_strcmp0(dirname.get(), ".")) {
> 
> What about ".."?
> 
> It would probably be easier to just use realpath(). Is there some
> disadvantage to doing that?

The only advantage I see of using g_read_link is that it can read file paths of any length, because glib uses a loop to try with bigger buffers if previous call to readlink failed because the buffer size was not enough. When using realpath the buffer size should be PATH_MAX. That's usually enough, and realpath not only follows symlinks but also resolves '. and '..'. And that's what I ended up doing for the plugins, without adding any new function to FileSystem:

Vector<String> result;
char normalizedPath[PATH_MAX];
for (const auto& path : listDirectory(directory, String("*.so"))) {
    CString filename = fileSystemRepresentation(path);
    if (realpath(filename.data(), normalizedPath))
        result.append(stringFromFileSystemRepresentation(normalizedPath));
}


> > Source/WebCore/platform/text/hyphen/HyphenationLibHyphen.cpp:181
> > +    static TinyLRUCache<AtomicString, RefPtr<WebCore::HyphenationDictionary>, 8>& cache()
> 
> Maybe we should just use HashTable for this.

That's out of the scope of this patch, I would say.

[Zan Dobersek]

Comment on attachment 298478 [details]
WIP patch

View in context: https://bugs.webkit.org/attachment.cgi?id=298478&action=review

>>> Source/WebCore/platform/glib/FileSystemGlib.cpp:411
>>> +#if PLATFORM(GTK)
>> 
>> Since there's nothing GTK-specific here, I would omit the guards. This function works for any GLib port.
> 
> Yes, we don't need any GTK guards in this file, only in the header. This is a bit confusing, in the case of FileSystem, this used to be FileSystemGtk, because it was only used by the GTK port, but we renamed it to glib because there's nothing specific to GTK+, but it's still only used by GTK+ port, other glib based ports use their own FileSystemFoo.

The flag is used because I think the implementation wouldn't work on OS(WINDOWS).

>>> Source/WebCore/platform/glib/FileSystemGlib.cpp:423
>>> +    if (!g_strcmp0(dirname.get(), ".")) {
>> 
>> What about ".."?
>> 
>> It would probably be easier to just use realpath(). Is there some disadvantage to doing that?
> 
> The only advantage I see of using g_read_link is that it can read file paths of any length, because glib uses a loop to try with bigger buffers if previous call to readlink failed because the buffer size was not enough. When using realpath the buffer size should be PATH_MAX. That's usually enough, and realpath not only follows symlinks but also resolves '. and '..'. And that's what I ended up doing for the plugins, without adding any new function to FileSystem:
> 
> Vector<String> result;
> char normalizedPath[PATH_MAX];
> for (const auto& path : listDirectory(directory, String("*.so"))) {
>     CString filename = fileSystemRepresentation(path);
>     if (realpath(filename.data(), normalizedPath))
>         result.append(stringFromFileSystemRepresentation(normalizedPath));
> }

I would actually prefer using realpath() directly in HyphenationLibHyphen, rather than adding a GTK+-specific function to FileSystem.h.

>>> Source/WebCore/platform/text/hyphen/HyphenationLibHyphen.cpp:181
>>> +    static TinyLRUCache<AtomicString, RefPtr<WebCore::HyphenationDictionary>, 8>& cache()
>> 
>> Maybe we should just use HashTable for this.
> 
> That's out of the scope of this patch, I would say.

A LRU cache is better because it does limit the amount of entries, and hence the memory consumption of the contained objects. OTOH if somehow the system ends up with multiple locale files being used, then this again is eating up CPU while opening these dictionaries over and over.

A larger capacity would maybe fit better, say 32 objects. It also helps that the dictionary files are not big (though I don't know how much memory the parsed dictionary data consumes), but I would prefer it to be capped off at some value.

[Zan Dobersek]

Created attachment 298604 [details]
Patch

[Michael Catanzaro]

Comment on attachment 298604 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=298604&action=review

Nice!

> Source/WebCore/ChangeLog:8
> +        No new tests (OOPS!).

Seems like it's ready for r+, except for this bit.

> Source/WebCore/platform/text/hyphen/HyphenationLibHyphen.cpp:189
> -    static TinyLRUCache<AtomicString, RefPtr<WebCore::HyphenationDictionary>>& cache()
> +    static TinyLRUCache<AtomicString, RefPtr<WebCore::HyphenationDictionary>, 8>& cache()
>      {
> -        static NeverDestroyed<TinyLRUCache<AtomicString, RefPtr<WebCore::HyphenationDictionary>>> cache;
> +        static NeverDestroyed<TinyLRUCache<AtomicString, RefPtr<WebCore::HyphenationDictionary>, 8>> cache;

32 or even 16 seems a lot safer than 8, indeed. It's easy to imagine a user has more than 8 hyphenation dictionaries installed. Anyway, I expect the primary improvement will come from resolving the symlinks.

[Carlos Garcia Campos]

Comment on attachment 298604 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=298604&action=review

> Source/WebCore/platform/text/hyphen/HyphenationLibHyphen.cpp:75
>          availableLocales.add(locale, Vector<String>()).iterator->value.append(filePath);

Do we really want to add the path even when realpath fails? Some of the errors could be serious, like ELOOP.

[Zan Dobersek]

Comment on attachment 298604 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=298604&action=review

>> Source/WebCore/platform/text/hyphen/HyphenationLibHyphen.cpp:75
>>          availableLocales.add(locale, Vector<String>()).iterator->value.append(filePath);
> 
> Do we really want to add the path even when realpath fails? Some of the errors could be serious, like ELOOP.

OK, makes sense.

>> Source/WebCore/platform/text/hyphen/HyphenationLibHyphen.cpp:189
>> +        static NeverDestroyed<TinyLRUCache<AtomicString, RefPtr<WebCore::HyphenationDictionary>, 8>> cache;
> 
> 32 or even 16 seems a lot safer than 8, indeed. It's easy to imagine a user has more than 8 hyphenation dictionaries installed. Anyway, I expect the primary improvement will come from resolving the symlinks.

Let's do 32.

[Zan Dobersek]

Created attachment 298669 [details]
Patch

[Build Bot]

Comment on attachment 298669 [details]
Patch

Attachment 298669 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/2877278

New failing tests:
fast/mediastream/MediaStream-video-element-track-stop.html

[Build Bot]

Created attachment 298676 [details]
Archive of layout-test-results from ews107 for mac-elcapitan-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews107  Port: mac-elcapitan-wk2  Platform: Mac OS X 10.11.6

[Zan Dobersek]

Comment on attachment 298669 [details]
Patch

Clearing flags on attachment: 298669

Committed r210670: <http://trac.webkit.org/changeset/210670>

[Zan Dobersek]

All reviewed patches have been landed.  Closing bug.