Bug 35263

Summary: [Qt] fast/encoding/invalid-UTF-8-2.html fails
Product: WebKit Reporter: Csaba Osztrogonác <ossy>
Component: Tools / TestsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED INVALID    
Severity: Normal CC: ap, kling, robert
Priority: P3 Keywords: LayoutTestFailure, Qt, QtTriaged
Version: 528+ (Nightly build)   
Hardware: PC   
OS: All   

Csaba Osztrogonác
Reported 2010-02-22 14:59:20 PST
fast/encoding/invalid-UTF-8-2.html was introduced by http://trac.webkit.org/changeset/55103, and it fails on Qt bot from the beginning: http://build.webkit.org/results/Qt%20Linux%20Release/r55103%20%287674%29/results.html original bug: https://bugs.webkit.org/show_bug.cgi?id=32937
Attachments
Add attachment proposed patch, testcase, etc.
Csaba Osztrogonác
Comment 1 2010-02-22 15:02:27 PST
expected file says: "The following should be: "т��т ���" or "т��т ��������":" Which one is correct? With this testing tool we can pass only one of them. Committed expected file says: т��т �������� , the buildbot says: т��т ���.
Alexey Proskuryakov
Comment 2 2010-02-22 15:11:09 PST
The shorter result is recommended by the Unicode spec, but both are correct. Robert's version of Qt apparently produces the longer result - per bug 32937 comment 2, that's intended, so I'm surprised that there is variance here.
Csaba Osztrogonác
Comment 3 2010-02-22 15:32:17 PST
Qt specific expected file for fast/encoding/invalid-UTF-8-2.html was removed by http://trac.webkit.org/changeset/55110 to make buildbot green again. Tomorrow we should find out what cause this difference.
Robert Hogan
Comment 4 2010-02-23 12:44:16 PST
(In reply to comment #1) > expected file says: "The following should be: "т��т ���" or "т��т ��������":" > > Which one is correct? With this testing tool we can pass only one of them. > Committed expected file says: т��т �������� , the buildbot says: т��т ���. I get т��т �������� with QtWebKit built against qt master. So I just tried against 4.6 and got the same result! robert@mwenge:~$ cd Development/webkit/ robert@mwenge:~/Development/webkit$ ./WebKitTools/Scripts/build-webkit --qt --makeargs=-j5 --minimal Generating derived sources Calling 'qmake -r DEFINES+=ENABLE_3D_RENDERING=0 DEFINES+=ENABLE_CHANNEL_MESSAGING=0 DEFINES+=ENABLE_DATABASE=0 DEFINES+=ENABLE_DATALIST=0 DEFINES+=ENABLE_DOM_STORAGE=0 DEFINES+=ENABLE_EVENTSOURCE=0 DEFINES+=ENABLE_FILTERS=0 DEFINES+=ENABLE_ICONDATABASE=0 DEFINES+=ENABLE_JAVASCRIPT_DEBUGGER=0 DEFINES+=ENABLE_OFFLINE_WEB_APPLICATIONS=0 DEFINES+=ENABLE_RUBY=0 DEFINES+=ENABLE_SHARED_WORKERS=0 DEFINES+=ENABLE_SVG=0 DEFINES+=ENABLE_SVG_ANIMATION=0 DEFINES+=ENABLE_SVG_AS_IMAGE=0 DEFINES+=ENABLE_SVG_FONTS=0 DEFINES+=ENABLE_SVG_FOREIGN_OBJECT=0 DEFINES+=ENABLE_SVG_USE=0 DEFINES+=ENABLE_WEB_SOCKETS=0 DEFINES+=ENABLE_WORKERS=0 DEFINES+=ENABLE_XPATH=0 DEFINES+=ENABLE_XSLT=0 -r /home/robert/Development/webkit/DerivedSources.pro -o Makefile.DerivedSources' in /home/robert/Development/webkit/WebKitBuild/utf8/Release Reading /home/robert/Development/webkit/JavaScriptCore/DerivedSources.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/JavaScriptCore] Reading /home/robert/Development/webkit/WebCore/DerivedSources.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebCore] Reading /home/robert/Development/webkit/WebKit/qt/Api/DerivedSources.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/Api] Calling 'make -j5 -f Makefile.DerivedSources generated_files' in /home/robert/Development/webkit/WebKitBuild/utf8/Release/JavaScriptCore make: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/JavaScriptCore' make: Nothing to be done for `generated_files'. make: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/JavaScriptCore' Calling 'make -j5 -f Makefile.DerivedSources generated_files' in /home/robert/Development/webkit/WebKitBuild/utf8/Release/WebCore make: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebCore' make: Nothing to be done for `generated_files'. make: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebCore' Calling 'make -j5 -f Makefile.DerivedSources generated_files' in /home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/Api make: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/Api' make: Nothing to be done for `generated_files'. make: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/Api' Calling 'qmake -r DEFINES+=ENABLE_3D_RENDERING=0 DEFINES+=ENABLE_CHANNEL_MESSAGING=0 DEFINES+=ENABLE_DATABASE=0 DEFINES+=ENABLE_DATALIST=0 DEFINES+=ENABLE_DOM_STORAGE=0 DEFINES+=ENABLE_EVENTSOURCE=0 DEFINES+=ENABLE_FILTERS=0 DEFINES+=ENABLE_ICONDATABASE=0 DEFINES+=ENABLE_JAVASCRIPT_DEBUGGER=0 DEFINES+=ENABLE_OFFLINE_WEB_APPLICATIONS=0 DEFINES+=ENABLE_RUBY=0 DEFINES+=ENABLE_SHARED_WORKERS=0 DEFINES+=ENABLE_SVG=0 DEFINES+=ENABLE_SVG_ANIMATION=0 DEFINES+=ENABLE_SVG_AS_IMAGE=0 DEFINES+=ENABLE_SVG_FONTS=0 DEFINES+=ENABLE_SVG_FOREIGN_OBJECT=0 DEFINES+=ENABLE_SVG_USE=0 DEFINES+=ENABLE_WEB_SOCKETS=0 DEFINES+=ENABLE_WORKERS=0 DEFINES+=ENABLE_XPATH=0 DEFINES+=ENABLE_XSLT=0 OUTPUT_DIR=/home/robert/Development/webkit/WebKitBuild/utf8/Release /home/robert/Development/webkit/WebKit.pro CONFIG+=release CONFIG-=debug' in /home/robert/Development/webkit/WebKitBuild/utf8/Release Reading /home/robert/Development/webkit/WebCore/WebCore.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebCore] /home/robert/qtsdk-2009.05/qt/bin/rcc: File does not exist '../../../../WebCore/WebCore.qrc' /home/robert/qtsdk-2009.05/qt/bin/rcc: File does not exist '../../../../WebCore/inspector/front-end/WebKit.qrc' Reading /home/robert/Development/webkit/WebKitTools/QtLauncher/QtLauncher.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKitTools/QtLauncher] Reading /home/robert/Development/webkit/WebKit/qt/QGVLauncher/QGVLauncher.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/QGVLauncher] Reading /home/robert/Development/webkit/JavaScriptCore/jsc.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/JavaScriptCore] Reading /home/robert/Development/webkit/WebKit/qt/tests/tests.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests] Reading /home/robert/Development/webkit/WebKit/qt/tests/qwebframe/qwebframe.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qwebframe] Reading /home/robert/Development/webkit/WebKit/qt/tests/qwebpage/qwebpage.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qwebpage] Reading /home/robert/Development/webkit/WebKit/qt/tests/qwebelement/qwebelement.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qwebelement] Reading /home/robert/Development/webkit/WebKit/qt/tests/qgraphicswebview/qgraphicswebview.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qgraphicswebview] Reading /home/robert/Development/webkit/WebKit/qt/tests/qwebhistoryinterface/qwebhistoryinterface.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qwebhistoryinterface] Reading /home/robert/Development/webkit/WebKit/qt/tests/qwebview/qwebview.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qwebview] Reading /home/robert/Development/webkit/WebKit/qt/tests/qwebhistory/qwebhistory.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qwebhistory] Reading /home/robert/Development/webkit/WebKit/qt/tests/qwebinspector/qwebinspector.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qwebinspector] Reading /home/robert/Development/webkit/WebKit/qt/tests/hybridPixmap/hybridPixmap.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/hybridPixmap] Reading /home/robert/Development/webkit/WebKit/qt/tests/benchmarks/painting/painting.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/benchmarks/painting] Reading /home/robert/Development/webkit/WebKit/qt/tests/benchmarks/loading/loading.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/benchmarks/loading] Reading /home/robert/Development/webkit/WebKitTools/DumpRenderTree/qt/DumpRenderTree.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKitTools/DumpRenderTree/qt] Reading /home/robert/Development/webkit/WebKitTools/DumpRenderTree/qt/ImageDiff.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKitTools/DumpRenderTree/qt] Reading /home/robert/Development/webkit/WebKitTools/DumpRenderTree/qt/TestNetscapePlugin/TestNetscapePlugin.pro [/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKitTools/DumpRenderTree/qt/TestNetscapePlugin] Calling 'make -j5' in /home/robert/Development/webkit/WebKitBuild/utf8/Release cd WebCore/ && make -f Makefile make[1]: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebCore' make[1]: Nothing to be done for `first'. make[1]: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebCore' cd WebKitTools/QtLauncher/ && make -f Makefile make[1]: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKitTools/QtLauncher' make[1]: Nothing to be done for `first'. make[1]: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKitTools/QtLauncher' cd WebKit/qt/QGVLauncher/ && make -f Makefile make[1]: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/QGVLauncher' make[1]: Nothing to be done for `first'. make[1]: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/QGVLauncher' cd JavaScriptCore/ && make -f Makefile.jsc make[1]: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/JavaScriptCore' make[1]: Nothing to be done for `first'. make[1]: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/JavaScriptCore' cd WebKit/qt/tests/ && make -f Makefile make[1]: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests' cd qwebframe/ && make -f Makefile cd qwebpage/ && make -f Makefile cd qwebelement/ && make -f Makefile cd qgraphicswebview/ && make -f Makefile make[2]: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qwebframe' make[2]: Nothing to be done for `first'. make[2]: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qwebframe' cd qwebhistoryinterface/ && make -f Makefile make[2]: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qwebpage' make[2]: Nothing to be done for `first'. make[2]: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qwebpage' make[2]: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qwebhistoryinterface' make[2]: Nothing to be done for `first'. make[2]: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qwebhistoryinterface' cd qwebview/ && make -f Makefile make[2]: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qwebview' make[2]: Nothing to be done for `first'. make[2]: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qwebview' cd qwebhistory/ && make -f Makefile make[2]: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qwebhistory' make[2]: Nothing to be done for `first'. make[2]: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qwebhistory' cd qwebinspector/ && make -f Makefile make[2]: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qgraphicswebview' make[2]: Nothing to be done for `first'. make[2]: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qgraphicswebview' make[2]: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qwebinspector' make[2]: Nothing to be done for `first'. make[2]: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qwebinspector' cd hybridPixmap/ && make -f Makefile make[2]: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qwebelement' make[2]: Nothing to be done for `first'. make[2]: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/qwebelement' make[2]: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/hybridPixmap' make[2]: Nothing to be done for `first'. make[2]: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/hybridPixmap' cd benchmarks/painting/ && make -f Makefile make[2]: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/benchmarks/painting' make[2]: Nothing to be done for `first'. make[2]: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/benchmarks/painting' cd benchmarks/loading/ && make -f Makefile make[2]: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/benchmarks/loading' make[2]: Nothing to be done for `first'. make[2]: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests/benchmarks/loading' make[1]: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKit/qt/tests' cd WebKitTools/DumpRenderTree/qt/ && make -f Makefile.DumpRenderTree make[1]: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKitTools/DumpRenderTree/qt' make[1]: Nothing to be done for `first'. make[1]: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKitTools/DumpRenderTree/qt' cd WebKitTools/DumpRenderTree/qt/ && make -f Makefile.ImageDiff make[1]: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKitTools/DumpRenderTree/qt' make[1]: Nothing to be done for `first'. make[1]: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKitTools/DumpRenderTree/qt' cd WebKitTools/DumpRenderTree/qt/TestNetscapePlugin/ && make -f Makefile make[1]: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKitTools/DumpRenderTree/qt/TestNetscapePlugin' /home/robert/qtsdk-2009.05/qt/bin/qmake -unix DEFINES+=ENABLE_3D_RENDERING=0 DEFINES+=ENABLE_CHANNEL_MESSAGING=0 DEFINES+=ENABLE_DATABASE=0 DEFINES+=ENABLE_DATALIST=0 DEFINES+=ENABLE_DOM_STORAGE=0 DEFINES+=ENABLE_EVENTSOURCE=0 DEFINES+=ENABLE_FILTERS=0 DEFINES+=ENABLE_ICONDATABASE=0 DEFINES+=ENABLE_JAVASCRIPT_DEBUGGER=0 DEFINES+=ENABLE_OFFLINE_WEB_APPLICATIONS=0 DEFINES+=ENABLE_RUBY=0 DEFINES+=ENABLE_SHARED_WORKERS=0 DEFINES+=ENABLE_SVG=0 DEFINES+=ENABLE_SVG_ANIMATION=0 DEFINES+=ENABLE_SVG_AS_IMAGE=0 DEFINES+=ENABLE_SVG_FONTS=0 DEFINES+=ENABLE_SVG_FOREIGN_OBJECT=0 DEFINES+=ENABLE_SVG_USE=0 DEFINES+=ENABLE_WEB_SOCKETS=0 DEFINES+=ENABLE_WORKERS=0 DEFINES+=ENABLE_XPATH=0 DEFINES+=ENABLE_XSLT=0 OUTPUT_DIR=/home/robert/Development/webkit/WebKitBuild/utf8/Release CONFIG+=release CONFIG-=debug -o Makefile /home/robert/Development/webkit/WebKitTools/DumpRenderTree/qt/TestNetscapePlugin/TestNetscapePlugin.pro make[1]: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKitTools/DumpRenderTree/qt/TestNetscapePlugin' make[1]: Entering directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKitTools/DumpRenderTree/qt/TestNetscapePlugin' make[1]: Nothing to be done for `first'. make[1]: Leaving directory `/home/robert/Development/webkit/WebKitBuild/utf8/Release/WebKitTools/DumpRenderTree/qt/TestNetscapePlugin' =========================================================== WebKit is now built (00m:22s). To run QtLauncher with this newly-built code, use the "WebKitTools/Scripts/run-launcher" script. =========================================================== robert@mwenge:~/Development/webkit$ qmake --version QMake version 2.01a Using Qt version 4.6.0 in /home/robert/qtsdk-2009.05/qt/lib robert@mwenge:~/Development/webkit$ WebKitBuild/utf8/Release/bin/DumpRenderTree -v LayoutTests/fast/encoding/invalid-UTF-8-2.html Source: <html><head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <style> /* Force font fallback on Windows to match Mac OS X */ @font-face { font-family: 'times'; src: local('Lucida Grande'); unicode-range: U+FFFD; } </style> </head><body> <script> if (window.layoutTestController) layoutTestController.dumpAsText(); </script> <p>This tests the decoding of invalid UTF-8 sequences.</p> The following should be: "т��т ���" or "т��т ��������": <p>т��т ��������</p> </body></html> This tests the decoding of invalid UTF-8 sequences. The following should be: "т��т ���" or "т��т ��������": т��т �������� #EOF #EOF #EOF robert@mwenge:~/Development/webkit$ WebKitTools/Scripts/run-webkit-tests --qt LayoutTests/fast/encoding/invalid-UTF-8-2.html Running build-dumprendertree Running tests from /home/robert/Development/webkit/LayoutTests Testing 1 test cases. fast/encoding . 0.89s total testing time all 1 test cases succeeded robert@mwenge:~/Development/webkit$
Robert Hogan
Comment 5 2010-02-23 13:50:15 PST
I've tried clean builds against Qt 4.6.0 and Qt 4.6.2 with the same results. I've even tried changing my locale settings (per Ossy's suggestion) without effect. robert@mwenge:~/Development/webkit$ locale LANG=en_IE.UTF-8 LANGUAGE=en_IE.UTF-8 LC_CTYPE="en_IE.UTF-8" LC_NUMERIC="en_IE.UTF-8" LC_TIME="en_IE.UTF-8" LC_COLLATE="en_IE.UTF-8" LC_MONETARY="en_IE.UTF-8" LC_MESSAGES="en_IE.UTF-8" LC_PAPER="en_IE.UTF-8" LC_NAME="en_IE.UTF-8" LC_ADDRESS="en_IE.UTF-8" LC_TELEPHONE="en_IE.UTF-8" LC_MEASUREMENT="en_IE.UTF-8" LC_IDENTIFICATION="en_IE.UTF-8" LC_ALL=en_IE.UTF-8 I've also tried LC_ALL=en_US.utf8 So this is a complete mystery to me.
Robert Hogan
Comment 6 2010-02-23 14:07:05 PST
OK, the penny has dropped (sort of), I see т��т �������� when viewing fast/encoding/invalid-UTF-8-2.html in cat while Ossy sees т��т ���, so it must be something to do with localisation settings on my distro (Kubuntu Karmic). I haven't a clue what those settings could be.
Alexey Proskuryakov
Comment 7 2010-02-24 08:35:43 PST
A good answer to this would be switching to our own UTF-8 decoder in WebCore - we already have one in WTF.
Csaba Osztrogonác
Comment 8 2010-02-24 08:38:42 PST
(In reply to comment #6) > OK, the penny has dropped (sort of), I see т��т �������� when viewing > fast/encoding/invalid-UTF-8-2.html in cat while Ossy sees т��т ���, so it must > be something to do with localisation settings on my distro (Kubuntu Karmic). > > I haven't a clue what those settings could be. I tried it on our Ubuntu Karmic and DRT also dumped т��т ��� .
Jocelyn Turcotte
Comment 9 2014-02-03 03:13:23 PST
=== Bulk closing of Qt bugs === If you believe that this bug report is still relevant for a non-Qt port of webkit.org, please re-open it and remove [Qt] from the summary. If you believe that this is still an important QtWebKit bug, please fill a new report at https://bugreports.qt-project.org and add a link to this issue. See http://qt-project.org/wiki/ReportingBugsInQt for additional guidelines.
Note You need to log in before you can comment on or make changes to this bug.