Bug 95749 - [Qt] Enable the LLInt on Linux
Summary: [Qt] Enable the LLInt on Linux
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: 528+ (Nightly build)
Hardware: All All
: P2 Normal
Assignee: Csaba Osztrogonác
URL:
Keywords: Qt, QtTriaged
: 80839 (view as bug list)
Depends on: 96509
Blocks: 97584 97587 97589 97648
  Show dependency treegraph
 
Reported: 2012-09-04 08:00 PDT by Csaba Osztrogonác
Modified: 2012-09-27 06:31 PDT (History)
15 users (show)

See Also:


Attachments
Patch (525.44 KB, patch)
2012-09-07 08:19 PDT, Csaba Osztrogonác
no flags Details | Formatted Diff | Diff
LLInt C loop (444.39 KB, patch)
2012-09-07 08:56 PDT, Csaba Osztrogonác
no flags Details | Formatted Diff | Diff
LLInt C loop (442.18 KB, patch)
2012-09-10 06:31 PDT, Csaba Osztrogonác
no flags Details | Formatted Diff | Diff
LLInt C loop - JSC test results (61.47 KB, text/html)
2012-09-10 06:36 PDT, Csaba Osztrogonác
no flags Details
LLInt asm ver1 (5.45 KB, patch)
2012-09-10 13:08 PDT, Csaba Osztrogonác
no flags Details | Formatted Diff | Diff
LLInt asm ver1 - build log (9.35 KB, text/plain)
2012-09-10 13:08 PDT, Csaba Osztrogonác
no flags Details
LLInt asm ver2 (4.99 KB, patch)
2012-09-10 13:09 PDT, Csaba Osztrogonác
no flags Details | Formatted Diff | Diff
LLInt asm ver2 - build log (9.46 KB, text/plain)
2012-09-10 13:09 PDT, Csaba Osztrogonác
no flags Details
patch with change log (5.69 KB, patch)
2012-09-25 07:34 PDT, Csaba Osztrogonác
no flags Details | Formatted Diff | Diff
updated patch (5.75 KB, patch)
2012-09-26 10:08 PDT, Csaba Osztrogonác
no flags Details | Formatted Diff | Diff
updated patch (5.77 KB, patch)
2012-09-26 10:11 PDT, Csaba Osztrogonác
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Csaba Osztrogonác 2012-09-04 08:00:13 PDT
Once we will need to enable LLInt.
( GTK did it previously - https://bugs.webkit.org/show_bug.cgi?id=88315 )
Comment 1 Csaba Osztrogonác 2012-09-04 08:29:58 PDT
to help fixing this bug, there are some useful commit:
- LLInt introduced in https://trac.webkit.org/changeset/108444
- enabled on GTK - https://trac.webkit.org/changeset/119593
- LLInt C loop backand landed in https://trac.webkit.org/changeset/127374
Comment 2 Csaba Osztrogonác 2012-09-06 02:15:19 PDT
I managed to build JSC with LLInt enabled manually. 
But inegrating it to the build system needs more work.

As far as I see we have to do the following things:
- generate LLIntDesiredOffsets.h: ruby offlineasm/generate_offset_extractor.rb llint/LowLevelInterpreter.asm generated/LLIntDesiredOffsets.h
- build LLIntOffsetsExtractor binary (It depends on LLIntOffsetsExtractor.cpp,h and the new generated LLIntDesiredOffsets.h)
- generate LLIntAssembly.h: ruby offlineasm/asm.rb llint/LowLevelInterpreter.asm LLIntOffsetsExtractor LLIntAssembly.h (It depends on the binary LLIntOffsetsExtractor)

Am I correct?
Comment 3 Filip Pizlo 2012-09-06 10:21:27 PDT
(In reply to comment #2)
> I managed to build JSC with LLInt enabled manually. 
> But inegrating it to the build system needs more work.
> 
> As far as I see we have to do the following things:
> - generate LLIntDesiredOffsets.h: ruby offlineasm/generate_offset_extractor.rb llint/LowLevelInterpreter.asm generated/LLIntDesiredOffsets.h
> - build LLIntOffsetsExtractor binary (It depends on LLIntOffsetsExtractor.cpp,h and the new generated LLIntDesiredOffsets.h)
> - generate LLIntAssembly.h: ruby offlineasm/asm.rb llint/LowLevelInterpreter.asm LLIntOffsetsExtractor LLIntAssembly.h (It depends on the binary LLIntOffsetsExtractor)
> 
> Am I correct?

Yup, that's right!
Comment 4 Andras Becsi 2012-09-07 05:53:43 PDT
Note that the old interpreter will be removed on Sept 24th:
http://lists.webkit.org/pipermail/webkit-dev/2012-September/022157.html

Ossy, is someone working on this on your side?
Comment 5 Csaba Osztrogonác 2012-09-07 05:56:09 PDT
(In reply to comment #4)
> Note that the old interpreter will be removed on Sept 24th:
> http://lists.webkit.org/pipermail/webkit-dev/2012-September/022157.html
> 
> Ossy, is someone working on this on your side?

Yes, I read this mail, so the timing is the best. :) I started working on it 
and I think Gábor Ballabás will join us with the ARM related fixes soon.
Comment 6 Csaba Osztrogonác 2012-09-07 07:26:29 PDT
*** Bug 80839 has been marked as a duplicate of this bug. ***
Comment 7 Csaba Osztrogonác 2012-09-07 08:19:21 PDT
Created attachment 162778 [details]
Patch

Preliminary patch for enabling LLInt (not C loop now). Now I added the generated LLIntAssembly.h for x86_64. Of course, I'll make the build system to generate it automatically soon.
Comment 8 Csaba Osztrogonác 2012-09-07 08:56:54 PDT
Created attachment 162783 [details]
LLInt C loop

I'm fighting with LLInt C loop too. I generated the 
LLIntAssembly.h manually and then tried to build JSC.

I had to fix the following opcodes, because GCC suggested parenthesis:
- OFFLINE_ASM_OPCODE_LABEL(op_pre_inc)
- OFFLINE_ASM_OPCODE_LABEL(op_pre_dec)
- OFFLINE_ASM_OPCODE_LABEL(op_post_inc)
- OFFLINE_ASM_OPCODE_LABEL(op_post_dec)
- OFFLINE_ASM_OPCODE_LABEL(op_add)
- OFFLINE_ASM_OPCODE_LABEL(op_sub)

SIGN_BIT32(b-a) --> SIGN_BIT32((b-a))
SIGN_BIT32(b+a) --> SIGN_BIT32((b+a))

I assume it can be fixed in the original code somewhere.

But the bigger problem is the following error messege:
/home/oszi/WebKit/Source/JavaScriptCore/llint/LLIntAssembly.h:125: error: ‘assert’ was not declared in this scope

Have you got any ide what can be the problem and how can we fix it?
Comment 9 Mark Lam 2012-09-07 10:43:46 PDT
(In reply to comment #8)
> SIGN_BIT32(b-a) --> SIGN_BIT32((b-a))
> SIGN_BIT32(b+a) --> SIGN_BIT32((b+a))

> But the bigger problem is the following error messege:
> /home/oszi/WebKit/Source/JavaScriptCore/llint/LLIntAssembly.h:125: error: ‘assert’ was not declared in this scope

These are both bugs.  The fix is being submitted in:
https://bugs.webkit.org/show_bug.cgi?id=96127
Comment 10 Mark Lam 2012-09-07 17:01:47 PDT
(In reply to comment #9)
> These are both bugs.  The fix is being submitted in:
> https://bugs.webkit.org/show_bug.cgi?id=96127

Fix landed at <http://trac.webkit.org/changeset/127938>.
Comment 11 Csaba Osztrogonác 2012-09-07 23:51:37 PDT
Thanks for the quick fix, I'll try the LLInt C loop on monday again. ;-)
Comment 12 Mark Lam 2012-09-09 22:35:11 PDT
r128015: <http://trac.webkit.org/changeset/128015>(In reply to comment #10)
> (In reply to comment #9)
> > These are both bugs.  The fix is being submitted in:
> > https://bugs.webkit.org/show_bug.cgi?id=96127
> 
> Fix landed at <http://trac.webkit.org/changeset/127938>.

r127938 was rolled out.  New fix landed in r128015: <http://trac.webkit.org/changeset/128015>.
Comment 13 Csaba Osztrogonác 2012-09-10 06:31:05 PDT
Created attachment 163113 [details]
LLInt C loop

updated to top of trunk (automatical LLIntAssembly.h generating is still missing)
Comment 14 Csaba Osztrogonác 2012-09-10 06:36:09 PDT
Created attachment 163114 [details]
LLInt C loop - JSC test results

With LLInt C loop 63 JSC tests fail (log attached) and ~1300 layout tests crash.

Is it "normal" now? Is there a bug report for it? 
Is there a build bot run "LLInt C loop" jsc and layout tests?
Comment 15 Mark Lam 2012-09-10 11:00:40 PDT
(In reply to comment #14)
> Created an attachment (id=163114) [details]
> LLInt C loop - JSC test results
> 
> With LLInt C loop 63 JSC tests fail (log attached) and ~1300 layout tests crash.
> 
> Is it "normal" now? Is there a bug report for it? 
> Is there a build bot run "LLInt C loop" jsc and layout tests?

I only ran fast/js, fast/regex, ietestcenter/Javascript, and sputnik at the time.  There were only 3 failures: 1 GC time out (runs fine when run by itself), and 2 crashes in regex that were pre-existing bugs (https://bugs.webkit.org/show_bug.cgi?id=94191).  Can you try running those tests and see if you are getting the same result?

I will kick off a full layout test and see if there are issues due to the llint C loop.  If there are additional issues to this interpreter, I will fix it.  Thanks.
Comment 16 Thiago Marcos P. Santos 2012-09-10 12:34:54 PDT
Comment on attachment 163113 [details]
LLInt C loop

View in context: https://bugs.webkit.org/attachment.cgi?id=163113&action=review

> Source/WTF/wtf/Platform.h:922
> +    && (PLATFORM(MAC) || PLATFORM(IOS) || PLATFORM(GTK)) || PLATFORM(QT) \

Isn't the PLATFORM(QT) misplaced here outside the main ( )?
Comment 17 Csaba Osztrogonác 2012-09-10 12:37:51 PDT
(In reply to comment #16)
> (From update of attachment 163113 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=163113&action=review
> 
> > Source/WTF/wtf/Platform.h:922
> > +    && (PLATFORM(MAC) || PLATFORM(IOS) || PLATFORM(GTK)) || PLATFORM(QT) \
> 
> Isn't the PLATFORM(QT) misplaced here outside the main ( )?

Thanks, it is a typo.
Comment 18 Csaba Osztrogonác 2012-09-10 13:08:14 PDT
Created attachment 163185 [details]
LLInt asm ver1
Comment 19 Csaba Osztrogonác 2012-09-10 13:08:44 PDT
Created attachment 163186 [details]
LLInt asm ver1 - build log
Comment 20 Csaba Osztrogonác 2012-09-10 13:09:16 PDT
Created attachment 163187 [details]
LLInt asm ver2
Comment 21 Csaba Osztrogonác 2012-09-10 13:09:44 PDT
Created attachment 163188 [details]
LLInt asm ver2 - build log
Comment 22 Csaba Osztrogonác 2012-09-10 13:13:02 PDT
I haven't managed to add the 3 step to Qt's build system. :-/

Tor Arne, could you check "LLInt asm ver1" and "LLInt asm ver2"
patches and their build logs? It seems we generating derived
sources always run before build anything else ... Did I something
wrong? I was lost in the qmake's magic world. :-/
Comment 23 Mark Lam 2012-09-10 15:52:21 PDT
(In reply to comment #14)
> Created an attachment (id=163114) [details]
> LLInt C loop - JSC test results
> 
> With LLInt C loop 63 JSC tests fail (log attached) and ~1300 layout tests crash.
> 
> Is it "normal" now? Is there a bug report for it? 
> Is there a build bot run "LLInt C loop" jsc and layout tests?

(In reply to comment #15)
> I will kick off a full layout test and see if there are issues due to the llint C loop.

The full results of the layout test (on platform mac 64bit x86) based on  r128016 are recorded in comment <https://bugs.webkit.org/show_bug.cgi?id=91052#c33> and attachment <https://bug-91052-attachments.webkit.org/attachment.cgi?id=163231>.  Please use that as your reference.  A bug was filed to investigate the found failures <https://bugs.webkit.org/show_bug.cgi?id=96333>, but that is a lower priority right now because the failures don't appear to be regressions.

For comparison with your results, on the C++ llint, I only saw 11 failures w/ 2 crashes.  In short, I would not expect to see 63 JSC test failures, and certainly not 1300 layout test crashes.  Are you building for a 64bit x86 or some other target?

And no, there isn't currently any bot that runs the C++ llint.  That is unless one of the ports will adopt it as its main configuration going forward.
Comment 24 Tor Arne Vestbø 2012-09-11 05:03:27 PDT
Comment on attachment 163185 [details]
LLInt asm ver1

View in context: https://bugs.webkit.org/attachment.cgi?id=163185&action=review

> Source/JavaScriptCore/LLIntOffsetsExtractor.pro:2
> +# Project file for the jsc binary (interactive interpreter)

LLIntOffsetsExtractor

> Source/JavaScriptCore/LLIntOffsetsExtractor.pro:6
> + TEMPLATE = app

added space

> Source/JavaScriptCore/LLIntOffsetsExtractor.pro:9
> +DESTDIR = $${ROOT_BUILD_DIR}/bin

I think we can put this binary in $${ROOT_BUILD_DIR}/Source/JavaScriptCore/, or $$OUT_PWD, since it's not useful except for the build

> Source/JavaScriptCore/LLIntOffsetsExtractor.pro:11
> +QT -= gui

QT =

> Source/JavaScriptCore/LLIntOffsetsExtractor.pro:15
> +JAVASCRIPTCORE_GENERATED_SOURCES_DIR = $${ROOT_BUILD_DIR}/Source/JavaScriptCore/$${GENERATED_SOURCES_DESTDIR}

We don't need this, since the binary is built before the derived sources are generated

> Source/JavaScriptCore/LLIntOffsetsExtractor.pro:37
> +    $$SOURCE_DIR \
> +    $$SOURCE_DIR/.. \
> +    $$SOURCE_DIR/../WTF \
> +    $$SOURCE_DIR/assembler \
> +    $$SOURCE_DIR/bytecode \
> +    $$SOURCE_DIR/bytecompiler \
> +    $$SOURCE_DIR/heap \
> +    $$SOURCE_DIR/dfg \
> +    $$SOURCE_DIR/debugger \
> +    $$SOURCE_DIR/disassembler \
> +    $$SOURCE_DIR/interpreter \
> +    $$SOURCE_DIR/jit \
> +    $$SOURCE_DIR/llint \
> +    $$SOURCE_DIR/parser \
> +    $$SOURCE_DIR/profiler \
> +    $$SOURCE_DIR/runtime \
> +    $$SOURCE_DIR/tools \
> +    $$SOURCE_DIR/yarr \
> +    $$SOURCE_DIR/API \
> +    $$SOURCE_DIR/ForwardingHeaders \

Do we need all of these include paths?

> Source/JavaScriptCore/LLIntOffsetsExtractor.pro:42
> +win32-msvc*|win32-icc: INCLUDEPATH += $$ROOT_WEBKIT_DIR/Source/JavaScriptCore/os-win32

Do we need this?

> Source/JavaScriptCore/LLIntOffsetsExtractor.pro:56
> +mac {
> +    LIBS_PRIVATE += -framework AppKit
> +}
> +
> +win* {
> +    LIBS += -ladvapi32
> +}
> +
> +wince* {
> +    LIBS += mmtimer.lib
> +}

Do we need these?

> Source/JavaScriptCore/LLIntOffsetsExtractor.pro:72
> +#GENERATOR: LLIntOffsetsExtractor
> +llint.output = LLIntDesiredOffsets.h
> +llint.script = $$PWD/offlineasm/generate_offset_extractor.rb
> +llint.input = INPUT_FILES
> +llint.commands = ruby $$llint.script ${QMAKE_FILE_NAME} ${QMAKE_FILE_OUT}
> +llint.CONFIG += no_link
> +QMAKE_EXTRA_COMPILERS  += llint
> +#GENERATORS += llint

We might need an extra depends here, to ensure the compilation of LLIntOffsetsExtractor.cpp happens after the generator has run

> Tools/Scripts/build-jsc:66
> +    my @projects = ("WTF", "LLIntOffsetsExtractor", "JavaScriptCore");

No need for this

> WebKit.pro:50
> +LLIntOffsetsExtractor.file = Source/JavaScriptCore/LLIntOffsetsExtractor.pro
> +LLIntOffsetsExtractor.makefile = Makefile.LLIntOffsetsExtractor
> +SUBDIRS += LLIntOffsetsExtractor

This should be in JavaScriptCore.pro
Comment 25 Tor Arne Vestbø 2012-09-11 05:07:12 PDT
Comment on attachment 163187 [details]
LLInt asm ver2

View in context: https://bugs.webkit.org/attachment.cgi?id=163187&action=review

> Source/JavaScriptCore/JavaScriptCore.pro:16
> +SUBDIRS += LLIntOffsetsExtractor
>  derived_sources.file = DerivedSources.pri
>  target.file = Target.pri
>  
> -SUBDIRS += derived_sources target
> +SUBDIRS += LLIntOffsetsExtractor derived_sources target

Duplicate SUBDIRS += here
Comment 26 Tor Arne Vestbø 2012-09-12 06:23:52 PDT
Comment on attachment 163187 [details]
LLInt asm ver2

View in context: https://bugs.webkit.org/attachment.cgi?id=163187&action=review

> Source/WTF/wtf/Platform.h:33
> +#define ENABLE_JIT 0
> +#define ENABLE_LLINT 1
> +#define ENABLE_CLASSIC_INTERPRETER 0

I assume these are just for debugging purposes? The addition of || PLATFORM(QT) below, to enable LLINT, should be enough?
Comment 27 Csaba Osztrogonác 2012-09-12 06:30:10 PDT
(In reply to comment #26)
> (From update of attachment 163187 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=163187&action=review
> 
> > Source/WTF/wtf/Platform.h:33
> > +#define ENABLE_JIT 0
> > +#define ENABLE_LLINT 1
> > +#define ENABLE_CLASSIC_INTERPRETER 0
> 
> I assume these are just for debugging purposes? The addition of || PLATFORM(QT) below, to enable LLINT, should be enough?

It is for testing the LLInt C loop backend. (LLInt without classical JIT and classical interpreter) If we only add "PLATFORM(QT)", it enables LLINT, not
LLINT C loop.
Comment 28 Csaba Osztrogonác 2012-09-12 06:57:22 PDT
Qt LLInt build will be fixed soon ... but it seems LLInt C loop build is 
broken now - https://bugs.webkit.org/show_bug.cgi?id=96509

Could you guys check it, please?
Comment 29 Tor Arne Vestbø 2012-09-12 07:14:04 PDT
This patch should be ~ok build-wise 

https://gist.github.com/3706873
Comment 30 Tor Arne Vestbø 2012-09-12 07:34:15 PDT
(In reply to comment #29)
> This patch should be ~ok build-wise 
> 
> https://gist.github.com/3706873

Though I see this build error:

https://gist.github.com/3707018
Comment 31 Csaba Osztrogonác 2012-09-25 07:34:34 PDT
Created attachment 165611 [details]
patch with change log

We still need buildfix on Mac, and should check it on Windows.
Comment 32 Csaba Osztrogonác 2012-09-25 07:42:14 PDT
Zoltán, could you try it on Windows, please?
Comment 33 Csaba Osztrogonác 2012-09-25 07:54:04 PDT
Comment on attachment 165611 [details]
patch with change log

View in context: https://bugs.webkit.org/attachment.cgi?id=165611&action=review

> Source/WTF/wtf/Platform.h:914
> +    && (PLATFORM(MAC) || PLATFORM(IOS) || PLATFORM(GTK) || PLATFORM(QT)) \

conservative solution: enable it for Linux and QNX only: PLATFORM(QT) && ( OS(LINUX) || OS(QNX) )
But the best solution would be if we can enable it for PLATFORM(QT)
Comment 34 Csaba Osztrogonác 2012-09-25 07:56:59 PDT
OS(QNX) isn't needed, because JIT is disabled on it.
Comment 35 Milian Wolff 2012-09-25 07:57:30 PDT
(In reply to comment #31)
> Created an attachment (id=165611) [details]
> =patch with change log
> 
> We still need buildfix on Mac, and should check it on Windows.

This patch is also required now to built qtwebkit for QNX, as there JIT is not enabled and thus LLInt is enabled. The build will otherwise fail at linking due to missing references to LLInt methods.

The patch above is good as-is, the changed hunk in Platform.h does not need to include QNX as LLint is automatically enabled when JIT is disabled.
Comment 36 Simon Hausmann 2012-09-25 08:08:27 PDT
(In reply to comment #33)
> (From update of attachment 165611 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=165611&action=review
> 
> > Source/WTF/wtf/Platform.h:914
> > +    && (PLATFORM(MAC) || PLATFORM(IOS) || PLATFORM(GTK) || PLATFORM(QT)) \
> 
> conservative solution: enable it for Linux and QNX only: PLATFORM(QT) && ( OS(LINUX) || OS(QNX) )
> But the best solution would be if we can enable it for PLATFORM(QT)

Sounds like a good start, along with bugs to enable it on Mac and Windows that should block the Qt 5 meta bug.
Comment 37 Csaba Osztrogonác 2012-09-25 11:53:51 PDT
Mac build problem is tracked in https://bugs.webkit.org/show_bug.cgi?id=97587
Comment 38 Csaba Osztrogonác 2012-09-26 10:08:27 PDT
Created attachment 165824 [details]
updated patch

Add linux-* guards not to break Mac and Windows builds
Comment 39 Csaba Osztrogonác 2012-09-26 10:09:07 PDT
Comment on attachment 165824 [details]
updated patch

CQ- for now, because I'm not sure if we won't break the ARM build.
Comment 40 Csaba Osztrogonác 2012-09-26 10:11:50 PDT
Created attachment 165825 [details]
updated patch

add OS(LINUX) guard to platform.h too
Comment 41 Csaba Osztrogonác 2012-09-27 06:31:28 PDT
Landed in https://trac.webkit.org/changeset/129760 with guards for ARM.