Bug 228267 - resolve-asm-file-conflicts.rb build failure after upgrade to CMake 3.21.0; DWARF 5 incompatibility
Summary: resolve-asm-file-conflicts.rb build failure after upgrade to CMake 3.21.0; DW...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: JavaScriptCore (show other bugs)
Version: WebKit Nightly Build
Hardware: PC Linux
: P2 Normal
Assignee: Angelos Oikonomopoulos
URL:
Keywords: InRadar
: 229491 (view as bug list)
Depends on:
Blocks:
 
Reported: 2021-07-24 09:48 PDT by Adrian Vovk
Modified: 2021-09-08 13:36 PDT (History)
11 users (show)

See Also:


Attachments
BuildStream log of the failing build (133.14 KB, text/x-log)
2021-07-24 09:48 PDT, Adrian Vovk
no flags Details
File directives in LowLevelInterpreter.cpp.pre.s (3.32 KB, text/plain)
2021-07-24 09:49 PDT, Adrian Vovk
no flags Details
File directives in LowLevelInterpreter.cpp.s (3.33 KB, text/plain)
2021-07-24 09:50 PDT, Adrian Vovk
no flags Details
The arguments passed into postprocess-asm in cmake 3.21.0 vs 3.20.5 (7.55 KB, text/plain)
2021-07-24 09:56 PDT, Adrian Vovk
no flags Details
Patch (4.23 KB, patch)
2021-07-28 05:59 PDT, Angelos Oikonomopoulos
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Adrian Vovk 2021-07-24 09:48:05 PDT
Created attachment 434170 [details]
BuildStream log of the failing build

After upgrading to CMake 3.21.0, WebKitGTK would fail to build. I've attached the full build log. Here's the important error message:

> Source/JavaScriptCore/CMakeFiles/LowLevelInterpreterLib.dir/llint/LowLevelInterpreter.cpp.s:109399: Error: file table slot 1 is already occupied by a different file (/buildstream/carbonOS/pkgs/webkitgtk.bst/Source/JavaScriptCore/llint/LowLevelInterpreter.cpp vs /buildstream/carbonOS/pkgs/webkitgtk.bst/_builddir//buildstream/carbonOS/pkgs/webkitgtk.bst/Source/JavaScriptCore/llint/LowLevelInterpreter.cpp)
This is a symptom of resolve-asm-file-conflict's incompatibility with DWARF 5



I followed this bug down through the stack and I think I have a pretty solid understanding of what happened to cause it:

- CMake 3.21.0 switched from relative to absolute paths for the Ninja generator [1]. This means that GCC started encoding all of its `.file` directives as absolute paths
- To comply with DWARF 5 [2], GCC was updated [3] to emit a `.file` directive with a number of 0, the path to the current working directory, and the path to the source file. The working directory is emitted even if the source file path is absolute
- resolve-asm-file-conflicts.rb concatenates the CWD and the source path together to get "/path/to/builddir//path/to/source". This causes it to misbehave
- The same script renumbers the DWARF 5 `.file 0` directive and makes it match the rest of the file directives. This leads to a situation where the generated assembly has two `.file 1` directives with the same source path; however, one has a CWD and the other does not have a CWD.
- `as` cannot handle two .file directives with the same number where one has a CWD and the other doesn't. Also, by changing the file number from 0, the script breaks the DWARF 5 format


To fix this, resolve-asm-file-conflicts.rb should detect a `.file 0` directive and ensure that it is left unchanged. To test this fix, I edited the generated LowLevelInterpreter.cpp.s to renumber the DWARF 5 directive back to 0. That allowed `as` to build the file

I've attached the .file directives of LowLevelInterpreter.cpp.pre.s and LowLevelInterpreter.cpp.s

[1]: https://gitlab.kitware.com/cmake/cmake/-/commit/c564a3e3fff52ef811291c5ba8fb07a5a1b47f97
[2]: https://sourceware.org/bugzilla/show_bug.cgi?id=25614#c9
[3]: https://github.com/gcc-mirror/gcc/commit/3a9e6ee42acf1e3d00e4391ab1b1a56bb0b32ad2
Comment 1 Adrian Vovk 2021-07-24 09:49:36 PDT
Created attachment 434171 [details]
File directives in LowLevelInterpreter.cpp.pre.s
Comment 2 Adrian Vovk 2021-07-24 09:50:13 PDT
Created attachment 434172 [details]
File directives in LowLevelInterpreter.cpp.s
Comment 3 Adrian Vovk 2021-07-24 09:56:38 PDT
Created attachment 434173 [details]
The arguments passed into postprocess-asm in cmake 3.21.0 vs 3.20.5

I put this here just in case it's useful. As you can see cmake 3.21.0 uses absolute paths and cmake 3.20.5 uses relative paths
Comment 4 Adrian Perez 2021-07-26 05:14:23 PDT
Thanks a lot for the thorough bug report and for doing the initial
troubleshooting—much appreciated.

We had not seen this ourselves probably because the Flatpak SDK we
use for developing does not always have the latest CMake version; but
we definitely need to fix this.
Comment 5 Angelos Oikonomopoulos 2021-07-27 08:48:54 PDT
Indeed, thanks for the investigation! Will work on a patch, but need to build a newer GCC first.
Comment 6 Angelos Oikonomopoulos 2021-07-28 05:59:43 PDT
Created attachment 434423 [details]
Patch
Comment 7 Angelos Oikonomopoulos 2021-07-28 06:01:44 PDT
Unfortunately, I haven't been able to reproduce the issue with cmake 3.21.0 and GCC 11.1.0 (with --debug, so the asm postprocessing was enabled).

Adrian, could you test if the patch I just uploaded fixes things for you?
Comment 8 Angelos Oikonomopoulos 2021-07-28 06:02:27 PDT
(In reply to Angelos Oikonomopoulos from comment #7)

> Adrian, could you test if the patch I just uploaded fixes things for you?

Meaning Adrian Vovk, of course.
Comment 9 Adrian Vovk 2021-07-28 10:33:33 PDT
Thank you for the patch, Angelos! This patch seems to fix the problem.
Comment 10 Michael Catanzaro 2021-08-30 08:19:29 PDT
*** Bug 229491 has been marked as a duplicate of this bug. ***
Comment 11 EWS 2021-08-30 08:55:40 PDT
Committed r281758 (241100@main): <https://commits.webkit.org/241100@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 434423 [details].
Comment 12 Radar WebKit Bug Importer 2021-08-30 08:56:31 PDT
<rdar://problem/82530292>
Comment 13 Michael Catanzaro 2021-09-03 13:50:03 PDT
(In reply to Adrian Vovk from comment #0)
> > Source/JavaScriptCore/CMakeFiles/LowLevelInterpreterLib.dir/llint/LowLevelInterpreter.cpp.s:109399: Error: file table slot 1 is already occupied by a different file (/buildstream/carbonOS/pkgs/webkitgtk.bst/Source/JavaScriptCore/llint/LowLevelInterpreter.cpp vs /buildstream/carbonOS/pkgs/webkitgtk.bst/_builddir//buildstream/carbonOS/pkgs/webkitgtk.bst/Source/JavaScriptCore/llint/LowLevelInterpreter.cpp)
> This is a symptom of resolve-asm-file-conflict's incompatibility with DWARF 5

I'm still seeing this exact same error even though this has landed:

/tmp/ccHybVZr.s: Assembler messages:
/tmp/ccHybVZr.s:23: Error: file table slot 1 is already occupied by a different file (../../Source/JavaScriptCore/llint/LowLevelInterpreter.cpp vs /home/mcatanzaro/Projects/WebKit/Source/JavaScriptCore/llint/LowLevelInterpreter.asm)
/tmp/ccHybVZr.s:103670: Error: file table slot 2 is already occupied by a different file (/home/mcatanzaro/Projects/WebKit/WebKitBuild/GNOME/JavaScriptCore/DerivedSources/InitBytecodes.asm vs ../../Source/JavaScriptCore/heap/SlotVisitorInlines.h)
/tmp/ccHybVZr.s:103689: Error: file table slot 3 is already occupied by a different file (/home/mcatanzaro/Projects/WebKit/Source/JavaScriptCore/llint/LowLevelInterpreter64.asm vs WTF/Headers/wtf/Atomics.h)
/tmp/ccHybVZr.s:103697: Error: file table slot 4 is already occupied by a different file (/home/mcatanzaro/Projects/WebKit/Source/JavaScriptCore/llint/LowLevelInterpreter32_64.asm vs ../../Source/JavaScriptCore/heap/HeapCellInlines.h)
/tmp/ccHybVZr.s:103702: Error: file table slot 5 is already occupied by a different file (/home/mcatanzaro/Projects/WebKit/WebKitBuild/GNOME/JavaScriptCore/DerivedSources/InitWasm.asm vs ../../Source/JavaScriptCore/heap/PreciseAllocation.h)
/tmp/ccHybVZr.s:103728: Error: file table slot 6 is already occupied by a different file (/home/mcatanzaro/Projects/WebKit/Source/JavaScriptCore/llint/WebAssembly.asm vs ../../Source/JavaScriptCore/heap/MarkedBlock.h)

Reopening.
Comment 14 Michael Catanzaro 2021-09-03 14:32:08 PDT
I've disabled this feature for now via bug #229893. Please remember to revert that when fixing this bug!
Comment 15 Angelos Oikonomopoulos 2021-09-07 09:57:12 PDT
(In reply to Michael Catanzaro from comment #13)
> (In reply to Adrian Vovk from comment #0)
> > > Source/JavaScriptCore/CMakeFiles/LowLevelInterpreterLib.dir/llint/LowLevelInterpreter.cpp.s:109399: Error: file table slot 1 is already occupied by a different file (/buildstream/carbonOS/pkgs/webkitgtk.bst/Source/JavaScriptCore/llint/LowLevelInterpreter.cpp vs /buildstream/carbonOS/pkgs/webkitgtk.bst/_builddir//buildstream/carbonOS/pkgs/webkitgtk.bst/Source/JavaScriptCore/llint/LowLevelInterpreter.cpp)
> > This is a symptom of resolve-asm-file-conflict's incompatibility with DWARF 5
> 
> I'm still seeing this exact same error even though this has landed:
> 
> /tmp/ccHybVZr.s: Assembler messages:
> /tmp/ccHybVZr.s:23: Error: file table slot 1 is already occupied by a
> different file (../../Source/JavaScriptCore/llint/LowLevelInterpreter.cpp vs
> /home/mcatanzaro/Projects/WebKit/Source/JavaScriptCore/llint/
> LowLevelInterpreter.asm)
> /tmp/ccHybVZr.s:103670: Error: file table slot 2 is already occupied by a
> different file
> (/home/mcatanzaro/Projects/WebKit/WebKitBuild/GNOME/JavaScriptCore/
> DerivedSources/InitBytecodes.asm vs
> ../../Source/JavaScriptCore/heap/SlotVisitorInlines.h)
> /tmp/ccHybVZr.s:103689: Error: file table slot 3 is already occupied by a
> different file
> (/home/mcatanzaro/Projects/WebKit/Source/JavaScriptCore/llint/
> LowLevelInterpreter64.asm vs WTF/Headers/wtf/Atomics.h)
> /tmp/ccHybVZr.s:103697: Error: file table slot 4 is already occupied by a
> different file
> (/home/mcatanzaro/Projects/WebKit/Source/JavaScriptCore/llint/
> LowLevelInterpreter32_64.asm vs
> ../../Source/JavaScriptCore/heap/HeapCellInlines.h)
> /tmp/ccHybVZr.s:103702: Error: file table slot 5 is already occupied by a
> different file
> (/home/mcatanzaro/Projects/WebKit/WebKitBuild/GNOME/JavaScriptCore/
> DerivedSources/InitWasm.asm vs
> ../../Source/JavaScriptCore/heap/PreciseAllocation.h)
> /tmp/ccHybVZr.s:103728: Error: file table slot 6 is already occupied by a
> different file
> (/home/mcatanzaro/Projects/WebKit/Source/JavaScriptCore/llint/WebAssembly.
> asm vs ../../Source/JavaScriptCore/heap/MarkedBlock.h)
> 
> Reopening.

Ouch. Any hints on where I should try to reproduce this?
Comment 16 Michael Catanzaro 2021-09-07 14:02:34 PDT
(In reply to Angelos Oikonomopoulos from comment #15)
> Ouch. Any hints on where I should try to reproduce this?

Try building in an F34 toolbox?
Comment 17 Angelos Oikonomopoulos 2021-09-08 10:42:44 PDT
(In reply to Michael Catanzaro from comment #16)
> (In reply to Angelos Oikonomopoulos from comment #15)
> > Ouch. Any hints on where I should try to reproduce this?
> 
> Try building in an F34 toolbox?

Thanks. Unfortunately, building with

./Tools/Scripts/build-jsc --jsc-only --debug --cmakeargs=-DGCC_OFFLINEASM_SOURCE_MAP=ON

(which gives the -- Enabling asm postprocessing message) fails to reproduce the issue in the official fedora:34 docker image. For reference:

-- The CXX compiler identification is GNU 11.2.1

And gcc -v reports: gcc version 11.2.1 20210728 (Red Hat 11.2.1-1) (GCC).

If there's anything special about your setup that I'm missing, any tips would be welcome.
Comment 18 Michael Catanzaro 2021-09-08 13:36:42 PDT
(In reply to Angelos Oikonomopoulos from comment #17)
> If there's anything special about your setup that I'm missing, any tips
> would be welcome.

Well I'll be, seems it's only broken when built with -flto=auto. Let me close this and open a new bug.