Environment: VS 2019 + Windows Server 2016 Issue description: The MSVC team uses Webkit project as a test case to detect compiler regressions. Recently, two Webkit ports WinCairo and AppleWin triggered an old lambda processor issue on MSVC, which is an irreconcilable old lambda processor issue. Therefore, it is recommended to turn on "/Zc:lambda" switch to use new lambda processor to avoid this problem. Some opinions from MSVC developers: Note: the Webkit failure is an issue with the old lambda processor in which one or more lambdas (each of which contains one or more nested lambdas) are generated from a macro expansion. Trying to tell the lambdas apart is an exercise in futility (in the old, token based lambda processor the ‘identity’ for a lambda is the SFA of the initial ‘[‘) and code like this was one of the motivating reasons for starting on the lambda rejuvenation effort. Repro Steps: 1. git clone https://github.com/WebKit/webkit F:\gitP\WebKit\webkit 2. open a VS 2019 x64 command prompt as admin and browse to F:\gitP\WebKit\webkit 3. download latest WebKitAuxiliaryLibrary.zip from https://developer.apple.com/opensource/internet/WebKitAuxiliaryLibrary.zip and unzip to F:\gitP\WebKit\webkit\WebKitLibraries\win 4. download latest WebKitSupportLibrary.zip from https://developer.apple.com/opensource/internet/webkit_sptlib_agree.html and unzip to F:\gitP\WebKit\webkit\WebKitLibraries\win 5. mkdir F:\gitP\WebKit\webkit\build_amd64_AppleWin 6. cd F:\gitP\WebKit\webkit\build_amd64_AppleWin 7. cmake -G "Visual Studio 16 2019" -A x64 -DCMAKE_SYSTEM_VERSION=10.0.18362.0 -DCMAKE_BUILD_TYPE=Release -DRUBY_LIBRARY=C:\tools\ruby26\lib -DPORT="AppleWin" .. 8. msbuild /p:Platform=x64 /p:Configuration=Release WebKit.sln /t:Rebuild Error Info: F:\gitP\WebKit\webkit\Source\JavaScriptCore\heap\Heap.cpp(2827): Assertion failed: iter->second.EncodedIndex() != encodedIndex, file d:\agent\_work\4\s\src\vctools\Compiler\CxxFE\sl\p1\c\lambdas.cpp, line 2464 [when compiling F:\gitP\WebKit\webkit\build_amd64_AppleWin\DerivedSources\JavaScriptCore\unified-sources\UnifiedSource-ee8a7a7a-3.cpp] c1xx!assertfe()+0x106 d:\agent\_work\4\s\src\vctools\Compiler\CxxFE\sl\p1\c\debug.c Line 333 c1xx!LambdaProcessor::Process()+0x8c3 d:\agent\_work\4\s\src\vctools\Compiler\CxxFE\sl\p1\c\lambdas.cpp Line 2462 c1xx!PrimaryParser::Parse()+0xd9ae d:\agent\_work\4\s\src\vctools\Compiler\CxxFE\sl\p1\cxx\grammar.y Line 11319 c1xx!`CallPrimaryParser'::`2'::<lambda_1>::operator()()+0x19 d:\agent\_work\4\s\src\vctools\Compiler\CxxFE\sl\p1\cxx\grammar.y Line 13808 c1xx!CallPrimaryParser()+0x14e d:\agent\_work\4\s\src\vctools\Compiler\CxxFE\sl\p1\cxx\grammar.y Line 13829 c1xx!main_compile()+0xfa2 d:\agent\_work\4\s\src\vctools\Compiler\CxxFE\sl\p1\c\main.c Line 3094
/Zc:lambda seems an undocumented switch. Which MSVC version started to support it? IIRC, AppleWin is still using Visual Studio 2019 version 16.3
See also: Bug 209311 – [WinCairo][MSVC] NetworkProcess.cpp(1896): fatal error C1001: Internal compiler error with Visual Studio 16.5.0
Thanks for providing more information, Fujii. >>/Zc:lambda seems an undocumented switch. Which MSVC version started to support it? This does not seem to have official documents at present. I can find that the earliest VS version that contains the modification(Some people apparently don't like the continued use of 'experimental', so the team removed "/experimental:newLambdaProcessor" and added "/Zc:lambda" switch) is VS 16.4 Preview2, and the official version may be VS 16.5 and later. >>See also: Bug 209311 – [WinCairo][MSVC] NetworkProcess.cpp(1896): fatal error C1001: Internal compiler error with Visual Studio 16.5.0 Check the datails of dup bug https://bugs.webkit.org/show_bug.cgi?id=209358, the internal error occurred on VS 16.5.0, which should be caused by the above change. So now it may be necessary to add the new lambda processor back. >>IIRC, AppleWin is still using Visual Studio 2019 version 16.3 On my side, AppleWin and WinCairo are currently compiled successfully with "/Zc:lambda". I tested it on VS 16.9.1 and the latest version.
Created attachment 429692 [details] Patch
Isn't VS 2022 releasing soon? Is the new lambda processor on by default for it?
WinCairo EWS is reporting an compilation error for the patch. https://ews-build.webkit.org/#/builders/12/builds/53128 FAILED: Source/JavaScriptCore/CMakeFiles/JavaScriptCore.dir/__/__/JavaScriptCore/DerivedSources/unified-sources/UnifiedSource-ee8a7a7a-3.cpp.obj C:\MSVS\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\cl.exe /nologo /TP -DBUILDING_JavaScriptCore -DBUILDING_WITH_CMAKE=1 -DHAVE_CONFIG_H=1 -DJavaScriptCore_EXPORTS -DNOCRYPT -DNOMINMAX -DUNICODE -DWINVER=0x601 -DWTF_PLATFORM_WIN_CAIRO=1 -D_CRT_SECURE_CPP_OVERLOAD_STANDARD_NAMES=1 -D_CRT_SECURE_NO_WARNINGS -D_HAS_EXCEPTIONS=0 -D_UNICODE -D_WIN32_WINNT=0x601 -D_WINDOWS -D_WINSOCKAPI_="" -IJavaScriptCore\Headers -I. -I..\..\Source\JavaScriptCore -I..\..\Source\JavaScriptCore\API -I..\..\Source\JavaScriptCore\assembler -I..\..\Source\JavaScriptCore\b3 -I..\..\Source\JavaScriptCore\b3\air -I..\..\Source\JavaScriptCore\bindings -I..\..\Source\JavaScriptCore\builtins -I..\..\Source\JavaScriptCore\bytecode -I..\..\Source\JavaScriptCore\bytecompiler -I..\..\Source\JavaScriptCore\dfg -I..\..\Source\JavaScriptCore\disassembler -I..\..\Source\JavaScriptCore\disassembler\ARM64 -I..\..\Source\JavaScriptCore\disassembler\udis86 -I..\..\Source\JavaScriptCore\domjit -I..\..\Source\JavaScriptCore\ftl -I..\..\Source\JavaScriptCore\heap -I..\..\Source\JavaScriptCore\debugger -I..\..\Source\JavaScriptCore\inspector -I..\..\Source\JavaScriptCore\inspector\agents -I..\..\Source\JavaScriptCore\inspector\augmentable -I..\..\Source\JavaScriptCore\inspector\remote -I..\..\Source\JavaScriptCore\interpreter -I..\..\Source\JavaScriptCore\jit -I..\..\Source\JavaScriptCore\llint -I..\..\Source\JavaScriptCore\parser -I..\..\Source\JavaScriptCore\profiler -I..\..\Source\JavaScriptCore\runtime -I..\..\Source\JavaScriptCore\tools -I..\..\Source\JavaScriptCore\wasm -I..\..\Source\JavaScriptCore\wasm\js -I..\..\Source\JavaScriptCore\yarr -IJavaScriptCore\DerivedSources -IJavaScriptCore\DerivedSources\inspector -IJavaScriptCore\DerivedSources\runtime -IJavaScriptCore\DerivedSources\yarr -I..\include\private -I..\..\Source\JavaScriptCore\inspector\remote\socket -IWTF\Headers -I..\..\WebKitLibraries\win\include /W4 /DWIN32 /D_WINDOWS /GR- /EHs- /EHc- /MD /O2 /Ob2 /DNDEBUG /wd4018 /wd4060 /wd4068 /wd4100 /wd4127 /wd4146 /wd4189 /wd4201 /wd4244 /wd4245 /wd4251 /wd4275 /wd4267 /wd4305 /wd4309 /wd4312 /wd4324 /wd4389 /wd4456 /wd4457 /wd4458 /wd4459 /wd4505 /wd4611 /wd4646 /wd4701 /wd4702 /wd4706 /wd4715 /wd4722 /wd4838 /wd4840 /wd4996 /wd5205 /Zi /GS /EHa- /EHc- /EHs- /fp:except- /analyze- /bigobj /utf-8 /validate-charset /Oy- /Zc:lambda -std:c++17 /showIncludes /FoSource\JavaScriptCore\CMakeFiles\JavaScriptCore.dir\__\__\JavaScriptCore\DerivedSources\unified-sources\UnifiedSource-ee8a7a7a-3.cpp.obj /FdSource\JavaScriptCore\CMakeFiles\JavaScriptCore.dir\ /FS -c JavaScriptCore\DerivedSources\unified-sources\UnifiedSource-ee8a7a7a-3.cpp C:\Buildbot\WinCairo-EWS\build\Source\JavaScriptCore\heap/Heap.cpp(2874): error C2065: 'Visitor': undeclared identifier C:\Buildbot\WinCairo-EWS\build\Source\JavaScriptCore\heap/Heap.cpp(2901): note: see reference to function template instantiation 'auto JSC::Heap::addCoreConstraints::<lambda_12>::()::<lambda_2>::operator ()<JSC::IsoCellSet>(_T1 &) const' being compiled with [ _T1=JSC::IsoCellSet ] C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/Function.h(52): note: see reference to function template instantiation 'auto JSC::Heap::addCoreConstraints::<lambda_12>::operator ()<JSC::SlotVisitor>(_T1 &) const' being compiled with [ _T1=JSC::SlotVisitor ] C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/Function.h(52): note: while compiling class template member function 'Out WTF::Detail::CallableWrapper<CallableType,Out,JSC::SlotVisitor &>::call(JSC::SlotVisitor &)' with [ Out=void, CallableType=JSC::Heap::addCoreConstraints::<lambda_12> ] C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/StdLibExtras.h(505): note: see reference to class template instantiation 'WTF::Detail::CallableWrapper<CallableType,Out,JSC::SlotVisitor &>' being compiled with [ CallableType=JSC::Heap::addCoreConstraints::<lambda_12>, Out=void ] C:\Buildbot\WinCairo-EWS\build\WebKitBuild\Release\WTF\Headers\wtf/Function.h(74): note: see reference to function template instantiation 'std::unique_ptr<T,std::default_delete<T>> WTF::makeUnique<WTF::Detail::CallableWrapper<CallableType,Out,JSC::SlotVisitor &>,_Ty>(_Ty &&)' being compiled with [ T=WTF::Detail::CallableWrapper<JSC::Heap::addCoreConstraints::<lambda_12>,void,JSC::SlotVisitor &>, CallableType=JSC::Heap::addCoreConstraints::<lambda_12>, Out=void, _Ty=JSC::Heap::addCoreConstraints::<lambda_12> ] C:\Buildbot\WinCairo-EWS\build\Source\JavaScriptCore\heap/Heap.cpp(2901): note: see reference to function template instantiation 'WTF::Function<void (JSC::SlotVisitor &)>::Function<JSC::Heap::addCoreConstraints::<lambda_12>,void>(CallableType &&)' being compiled with [ CallableType=JSC::Heap::addCoreConstraints::<lambda_12> ] C:\Buildbot\WinCairo-EWS\build\Source\JavaScriptCore\heap/Heap.cpp(2872): note: see reference to function template instantiation 'WTF::Function<void (JSC::SlotVisitor &)>::Function<JSC::Heap::addCoreConstraints::<lambda_12>,void>(CallableType &&)' being compiled with [ CallableType=JSC::Heap::addCoreConstraints::<lambda_12> ] C:\Buildbot\WinCairo-EWS\build\Source\JavaScriptCore\heap/Heap.cpp(2874): error C2672: 'JSC::IsoCellSet::forEachMarkedCellInParallel': no matching overloaded function found C:\Buildbot\WinCairo-EWS\build\Source\JavaScriptCore\heap/Heap.cpp(2901): error C2974: 'JSC::IsoCellSet::forEachMarkedCellInParallel': invalid template argument for 'Visitor', type expected C:\Buildbot\WinCairo-EWS\build\Source\JavaScriptCore\heap\IsoCellSet.h(62): note: see declaration of 'JSC::IsoCellSet::forEachMarkedCellInParallel' C:\Buildbot\WinCairo-EWS\build\Source\JavaScriptCore\heap/Heap.cpp(2874): error C2672: 'JSC::Subspace::forEachMarkedCellInParallel': no matching overloaded function found C:\Buildbot\WinCairo-EWS\build\Source\JavaScriptCore\heap/Heap.cpp(2901): error C2974: 'JSC::Subspace::forEachMarkedCellInParallel': invalid template argument for 'Visitor', type expected C:\Buildbot\WinCairo-EWS\build\Source\JavaScriptCore\heap\Subspace.h(90): note: see declaration of 'JSC::Subspace::forEachMarkedCellInParallel'
I don't see the error on my PC. Is this caused by specific MSVC version. I'm using MSVC 14.29 (Visual Studio 2019 Version 16.9). WinCairo EWS seems using MSVC 14.28 (Visual Studio 2019 Version 16.8).
Hmm, for error C2065,I found relevant information, it is indeed a bug of MSVC, currently fixed in 16.9. You may need to upgrade VS. https://developercommunity.visualstudio.com/t/captured-variable-not-recognized-when-calling-temp/1137293 https://developercommunity.visualstudio.com/t/nested-c20-template-lambdas-two-issues-with-captur/1150322 I don’t know much about VS2022, there is a related blog https://devblogs.microsoft.com/visualstudio/visual-studio-2022/.
(In reply to QuellaZhang from comment #8) > Hmm, for error C2065,I found relevant information, it is indeed a bug of > MSVC, currently fixed in 16.9. You may need to upgrade VS. > > https://developercommunity.visualstudio.com/t/captured-variable-not- > recognized-when-calling-temp/1137293 > https://developercommunity.visualstudio.com/t/nested-c20-template-lambdas- > two-issues-with-captur/1150322 > > I don’t know much about VS2022, there is a related blog > https://devblogs.microsoft.com/visualstudio/visual-studio-2022/. Fujii's patch adds a compiler flag based on versioning I'm just curious if that flag will still be valid with 2022. If not then there'd need to be an upper bound.
Hey Don, Confirmed with the developer: >>Isn't VS 2022 releasing soon? Is the new lambda processor on by default for it? Not: at the moment: it is the default if you compile with /std:c++latest or /permissive- -- it may become the default in VS2022 but we haven't made that decision yet.
<rdar://problem/78607135>
(In reply to QuellaZhang from comment #10) > Hey Don, > > Confirmed with the developer: > > >>Isn't VS 2022 releasing soon? Is the new lambda processor on by default for it? > > Not: at the moment: it is the default if you compile with /std:c++latest or > /permissive- -- it may become the default in VS2022 but we haven't made that > decision yet. Thanks for letting us know! Fujii I think its probably better to start compiling with /permissive- rather than this. Thoughts?
(In reply to Don Olmstead from comment #12) > (In reply to QuellaZhang from comment #10) > > Hey Don, > > > > Confirmed with the developer: > > > > >>Isn't VS 2022 releasing soon? Is the new lambda processor on by default for it? > > > > Not: at the moment: it is the default if you compile with /std:c++latest or > > /permissive- -- it may become the default in VS2022 but we haven't made that > > decision yet. > > Thanks for letting us know! > > Fujii I think its probably better to start compiling with /permissive- > rather than this. Thoughts? > the first public preview of Visual Studio 2022 will be released this summer. https://devblogs.microsoft.com/visualstudio/visual-studio-2022/ QuellaZhang, when do you want to solve the original problem (comment#0)? I think waiting for the official VS2022 release doesn't meat the QuellaZhang's schedule.
meet, not 🐷.
Hi Fujii, We added "/Zc:lambda" to command line as workaround to unblock us. Please follow your schedule to fix this issue. :)
Bug 230599 can be solved by the /Zc:lambda switch.
Created attachment 439085 [details] Patch
Comment on attachment 439085 [details] Patch Seems fine to me.
Comment on attachment 439085 [details] Patch Clearing flags on attachment: 439085 Committed r283007 (242072@main): <https://commits.webkit.org/242072@main>
All reviewed patches have been landed. Closing bug.
Reopening to attach new patch.
Created attachment 439568 [details] Wrong Patch
This patch modifies the JS builtins code generator. Please ensure that you have rebaselined any generator test results (i.e., by running `Tools/Scripts/run-builtins-generator-tests --reset-results`)
Hi QuellaZhang, After this change, we have another problem. Could you take a look? Bug 230877 – [Win] WebCore.dir\Debug\WebCore.lib : fatal error LNK1248: image size (1000309F0) exceeds maximum allowable size (FFFFFFFF)
Yes, the AppleWin port failed with one known bug: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1490389/
(In reply to April from comment #25) > Yes, the AppleWin port failed with one known bug: > https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1490389/ I can't see the page. > 401 - Uh-oh, you do not have access. If you have a problem, please create a new ticket.