-
Notifications
You must be signed in to change notification settings - Fork 12.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
regression: locally-built Clang can't find C++ stdlib headers on Mac #45225
Comments
This is a problem that I wish Apple would address, but I don't think it's a regression from Clang 10 so I don't think we can block the release on it. |
Jens Jorgensen's patch no longer applies, due to this (pretty sure): commit a3a2431
|
As explained in https://reviews.llvm.org/D88984, we would like to transition the libc++ headers from the Xcode toolchain to the SDK. This means that as long as Clang's sysroot is set correctly to the SDK, it should find the libc++ headers as expected. In particular, this means that running I know this doesn't solve the issue right now, however we're working on fixing this. |
Thanks Louis for your reply! I'm just a lowly user, not a clang hacker, so I don't fully grok your comment, but...
I believe I am specifying -DDEFAULT_SYSROOT when I build clang. My exact command is: cmake -Wno-dev -DCMAKE_BUILD_TYPE=RelWithDebInfo -DLLVM_ENABLE_ASSERTIONS=ON -DDEFAULT_SYSROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -DLLVM_ENABLE_PROJECTS='clang;clang-tools-extra;compiler-rt;libunwind' -DLLVM_CREATE_XCODE_TOOLCHAIN=ON -DLLVM_BUILD_TOOLS=ON -DCMAKE_INSTALL_PREFIX=~/llvm/llvm-install ../llvm the result is a clang executable that can't build a hello world: $ clang -fsyntax-only ~/test.cxx |
Right, so what I was trying to say is that this very incantation doesn't work today because we don't ship the libc++ headers as part of the SDK, however once we do, it should work out of the box. |
"the SDK" being the macOS SDK that's part of Xcode? If I understand correctly, 'once we ship the libc++ headers as part of the SDK' basically implies/requires a new version of Xcode, right? As new Xcodes never support old versions of macOS, doesn't this mean this would never be fixed for older macOS? I've just downloaded the official clang 11 binary here: and it somehow finds compiling the same file: $ /Users/sean/Downloads/clang+llvm-11.0.0-x86_64-apple-darwin/bin/clang-11 -fsyntax-only ~/Desktop/test.cxx How come that works? How can I build clang to also so work? Thanks. |
As it is often the case, the fix will require updating to a SDK that contains
I'm not sure how that Clang is configured. I think I remember there's a trick |
I feel like I'm missing something here... As of today, and until this new macOS SDK ships, how does anyone use clang on macOS (for C++)??? Aren't there clang developers that use macOS regularly? How do they build a working compiler?? |
You can build libc++ alongside Clang, and Clang should find the libc++ headers. Are you including libcxx in LLVM_ENABLE_PROJECTS? |
No. Wouldn't that build a bleeding-edge libcxx instead of using the system one? Also, the string "libcxx" does not even appear here: Anyway, I've given it a try, I modified step 3 to do: cmake -DLLVM_ENABLE_PROJECTS="clang;libcxx;libcxxabi" ../llvm The resulting clang is able to compile trivialities like "#include " but when I use it to build CMake, it fails with weird errors like: error: no member named 'aligned_alloc' in the global namespace error: no member named 'filesystem' in namespace 'std'; did you mean simply 'filesystem'? I never had weird errors like that using Jens' patch. |
I haven't tried to build recently but catching up now on this. Per Sean's comment about the official binary he downloaded which works: where can we find the build script that produces those official binaries? If those work I for one would be happy to just rely on them. |
The official binaries are built by this script: https://github.com/llvm/llvm-project/blob/llvmorg-11.0.0/llvm/utils/release/test-release.sh But there's no magic in there which makes them different from normal builds, really. |
So I just merged my working copy with the main branch. I did have to massage my commit. And what I did was expand the 2 cases to 3 and only use the platform SDK when the first two don't succeed. I'll note in particular that when my patch is disabled it of course doesn't find the std library headers, and with it, everything works as expected. Please find the patch attached. |
fix to search for standard library headers in the platform sdk automatically cmake -DLLVM_ENABLE_PROJECTS=clang -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release DCLANG_XCODE_TOOLCHAIN_ROOT=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain This allows me to access the standard library headers without need -sysroot |
So unless I'm mistaken the problem arose because Apple made the decision to no longer install the c++ std library headers included with the SDK into /usr/include. I would speculate that based on their trend of locking out changes to certain parts of the filesystem that they are unlikely to reverse this decision. |
Jens, I've only just got around to trying this now. But your new patch doesn't work for me. I've tried on two different Macs. Here are my exact steps: cd ~ Then build trivial C++ file: ~/clang-test/llvm-project/install/bin/clang++ ~/Desktop/test.cxx In file included from /Users/builder/Desktop/test.cxx:1: Am I doing anything wrong here? |
So this looks like the patch worked, but there's a different issue. I don't recall exactly but I think you just need to maybe run 'xcode-select --install" ? Something along those lines, google around a bit and I think you'll find the right incantation. |
Your patch definitely does something, because the resulting error message changes slightly wrt the paths. I'll try to find the incantation, but shouldn't the instructions at: https://clang.llvm.org/get_started.html be able to produce a compiler that can build 'hello world'? Am I trying to do something so weird here? |
Yeah the instructions certainly should, but bear in mind the way macos supplies their system headers/libraries is different and certainly non-standard w.r.t *nix norms. That's really the root issue behind all of this. If stuff were just there in /usr/include we wouldn't be here in the first place ;-). |
So I've downloaded the pre-built 13.0.0-rc1 binary from here: and it too can't build a 'hello world' example: $ /Users/builder/Downloads/clang+llvm-13.0.0-rc1-x86_64-apple-darwin/bin/clang -fsyntax-only test.c |
Sean, do you have an Apple SDK installed? If so, you need to point Clang to those headers using one of the options below, otherwise Clang can't possibly guess where to use those headers from. Apple made the decision not to ship headers as part of the root system (which makes sense IMO since those are not necessary to run any software, so there is no reason for non-developer users to have those headers), and that means you need to tell Clang where to find them. The only thing we could do here is somehow try to guess what SDK you want to compile with, and set the This issue was originally for C++, however that part of the issue was resolved. By default the libc++ headers shipped in /../include/c++/v1 will be used, but you still need to use xcrun so it can find the C library headers: $ echo "#include " | xcrun /path/to/clang++ -fsyntax-only -xc++ - The $ echo "#include " | /path/to/clang++ -fsyntax-only -xc++ - -isysroot $(xcrun --show-sdk-path) Another alternative, which might be what you're looking for since you seem to want to use $ export SDKROOT=$(xcrun --show-sdk-path) However, in all cases, that has to be performed on the user side because we can't know what the path of the SDK will be when we build Clang. TL;DR: I suspect what you want is to add |
Thanks for the comprehensive answer, Louis! Since people keep running into this, maybe some of the information should be added to the Clang user's manual?
Is there any reason the Clang driver couldn't just run |
As I said above, it would be possible, but we'd have to guess what SDK the user wants to use. There are multiple SDKs, for example the macOS SDK, the watchOS SDK, the iOS SDK, etc. One thing we could do is set the default to the result of running |
Louis, thanks for the detailed reply.
Yes; I have Xcode installed.
Why can't it guess? The Xcode clang seems to guess just fine. Contrast:
If |
The main difference is that when you invoke Apple Clang (e.g. Regardless, I went ahead and tried implementing it in Clang: https://reviews.llvm.org/D109460. This patch does solve your problem but it creates other problems as well. I opened that review to see whether this was worth pursuing or not. Reopening until we get to the bottom of this. |
Ah! Thanks for the education. It is a strange user experience that every other UNIX and also Apple's /usr/bin/clang shim work one way, but open source clang works another way. Although I finally understand why, from a purely user point of view, it's still weird. Hopefully your draft patch can be made to work magic! |
Relatedly: I believe this lack of automatically finding headers is what causes CMake's BootstrapTest to fail, see also: |
The latest instance of the patch is https://reviews.llvm.org/D136315, but it seems blocked. |
@llvm/issue-subscribers-clang-driver |
Hey guys. Just wanted a heads up on that xcrun command. Will it break my system ?
By the way , I wanted to ask if removing the xcode will help or not? |
This is a scaled down version of https://reviews.llvm.org/D136315. The intent is largely the same as before[^1], but I've scaled down the scope to try to avoid the issues that the previous patch caused: - the changes are now opt-in based on enabling `CLANG_USE_XCSELECT` - this only works when targeting macOS on a macOS host (this is the only case supported by `libxcselect`[^2]) We also introduce an environment variable `CLANG_NO_XCSELECT` that disables this behaviour if Clang is configured with `CLANG_USE_XCSELECT=ON`. This is needed to avoid breaking tests. Another reason to leave this as opt-in for now is that there are some bugs in libxcselect that need fixing before it is safe to use by default for all users. This has been reported to Apple as FB16081077. [^1]: See also https://reviews.llvm.org/D109460 and llvm#45225. [^2]: https://developer.apple.com/documentation/xcselect?language=objc
Extended Description
On macOS with Xcode:
Expected:
Actual:
This used to work. I'm not sure when it broke, I think it was in the svn to git switch.
It's been discussed on cfe-dev, I'm not alone in having this happen:
http://lists.llvm.org/pipermail/cfe-dev/2019-September/063394.html
List member Jens Jorgensen shared a patch, see attached. It works for me, but I have no idea if it's fit to commit.
The text was updated successfully, but these errors were encountered: