LLVM cannot be built as dynamic libraries #3573
Comments
|
Have you tried the CMake build generator? |
Superficially, yes. I hope to spend more time with it once I finish this tiny project :) Actually, I have a few fixes for it, somewhere… |
|
I've uploaded a full, compressed patch of all changes in the ‘danchr/build’ branch in my Mercurial repository, relative to the LLVM Subversion trunk, revision 60875. Also uploaded is a partial patch with irrelevant or machine-generated changes removed. It's still very large; as you can see from the diffstat summary at the end of this post. I've tested my modifications on 32-bit Red Hat Enterprise Linux 5.2, and on a G4 running Mac OS X 10.5. On RHEL, the only test suite failures I get seem to be caused by using LLVM GCC 2.4 and brokenness in the dummy Hello module. (The latter only during static builds, ironically.) I have also adapted Clang, PoolAlloc and (part of) VMKit to the modified build system. On my G4, dyld reports the launch time of clang — linked to 18 LLVM libraries and needing 10,331 ‘binding fixups’ — as circa 90ms at idle. The binary footprint of LLVM is decreased significantly. (Somehow, static builds have decreased in size too; this may be caused by the updated Libtool.) I've attached a file containing measurements of the binary sizes. What remains is mainly testing and documentation. Oh, and I need to figure out how to get this into the main LLVM repository too… I'll go into detail with what I've done, and why later, but I believe my modifications can be described as a general overhaul of the LLVM Makefile build system. I've touched almost all parts of it, I believe. Suggestions are more than welcome, as are test results :) -- |
|
Screenshot of build output |
|
Example patch containing changes to some critical files
As such, it should be a good example of the approach I took. |
|
Any progress with this? |
|
Independently of Dan's work, LLVM now builds PIC by default and we've removed libtool from the makefiles (but not autoconf). |
I've continued to regularly update and merge my Mercurial mirror & branch, but other than that, I haven't done much work on this lately, I'm afraid. My main problem is that testing build system modifications on an old G4 takes hours for each run, and proper testing requires several runs.
I've begun adapting my changes to this change, though, and even though I haven't spent much time on LLVM lately, I've thought of possible clean ups to make my changes more acceptable to the community. In the meantime, I've been hacking on HgSubversion trying to make it suitable for mirroring the LLVM repositories :) |
|
Updated patch against release_26 branch b/Makefile | 25 |
|
Screenshot of build output (updated) |
|
Having a single .so for LLVM raises a number of problems:
As long as the .so build is optional, and the .a are still built PIC I'm fine with it though. Keeping the current separation of one .a per LLVM module is good for observing code size issues (like codegen depending on asmprinter). |
I agree, but due to the both the way I do this and how the ‘lib’ directory is organised, I have no other choice. Building a dynamically linked libLLVM consists of the following steps:
The process for building Clang is similar to steps three through six. I do agree that one libLLVM.dylib is too much, but it's worth noting that calls into another library is slow. The current library granularity will cause significant slowdowns if directly translated to one library per archive. On the other hand, Clang seems to execute a bit faster in wall time on my G4, probably due to caching and so on. I don't have any recent timings on various library builds, and I actually haven't run the tests recently either; I simply don't have the hardware to do that on a regular basis…
I haven't tested whether it still works, but I know that Apple use LLVM_SUBMIT_VERSION for versioning their builds. Other users needing to distinguish arbitrary builds could do a similar trick. Most libraries have install names like this: darwin: $libdir/lib$NAME[.$MAJOR[.$MINOR[.$SUBMINOR]]].dylib
Indeed; except for libLTO, all libraries are versioned. I'd suggest putting the C bindings into a similarly unversioned library.
The default behaviour in my Mercurial branch is largely similar to that of the 2.6 release branch. The main differences are altered — and in my opinion, improved — build progress output, and one libLLVMTarget for all targets in one. I have the infrastructure in place in ‘llvm-config.in.in’ to provide hidden compatibility aliases. Oh, and I also told llvm-config about LTO and Clang :) The way it currently deal with shared builds is a bit of a hack, though… For reference, I put a listing of the results of two builds below. Both builds were my branch (based on 2.6), optimised and w/o PIC. (The PIC setting has no affect on dynamic libraries.) A static-only and a shared-only: $ export CC=gcc-4.2 CXX=g++-4.2 shared/Release/lib: static/Release/lib: |
Yes, having a separate .so for each module would cause slowdowns.
Doesn't --disable-pic work properly then? |
Yup. I plan on moving my changes from a branch to a Mercurial patch queue. Other than allowing me to split up my changes, it should also make reorganising easier. By the way, I forgot to mention something. In order to merge all the target libraries, I had to move five source files to other libraries. To prevent circular dependancies, I moved TargetData.cpp to Analysis, and TargetInstrInfo.cpp, TargetLoweringObjectFile.cpp, TargetMachine.cpp and TargetRegisterInfo.cpp to CodeGen. I shan't say whether this makes sense on a conceptual level, but the dependancies mandated it ;-)
I thought dynamic libraries had to be PIC everywhere? PIC is forced for shared libraries, and some loadable modules are skipped when building static w/o PIC. Disabling PIC for a shared build only affects non-library compilation; executables, archives and so on. |
|
Adding Chris Lattner to the Cc list; I'd appreciate his input on this. |
|
Oh, and yet another thing: I had to remove the ‘march’, ‘mcpu’ and ‘mattr’ definitions from ‘lib/ExecutionEngine/JIT/TargetSelect.cpp’. This is a consequence of linking everything into one library, as ‘llc’ also defines such options. |
|
Hi Dan, what specifically is the question? |
|
Jumbo patch against LLVM 2.6 & Clang 1.0 The patch can also be viewed on Bitbucket: Makefile | 25 +- |
The question is really, where do I go from here? I think the simplest and most independent part of the changes is moving five source files — TargetData.cpp, TargetInstrInfo.cpp, TargetLoweringObjectFile.cpp, TargetMachine.cpp & TargetRegisterInfo.cpp — from ‘lib/Target’ to either ‘lib/Analysis’ or ‘lib/CodeGen’. This allowed me to generate one single archive containing all targets. |
|
I figured it would be worth mentioning, but Unladen Swallow also has a bug for this, containing some additional discussion: http://code.google.com/p/unladen-swallow/issues/detail?id=130 I'm reassigning this to the default, as I'm no longer actively working on this. |
|
LLVM can be built as a shared library after http://llvm.org/viewvc/llvm-project?view=rev&revision=96559. I'm going to have a redhat distro guy look it over to see if I can improve anything and try to give llvm-config a flag to select the shared library before marking this as fixed. |
|
Commenting in here, but can open a new specific ticket if that's better. Although LLVM can be built with shared libraries, with ELF/PowerPC it fails to link at runtime because of non-PIC references to PPCCompilationCallbackC in PPCJITInfo.cpp. I don't know how to fix this because it's in inline asm and not a separate file. |
|
mentioned in issue llvm/llvm-bugzilla-archive#3239 |
|
mentioned in issue llvm/llvm-bugzilla-archive#3240 |
|
mentioned in issue llvm/llvm-bugzilla-archive#3476 |
…a2+10eb32f45d40+d09a21a0b378 🍒/FBI/50b40b051890+f9e6be5cc1a2+10eb32f45d40+d09a21a0b378
Extended Description
LLVM cannot be built as dynamic libraries, and this is a tracking bug for my effort to change that. I'll update the summary later.
The text was updated successfully, but these errors were encountered: