LLVM Bugzilla is read-only and represents the historical archive of all LLVM issues filled before November 26, 2021. Use github to submit LLVM bugs

Bug 3201 - LLVM cannot be built as dynamic libraries
Summary: LLVM cannot be built as dynamic libraries
Status: RESOLVED FIXED
Alias: None
Product: Build scripts
Classification: Unclassified
Component: Makefiles (show other bugs)
Version: trunk
Hardware: All All
: P enhancement
Assignee: Unassigned LLVM Bugs
URL: http://www.bitbucket.org/danchr/llvm/
Keywords:
Depends on: 3476 3239 3240
Blocks:
  Show dependency tree
 
Reported: 2008-12-11 18:03 PST by Dan Villiom Podlaski Christiansen
Modified: 2021-07-28 22:52 PDT (History)
17 users (show)

See Also:
Fixed By Commit(s):


Attachments
LLVM footprint measurements (5.36 KB, text/plain; charset=UTF-8)
2008-12-11 18:05 PST, Dan Villiom Podlaski Christiansen
Details
Compressed, complete patch (382.91 KB, patch)
2008-12-11 18:15 PST, Dan Villiom Podlaski Christiansen
Details
Partial patch, without auto-generated files (291.64 KB, patch)
2008-12-11 18:19 PST, Dan Villiom Podlaski Christiansen
Details
Screenshot of build output (45.37 KB, image/png)
2008-12-11 19:13 PST, Dan Villiom Podlaski Christiansen
Details
Another screenshot, also of a Linux build (46.46 KB, image/png)
2008-12-11 19:20 PST, Dan Villiom Podlaski Christiansen
Details
Example patch containing changes to some critical files (91.55 KB, patch)
2008-12-19 18:03 PST, Dan Villiom Podlaski Christiansen
Details
Updated patch against release_26 branch (206.92 KB, patch)
2009-08-22 16:03 PDT, Dan Villiom Podlaski Christiansen
Details
Screenshot of build output (updated) (150.55 KB, image/png)
2009-08-22 16:06 PDT, Dan Villiom Podlaski Christiansen
Details
Jumbo patch against LLVM 2.6 & Clang 1.0 (219.85 KB, patch)
2009-10-27 07:17 PDT, Dan Villiom Podlaski Christiansen
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Dan Villiom Podlaski Christiansen 2008-12-11 18:03:30 PST
LLVM cannot be built as dynamic libraries, and this is a tracking bug for my effort to change that. I'll update the summary later.
Comment 1 Dan Villiom Podlaski Christiansen 2008-12-11 18:05:12 PST
Created attachment 2294 [details]
LLVM footprint measurements
Comment 2 Dan Villiom Podlaski Christiansen 2008-12-11 18:07:11 PST
Comment on attachment 2294 [details]
LLVM footprint measurements

Setting encoding to UTF-8.
Comment 3 Dan Villiom Podlaski Christiansen 2008-12-11 18:15:40 PST
Created attachment 2295 [details]
Compressed, complete patch

Adding compressed, full patch of all current changes between my Mercurial branch and LLVM trunk, revision 60875.
Comment 4 Dan Villiom Podlaski Christiansen 2008-12-11 18:19:26 PST
Created attachment 2296 [details]
Partial patch, without auto-generated files
Comment 5 Óscar Fuentes 2008-12-11 18:21:03 PST
Have you tried the CMake build generator?

http://www.llvm.org/docs/CMake.html
Comment 6 Dan Villiom Podlaski Christiansen 2008-12-11 18:38:22 PST
(In reply to comment #5)
> Have you tried the CMake build generator?
> 
> http://www.llvm.org/docs/CMake.html

Superficially, yes. I hope to spend more time with it once I finish this tiny project :) Actually, I have a few fixes for it, somewhere…
Comment 7 Dan Villiom Podlaski Christiansen 2008-12-11 19:04:03 PST
I've uploaded a full, compressed patch of all changes in the ‘danchr/build’ branch in my Mercurial repository, relative to the LLVM Subversion trunk, revision 60875. Also uploaded is a partial patch with irrelevant or machine-generated changes removed. It's still very large; as you can see from the diffstat summary at the end of this post.

I've tested my modifications on 32-bit Red Hat Enterprise Linux 5.2, and on a G4 running Mac OS X 10.5. On RHEL, the only test suite failures I get seem to be caused by using LLVM GCC 2.4 and brokenness in the dummy Hello module. (The latter only during static builds, ironically.)

I have also adapted Clang, PoolAlloc and (part of) VMKit to the modified build system. On my G4, dyld reports the launch time of clang — linked to 18 LLVM libraries and needing 10,331 ‘binding fixups’ — as circa 90ms at idle. The binary footprint of LLVM is decreased significantly. (Somehow, static builds have decreased in size too; this may be caused by the updated Libtool.) I've attached a file containing measurements of the binary sizes.

What remains is mainly testing and documentation. Oh, and I need to figure out how to get this into the main LLVM repository too…

I'll go into detail with what I've done, and why later, but I believe my modifications can be described as a general overhaul of the LLVM Makefile build system. I've touched almost all parts of it, I believe. Suggestions are more than welcome, as are test results :)

--
diffstat’ summary of the partial patch:
 b/Makefile                                        |   17 
 b/Makefile.config.in                              |   64 +
 b/Makefile.rules                                  |  807 ++++++++++++----------
 b/autoconf/configure.ac                           |  328 ++++++--
 b/autoconf/m4/argz.m4                             |   79 ++
 b/autoconf/m4/bison.m4                            |   21 
 b/autoconf/m4/config_makefile.m4                  |    9 
 b/autoconf/m4/flex.m4                             |   21 
 b/autoconf/m4/path_tclsh.m4                       |    4 
 b/bindings/ocaml/executionengine/Makefile         |    2 
 b/examples/BrainF/Makefile                        |    2 
 b/examples/Fibonacci/Makefile                     |    2 
 b/examples/HowToUseJIT/Makefile                   |    2 
 b/examples/ModuleMaker/Makefile                   |    2 
 b/examples/ParallelJIT/Makefile                   |    2 
 b/include/llvm/Config/config.h.cmake              |  397 +++++-----
 b/include/llvm/Config/config.h.in                 |  107 +-
 b/include/llvm/Transforms/Utils/BasicInliner.h    |    3 
 b/lib/Analysis/Makefile                           |    5 
 b/lib/Archive/Makefile                            |    3 
 b/lib/AsmParser/Makefile                          |    6 
 b/lib/AsmParser/llvmAsmParser.cpp.cvs             |  604 ++++++++--------
 b/lib/AsmParser/llvmAsmParser.h.cvs               |    4 
 b/lib/AsmParser/llvmAsmParser.y.cvs               |    1 
 b/lib/Bitcode/Makefile                            |    5 
 b/lib/CodeGen/Makefile                            |    5 
 b/lib/CodeGen/PostRASchedulerList.cpp             |   22 
 b/lib/Debugger/Makefile                           |    3 
 b/lib/ExecutionEngine/Makefile                    |    6 
 b/lib/Linker/Linker.cpp                           |    2 
 b/lib/Linker/Makefile                             |    3 
 b/lib/Makefile                                    |    7 
 b/lib/Support/Makefile                            |    3 
 b/lib/System/Makefile                             |    2 
 b/lib/System/Path.cpp                             |    4 
 b/lib/System/Unix/Alarm.inc                       |    4 
 b/lib/System/Unix/Signals.inc                     |    2 
 b/lib/Target/ARM/AsmPrinter/ARMAsmPrinter.cpp     |    3 
 b/lib/Target/Alpha/AsmPrinter/AlphaAsmPrinter.cpp |    3 
 b/lib/Target/CBackend/CBackend.cpp                |    3 
 b/lib/Target/CellSPU/AsmPrinter/SPUAsmPrinter.cpp |    2 
 b/lib/Target/CppBackend/CPPBackend.cpp            |    3 
 b/lib/Target/IA64/IA64AsmPrinter.cpp              |    3 
 b/lib/Target/MSIL/MSILWriter.cpp                  |    3 
 b/lib/Target/Makefile                             |   53 +
 b/lib/Target/Mips/MipsAsmPrinter.cpp              |    3 
 b/lib/Target/PIC16/PIC16AsmPrinter.cpp            |    2 
 b/lib/Target/Sparc/AsmPrinter/SparcAsmPrinter.cpp |    3 
 b/lib/Target/X86/X86JITInfo.cpp                   |    3 
 b/lib/Target/XCore/XCoreAsmPrinter.cpp            |    3 
 b/lib/Transforms/Hello/Makefile                   |    2 
 b/lib/Transforms/IPO/Inliner.cpp                  |    5 
 b/lib/Transforms/Makefile                         |    7 
 b/lib/Transforms/Utils/BasicInliner.cpp           |   11 
 b/lib/VMCore/Makefile                             |   25 
 b/projects/sample/Makefile.common.in              |   11 
 b/projects/sample/lib/sample/Makefile             |    3 
 b/projects/sample/tools/sample/Makefile           |    2 
 b/runtime/GC/Makefile                             |    2 
 b/runtime/GC/SemiSpace/Makefile                   |    4 
 b/runtime/Makefile                                |    2 
 b/runtime/libprofile/Makefile                     |    4 
 b/test/Makefile                                   |   88 +-
 b/test/Makefile.tests                             |    7 
 b/tools/Makefile                                  |   22 
 b/tools/bugpoint/Makefile                         |    3 
 b/tools/bugpoint/ToolRunner.cpp                   |    2 
 b/tools/gccas/Makefile                            |    4 
 b/tools/gccld/Makefile                            |    4 
 b/tools/llc/Makefile                              |    9 
 b/tools/llc/llc.cpp                               |    2 
 b/tools/lli/Makefile                              |    2 
 b/tools/lli/lli.cpp                               |   38 +
 b/tools/llvm-as/Makefile                          |    2 
 b/tools/llvm-bcanalyzer/Makefile                  |    2 
 b/tools/llvm-config/CMakeLists.txt                |   15 
 b/tools/llvm-config/Makefile                      |   54 +
 b/tools/llvm-config/llvm-config.in.in             |  117 ++-
 b/tools/llvm-dis/Makefile                         |    2 
 b/tools/llvm-extract/Makefile                     |    2 
 b/tools/llvm-ld/Makefile                          |    2 
 b/tools/llvm-link/Makefile                        |    2 
 b/tools/llvm-nm/Makefile                          |    2 
 b/tools/llvm-prof/Makefile                        |    2 
 b/tools/llvmc/driver/Makefile                     |    4 
 b/tools/llvmc/plugins/Hello/Hello.cpp             |    3 
 b/tools/llvmc/plugins/Makefile                    |   36 
 b/tools/lto/LTOCodeGenerator.h                    |    5 
 b/tools/lto/Makefile                              |   32 
 b/tools/lto/lto.exports                           |   45 -
 b/tools/opt/Makefile                              |    2 
 b/utils/GenLibDeps.pl                             |   89 +-
 b/utils/TableGen/Makefile                         |    2 
 b/utils/fpcmp/Makefile                            |    2 
 b/win32/config.h                                  |    2 
 lib/Analysis/IPA/Makefile                         |   14 
 lib/Bitcode/Reader/Makefile                       |   15 
 lib/Bitcode/Writer/Makefile                       |   15 
 lib/CodeGen/AsmPrinter/Makefile                   |   15 
 lib/CodeGen/SelectionDAG/Makefile                 |   15 
 lib/ExecutionEngine/Interpreter/Makefile          |   12 
 lib/ExecutionEngine/JIT/Makefile                  |   37 -
 lib/Target/ARM/AsmPrinter/Makefile                |   15 
 lib/Target/ARM/Makefile                           |   23 
 lib/Target/Alpha/AsmPrinter/Makefile              |   15 
 lib/Target/Alpha/Makefile                         |   22 
 lib/Target/CBackend/Makefile                      |   14 
 lib/Target/CellSPU/AsmPrinter/Makefile            |   17 
 lib/Target/CellSPU/Makefile                       |   22 
 lib/Target/CppBackend/Makefile                    |   14 
 lib/Target/IA64/Makefile                          |   18 
 lib/Target/MSIL/Makefile                          |   14 
 lib/Target/Mips/Makefile                          |   21 
 lib/Target/PIC16/Makefile                         |   21 
 lib/Target/PowerPC/AsmPrinter/Makefile            |   15 
 lib/Target/PowerPC/Makefile                       |   22 
 lib/Target/Sparc/AsmPrinter/Makefile              |   15 
 lib/Target/Sparc/Makefile                         |   22 
 lib/Target/X86/AsmPrinter/Makefile                |   15 
 lib/Target/X86/Makefile                           |   23 
 lib/Target/XCore/Makefile                         |   21 
 lib/Transforms/IPO/Makefile                       |   15 
 lib/Transforms/Instrumentation/Makefile           |   15 
 lib/Transforms/Scalar/Makefile                    |   15 
 lib/Transforms/Utils/Makefile                     |   15 
 125 files changed, 1974 insertions(+), 1888 deletions(-)
Comment 8 Dan Villiom Podlaski Christiansen 2008-12-11 19:13:24 PST
Created attachment 2297 [details]
Screenshot of build output

This attachment is a screenshot showing what the output of a build looks like. Coloured output is only used when writing directly to the terminal, and can also be disabled using ‘configure’.
Comment 9 Dan Villiom Podlaski Christiansen 2008-12-11 19:20:23 PST
Created attachment 2298 [details]
Another screenshot, also of a Linux build
Comment 10 Dan Villiom Podlaski Christiansen 2008-12-19 18:03:01 PST
Created attachment 2323 [details]
Example patch containing changes to some critical files

The attached patch contains my changes to the following files:
- Makefile.rules
- Makefile
- Makefile.common
- Makefile.config.in
- lib/Target/Makefile
- utils/GenLibDeps.pl
- tools/llvm-config/Makefile
- tools/llvm-config/llvm-config.in.in

As such, it should be a good example of the approach I took.
Comment 11 Pekka Jääskeläinen 2009-03-09 06:14:12 PDT
Any progress with this?
Comment 12 Nick Lewycky 2009-03-09 11:46:13 PDT
Independently of Dan's work, LLVM now builds PIC by default and we've removed libtool from the makefiles (but not autoconf).
Comment 13 Dan Villiom Podlaski Christiansen 2009-03-11 06:59:17 PDT
(In reply to comment #11)
> Any progress with this?

I've continued to regularly update and merge my Mercurial mirror & branch, but other than that, I haven't done much work on this lately, I'm afraid. My main problem is that testing build system modifications on an old G4 takes hours for each run, and proper testing requires several runs.

(In reply to comment #12)
> Independently of Dan's work, LLVM now builds PIC by default and we've removed
> libtool from the makefiles (but not autoconf).

I've begun adapting my changes to this change, though, and even though I haven't spent much time on LLVM lately, I've thought of possible clean ups to make my changes more acceptable to the community.

In the meantime, I've been hacking on HgSubversion trying to make it suitable for mirroring the LLVM repositories :)
Comment 14 Dan Villiom Podlaski Christiansen 2009-08-22 16:03:51 PDT
Created attachment 3355 [details]
Updated patch against release_26 branch

This is a more recent patch against the ‘release_26’ branch. I've been cleaning it up lately, and it seems to work reasonably well :)

 b/Makefile                                  |   25 
 b/Makefile.config.in                        |  126 +-
 b/Makefile.rules                            | 1377 +++++++++++++++-------------
 b/autoconf/configure.ac                     |  377 +++++--
 b/autoconf/m4/config_makefile.m4            |    6 
 b/bindings/ocaml/Makefile                   |    1 
 b/bindings/ocaml/Makefile.ocaml             |   89 -
 b/bindings/ocaml/transforms/scalar/Makefile |    2 
 b/examples/HelloPass/Makefile               |   16 
 b/examples/Makefile                         |    2 
 b/include/llvm/Config/config.h.in           |   13 
 b/lib/Analysis/IPA/Makefile                 |    2 
 b/lib/Analysis/Makefile                     |    2 
 b/lib/Archive/Makefile                      |    2 
 b/lib/AsmParser/Makefile                    |    2 
 b/lib/Bitcode/Reader/Makefile               |    2 
 b/lib/Bitcode/Writer/Makefile               |    2 
 b/lib/CodeGen/AsmPrinter/Makefile           |    1 
 b/lib/CodeGen/Makefile                      |    2 
 b/lib/CodeGen/SelectionDAG/Makefile         |    1 
 b/lib/CompilerDriver/Makefile               |    1 
 b/lib/Debugger/Makefile                     |    2 
 b/lib/ExecutionEngine/Interpreter/Makefile  |    1 
 b/lib/ExecutionEngine/JIT/Makefile          |    1 
 b/lib/ExecutionEngine/JIT/TargetSelect.cpp  |   34 
 b/lib/ExecutionEngine/Makefile              |    1 
 b/lib/Linker/Makefile                       |    2 
 b/lib/MC/Makefile                           |    2 
 b/lib/Makefile                              |   28 
 b/lib/Support/Makefile                      |    1 
 b/lib/System/Makefile                       |    1 
 b/lib/System/Unix/Alarm.inc                 |    4 
 b/lib/System/Unix/Signals.inc               |    4 
 b/lib/System/Unix/Unix.h                    |    2 
 b/lib/Target/Makefile                       |   41 
 b/lib/Transforms/IPO/Makefile               |    2 
 b/lib/Transforms/Instrumentation/Makefile   |    2 
 b/lib/Transforms/Scalar/Makefile            |    2 
 b/lib/Transforms/Utils/Makefile             |    2 
 b/lib/VMCore/Makefile                       |   23 
 b/projects/Makefile                         |    2 
 b/projects/sample/Makefile.common.in        |   13 
 b/projects/sample/autoconf/AutoRegen.sh     |    8 
 b/projects/sample/autoconf/configure.ac     |   13 
 b/projects/sample/lib/sample/Makefile       |    3 
 b/runtime/Makefile                          |    2 
 b/runtime/libprofile/Makefile               |    5 
 b/test/Makefile                             |   94 +
 b/test/Makefile.tests                       |    7 
 b/tools/Makefile                            |   17 
 b/tools/gccas/Makefile                      |    4 
 b/tools/gccld/Makefile                      |    4 
 b/tools/llvm-config/CMakeLists.txt          |    3 
 b/tools/llvm-config/Makefile                |   79 +
 b/tools/llvm-config/llvm-config.in.in       |  132 +-
 b/tools/llvmc/example/Hello/Hello.cpp       |    4 
 b/tools/lto/Makefile                        |   27 
 b/tools/lto/lto.exports                     |   47 
 b/utils/GenLibDeps.pl                       |   69 -
 b/utils/Makefile                            |   10 
 b/utils/TableGen/Makefile                   |    4 
 b/utils/color-echo.py.in                    |   34 
 lib/Target/ARM/AsmPrinter/Makefile          |   15 
 lib/Target/ARM/Makefile                     |   23 
 lib/Target/ARM/TargetInfo/Makefile          |   15 
 lib/Target/Alpha/AsmPrinter/Makefile        |   15 
 lib/Target/Alpha/Makefile                   |   22 
 lib/Target/Alpha/TargetInfo/Makefile        |   15 
 lib/Target/Blackfin/AsmPrinter/Makefile     |   16 
 lib/Target/Blackfin/Makefile                |   23 
 lib/Target/Blackfin/TargetInfo/Makefile     |   15 
 lib/Target/CBackend/Makefile                |   17 
 lib/Target/CBackend/TargetInfo/Makefile     |   15 
 lib/Target/CellSPU/AsmPrinter/Makefile      |   17 
 lib/Target/CellSPU/Makefile                 |   22 
 lib/Target/CellSPU/TargetInfo/Makefile      |   15 
 lib/Target/CppBackend/Makefile              |   17 
 lib/Target/CppBackend/TargetInfo/Makefile   |   15 
 lib/Target/MSIL/Makefile                    |   17 
 lib/Target/MSIL/TargetInfo/Makefile         |   15 
 lib/Target/MSP430/Makefile                  |   23 
 lib/Target/MSP430/TargetInfo/Makefile       |   15 
 lib/Target/Mips/AsmPrinter/Makefile         |   17 
 lib/Target/Mips/Makefile                    |   23 
 lib/Target/Mips/TargetInfo/Makefile         |   15 
 lib/Target/PIC16/AsmPrinter/Makefile        |   15 
 lib/Target/PIC16/Makefile                   |   23 
 lib/Target/PIC16/TargetInfo/Makefile        |   15 
 lib/Target/PowerPC/AsmPrinter/Makefile      |   15 
 lib/Target/PowerPC/Makefile                 |   22 
 lib/Target/PowerPC/TargetInfo/Makefile      |   15 
 lib/Target/Sparc/AsmPrinter/Makefile        |   15 
 lib/Target/Sparc/Makefile                   |   22 
 lib/Target/Sparc/TargetInfo/Makefile        |   15 
 lib/Target/SystemZ/AsmPrinter/Makefile      |   15 
 lib/Target/SystemZ/Makefile                 |   22 
 lib/Target/SystemZ/TargetInfo/Makefile      |   15 
 lib/Target/X86/AsmParser/Makefile           |   15 
 lib/Target/X86/AsmPrinter/Makefile          |   15 
 lib/Target/X86/Makefile                     |   23 
 lib/Target/X86/TargetInfo/Makefile          |   15 
 lib/Target/XCore/Makefile                   |   23 
 lib/Target/XCore/TargetInfo/Makefile        |   15 
 105 files changed, 1666 insertions(+), 1840 deletions(-)
Comment 15 Dan Villiom Podlaski Christiansen 2009-08-22 16:06:35 PDT
Created attachment 3356 [details]
Screenshot of build output (updated)

This is a screenshot of what the build output currently looks like.
Comment 16 Török Edwin 2009-09-04 03:58:53 PDT
Having a single .so for LLVM raises a number of problems:
 - some projects using LLVM needs only parts of LLVM (only JIT for example)
 - the API/ABI of LLVM trunk is always changing, thus an installed libLLVM.so
raises problems: project A uses LLVM version X, project B uses LLVM version B,
installing them together would be impossible if they both dynamically link to LLVM
 - this wouldn't be a problem for releases though, where the soname would be changed for each release

As long as the .so build is optional, and the .a are still built PIC I'm fine with it though. Keeping the current separation of one .a per LLVM module is good for observing code size issues (like codegen depending on asmprinter).
Comment 17 Dan Villiom Podlaski Christiansen 2009-09-04 04:56:46 PDT
(In reply to comment #16)
> Having a single .so for LLVM raises a number of problems:
>  - some projects using LLVM needs only parts of LLVM (only JIT for example)

I agree, but due to the both the way I do this and how the ‘lib’ directory is organised, I have no other choice. Building a dynamically linked libLLVM consists of the following steps:
1) descend into lib/System & lib/Support as usual, and build static archives.
2) descind into utils and build tblgen linked against the archives
3) descend into lib
4) descend into the subdirectories lib, building static libraries unless disabled, and generating built sources
5) on shared builds, build all of the LLVM sources once more, always PIC, and link it into a dynamic library
6) at install time, tblgen is relinked against the dynamic libLLVM, and if not on Darwin, all libraries and tools are relinked

The process for building Clang is similar to steps three through six. I do agree that *one* libLLVM.dylib is too much, but it's worth noting that calls into another library is slow. The current library granularity will cause significant slowdowns if directly translated to one library per archive. On the other hand, Clang seems to execute a bit faster in wall time on my G4, probably due to caching and so on.

I don't have any recent timings on various library builds, and I actually haven't run the tests recently either; I simply don't have the hardware to do that on a regular basis…

>  - the API/ABI of LLVM trunk is always changing, thus an installed libLLVM.so
> raises problems: project A uses LLVM version X, project B uses LLVM version B,
> installing them together would be impossible if they both dynamically link to
> LLVM

I haven't tested whether it still works, but I know that Apple use LLVM_SUBMIT_VERSION for versioning their builds. Other users needing to distinguish arbitrary builds could do a similar trick. Most libraries have install names like this:

darwin: $libdir/lib$NAME[.$MAJOR[.$MINOR[.$SUBMINOR]]].dylib
*: $libdir/lib$NAME.so[.$MAJOR[.$MINOR[.$SUBMINOR]]]

>  - this wouldn't be a problem for releases though, where the soname would be
> changed for each release

Indeed; except for libLTO, all libraries are versioned. I'd suggest putting the C bindings into a similarly unversioned library.

> As long as the .so build is optional, and the .a are still built PIC I'm fine
> with it though. Keeping the current separation of one .a per LLVM module is
> good for observing code size issues (like codegen depending on asmprinter).

The default behaviour in my Mercurial branch is largely similar to that of the 2.6 release branch. The main differences are altered — and in my opinion, improved — build progress output, and one libLLVMTarget for all targets in one. I have the infrastructure in place in ‘llvm-config.in.in’ to provide hidden compatibility aliases. Oh, and I also told llvm-config about LTO and Clang :) The way it currently deal with shared builds is a bit of a hack, though…

For reference, I put a listing of the results of two builds below. Both builds were my branch (based on 2.6), optimised and w/o PIC. (The PIC setting has no affect on dynamic libraries.) A static-only and a shared-only:

$ export CC=gcc-4.2 CXX=g++-4.2
$ ../../configure --enable-optimized --enable-shared --disable-static --disable-pic
$ ../../configure --enable-optimized --disable-pic

shared/Release/lib:
total 56224
  11K LLVMHello.dylib*
 102B 13 Aug 17:40 clang/
 7,4M libClang.2.7.dylib*
  18B libClang.2.dylib@ -> libClang.2.7.dylib
  18B libClang.dylib@ -> libClang.2.7.dylib
 179K libCompilerDriver.a
  19M libLLVM.2.7.dylib*
  17B libLLVM.2.dylib@ -> libLLVM.2.7.dylib
  17B libLLVM.dylib@ -> libLLVM.2.7.dylib
 697K libLLVMSupport.a
 117K libLLVMSystem.a
  58K libLTO.dylib*
 156K libplugin_llvmc_Base.a
 101K libplugin_llvmc_Clang.a
 9,4K profile_rt.dylib*

static/Release/lib:
total 103080
 102B 13 Aug 17:53 clang/
 1,8M libClangAST.a
 2,0M libClangAnalysis.a
 473K libClangBasic.a
 2,1M libClangCodeGen.a
 463K libClangDriver.a
 1,9M libClangFrontend.a
 217K libClangIndex.a
 488K libClangLex.a
 757K libClangParse.a
  88K libClangRewrite.a
 3,1M libClangSema.a
 179K libCompilerDriver.a
 1,8M libLLVMAnalysis.a
  63K libLLVMArchive.a
 319K libLLVMAsmParser.a
 421K libLLVMAsmPrinter.a
 261K libLLVMBitReader.a
 282K libLLVMBitWriter.a
 3,7M libLLVMCodeGen.a
 2,4M libLLVMCore.a
 111K libLLVMDebugger.a
 126K libLLVMExecutionEngine.a
 143K libLLVMInstrumentation.a
 228K libLLVMInterpreter.a
 221K libLLVMJIT.a
 157K libLLVMLinker.a
 169K libLLVMMC.a
 3,0M libLLVMScalarOpts.a
 4,7M libLLVMSelectionDAG.a
 697K libLLVMSupport.a
 117K libLLVMSystem.a
  15M libLLVMTarget.a
 1,3M libLLVMTransformUtils.a
 351K libLLVMipa.a
 1,0M libLLVMipo.a
 156K libplugin_llvmc_Base.a
 101K libplugin_llvmc_Clang.a
 9,4K profile_rt.dylib*
Comment 18 Török Edwin 2009-09-04 05:02:30 PDT
(In reply to comment #17)
> The process for building Clang is similar to steps three through six. I do
> agree that *one* libLLVM.dylib is too much, but it's worth noting that calls
> into another library is slow. The current library granularity will cause
> significant slowdowns if directly translated to one library per archive. 

Yes, having a separate .so for each module would cause slowdowns.
> 
> For reference, I put a listing of the results of two builds below. Both builds
> were my branch (based on 2.6), optimised and w/o PIC. (The PIC setting has no
> affect on dynamic libraries.)

Doesn't --disable-pic work properly then?
Dynamic libraries must be PIC, or else they won't work on x86_64.


Comment 19 Dan Villiom Podlaski Christiansen 2009-09-04 05:35:01 PDT
(In reply to comment #18)
> (In reply to comment #17)
> > The process for building Clang is similar to steps three through six. I do
> > agree that *one* libLLVM.dylib is too much, but it's worth noting that calls
> > into another library is slow. The current library granularity will cause
> > significant slowdowns if directly translated to one library per archive. 
> 
> Yes, having a separate .so for each module would cause slowdowns.

Yup. I plan on moving my changes from a branch to a Mercurial patch queue. Other than allowing me to split up my changes, it should also make reorganising easier.

By the way, I forgot to mention something. In order to merge all the target libraries, I had to move five source files to other libraries.  To prevent circular dependancies, I moved TargetData.cpp to Analysis, and TargetInstrInfo.cpp, TargetLoweringObjectFile.cpp, TargetMachine.cpp and TargetRegisterInfo.cpp to CodeGen. I shan't say whether this makes sense on a conceptual level, but the dependancies mandated it ;-)

> > For reference, I put a listing of the results of two builds below. Both builds
> > were my branch (based on 2.6), optimised and w/o PIC. (The PIC setting has no
> > affect on dynamic libraries.)
> 
> Doesn't --disable-pic work properly then?
> Dynamic libraries must be PIC, or else they won't work on x86_64.

I thought dynamic libraries had to be PIC everywhere? PIC is forced for shared libraries, and some loadable modules are skipped when building static w/o PIC. Disabling PIC for a shared build only affects non-library compilation; executables, archives and so on.
Comment 20 Dan Villiom Podlaski Christiansen 2009-09-04 05:49:01 PDT
Adding Chris Lattner to the Cc list; I'd appreciate his input on this.
Comment 21 Dan Villiom Podlaski Christiansen 2009-09-04 06:31:01 PDT
Oh, and yet another thing: I had to remove the ‘march’, ‘mcpu’ and ‘mattr’ definitions from ‘lib/ExecutionEngine/JIT/TargetSelect.cpp’. This is a consequence of linking everything into one library, as ‘llc’ also defines such options.
Comment 22 Chris Lattner 2009-09-07 20:01:21 PDT
Hi Dan, what specifically is the question?
Comment 23 Dan Villiom Podlaski Christiansen 2009-10-27 07:17:54 PDT
Created attachment 3703 [details]
Jumbo patch against LLVM 2.6 & Clang 1.0

Sorry for not responding earlier; attached below is a full patch against the LLVM 2.6 & Clang 1.0 release branches. Any hints or suggestions on how to break this up would be greatly appreciated :-)

The patch can also be viewed on Bitbucket:
<http://bitbucket.org/danchr/llvm-2.6/diff/?diff2=dynamic-build&diff1=default>

 Makefile                                  |    25 +-
 Makefile.config.in                        |   126 +-
 Makefile.rules                            |  1432 ++++++++++++++++------------
 autoconf/AutoRegen.sh                     |    56 +-
 autoconf/configure.ac                     |   392 +++++-
 autoconf/m4/config_makefile.m4            |     6 +-
 autoconf/m4/subst_expanded.m4             |    11 +
 bindings/ocaml/Makefile                   |     1 -
 bindings/ocaml/Makefile.ocaml             |    93 +-
 bindings/ocaml/transforms/scalar/Makefile |     2 +-
 lib/Transforms/Hello/CMakeLists.txt       |   Bin 
 lib/Transforms/Hello/Hello.cpp            |   Bin 
 lib/Transforms/Hello/Makefile             |     3 +-
 examples/Makefile                         |     7 +-
 include/llvm/Config/config.h.in           |    13 +-
 lib/Analysis/IPA/Makefile                 |     2 +-
 lib/Analysis/Makefile                     |     2 +-
 lib/Target/TargetData.cpp                 |   Bin 
 lib/Archive/Makefile                      |     2 +-
 lib/AsmParser/Makefile                    |     2 +-
 lib/Bitcode/Reader/Makefile               |     2 +-
 lib/Bitcode/Writer/Makefile               |     2 +-
 lib/CodeGen/AsmPrinter/Makefile           |     1 +
 lib/CodeGen/Makefile                      |     2 +-
 lib/CodeGen/SelectionDAG/Makefile         |     1 +
 lib/Target/TargetAsmInfo.cpp              |   Bin 
 lib/Target/TargetInstrInfo.cpp            |   Bin 
 lib/Target/TargetLoweringObjectFile.cpp   |   Bin 
 lib/Target/TargetMachine.cpp              |   Bin 
 lib/Target/TargetRegisterInfo.cpp         |   Bin 
 lib/CompilerDriver/Makefile               |     1 +
 lib/Debugger/Makefile                     |     2 +-
 lib/ExecutionEngine/Interpreter/Makefile  |     1 +
 lib/ExecutionEngine/JIT/Makefile          |     1 +
 lib/ExecutionEngine/JIT/TargetSelect.cpp  |    74 +-
 lib/ExecutionEngine/Makefile              |     1 +
 lib/Linker/Makefile                       |     2 +-
 lib/MC/Makefile                           |     2 +-
 lib/Makefile                              |    30 +-
 lib/Support/Makefile                      |     1 +
 lib/System/Makefile                       |     1 +
 lib/System/Unix/Alarm.inc                 |     4 +-
 lib/System/Unix/Signals.inc               |     4 +-
 lib/System/Unix/Unix.h                    |     2 -
 lib/Target/ARM/AsmPrinter/Makefile        |    15 -
 lib/Target/ARM/Makefile                   |    23 -
 lib/Target/ARM/TargetInfo/Makefile        |    15 -
 lib/Target/Alpha/AsmPrinter/Makefile      |    15 -
 lib/Target/Alpha/Makefile                 |    22 -
 lib/Target/Alpha/TargetInfo/Makefile      |    15 -
 lib/Target/Blackfin/AsmPrinter/Makefile   |    16 -
 lib/Target/Blackfin/Makefile              |    23 -
 lib/Target/Blackfin/TargetInfo/Makefile   |    15 -
 lib/Target/CBackend/Makefile              |    17 -
 lib/Target/CBackend/TargetInfo/Makefile   |    15 -
 lib/Target/CellSPU/AsmPrinter/Makefile    |    17 -
 lib/Target/CellSPU/Makefile               |    22 -
 lib/Target/CellSPU/TargetInfo/Makefile    |    15 -
 lib/Target/CppBackend/Makefile            |    17 -
 lib/Target/CppBackend/TargetInfo/Makefile |    15 -
 lib/Target/MSIL/Makefile                  |    17 -
 lib/Target/MSIL/TargetInfo/Makefile       |    15 -
 lib/Target/MSP430/AsmPrinter/Makefile     |    15 -
 lib/Target/MSP430/Makefile                |    23 -
 lib/Target/MSP430/TargetInfo/Makefile     |    15 -
 lib/Target/Makefile                       |    42 +-
 lib/Target/Mips/AsmPrinter/Makefile       |    17 -
 lib/Target/Mips/Makefile                  |    23 -
 lib/Target/Mips/TargetInfo/Makefile       |    15 -
 lib/Target/PIC16/AsmPrinter/Makefile      |    15 -
 lib/Target/PIC16/Makefile                 |    23 -
 lib/Target/PIC16/TargetInfo/Makefile      |    15 -
 lib/Target/PowerPC/AsmPrinter/Makefile    |    15 -
 lib/Target/PowerPC/Makefile               |    22 -
 lib/Target/PowerPC/TargetInfo/Makefile    |    15 -
 lib/Target/Sparc/AsmPrinter/Makefile      |    15 -
 lib/Target/Sparc/Makefile                 |    22 -
 lib/Target/Sparc/TargetInfo/Makefile      |    15 -
 lib/Target/SystemZ/AsmPrinter/Makefile    |    15 -
 lib/Target/SystemZ/Makefile               |    22 -
 lib/Target/SystemZ/TargetInfo/Makefile    |    15 -
 lib/Target/X86/AsmParser/Makefile         |    15 -
 lib/Target/X86/AsmPrinter/Makefile        |    15 -
 lib/Target/X86/Makefile                   |    23 -
 lib/Target/X86/TargetInfo/Makefile        |    15 -
 lib/Target/XCore/AsmPrinter/Makefile      |    16 -
 lib/Target/XCore/Makefile                 |    23 -
 lib/Target/XCore/TargetInfo/Makefile      |    15 -
 lib/Transforms/IPO/Makefile               |     2 +-
 lib/Transforms/Instrumentation/Makefile   |     2 +-
 lib/Transforms/Makefile                   |     9 +-
 lib/Transforms/Scalar/Makefile            |     2 +-
 lib/Transforms/Utils/Makefile             |     2 +-
 lib/VMCore/Makefile                       |    23 +-
 projects/Makefile                         |     2 +-
 projects/sample/Makefile.common.in        |    15 +-
 projects/sample/autoconf/AutoRegen.sh     |     8 +-
 projects/sample/autoconf/configure.ac     |    13 +-
 projects/sample/lib/sample/Makefile       |     3 +-
 runtime/Makefile                          |     2 +
 runtime/libprofile/Makefile               |     5 +-
 test/Makefile                             |    96 +-
 test/Makefile.tests                       |     7 +-
 tools/Makefile                            |    17 +-
 tools/gccas/Makefile                      |     8 +-
 tools/gccld/Makefile                      |     8 +-
 tools/llvm-ar/Makefile                    |     4 +-
 tools/llvm-config/Makefile                |    81 +-
 tools/llvm-config/llvm-config.in.in       |   132 +-
 tools/llvmc/example/Hello/Hello.cpp       |     4 +
 tools/lto/Makefile                        |    29 +-
 tools/lto/lto.exports                     |    47 +-
 unittests/ADT/Makefile                    |     3 +-
 unittests/ExecutionEngine/JIT/Makefile    |     3 +-
 unittests/ExecutionEngine/Makefile        |     4 +-
 unittests/Makefile                        |     6 +-
 unittests/Makefile.unittest               |    35 -
 unittests/Support/Makefile                |     3 +-
 unittests/VMCore/Makefile                 |     3 +-
 utils/GenLibDeps.pl                       |    69 +-
 utils/Makefile                            |     2 +-
 utils/TableGen/Makefile                   |     4 +
 utils/color-echo.py.in                    |    34 +
 123 files changed, 1776 insertions(+), 2010 deletions(-)
Comment 24 Dan Villiom Podlaski Christiansen 2009-10-27 07:27:22 PDT
(In reply to comment #22)
> Hi Dan, what specifically is the question?

The question is really, where do I go from here? I think the simplest and most independent part of the changes is moving five source files — TargetData.cpp, TargetInstrInfo.cpp, TargetLoweringObjectFile.cpp, TargetMachine.cpp & TargetRegisterInfo.cpp — from ‘lib/Target’ to either ‘lib/Analysis’ or ‘lib/CodeGen’. This allowed me to generate one single archive containing all targets.
Comment 25 Dan Villiom Podlaski Christiansen 2010-02-01 14:14:57 PST
I figured it would be worth mentioning, but Unladen Swallow also has a bug for this, containing some additional discussion:

<http://code.google.com/p/unladen-swallow/issues/detail?id=130>

I'm reassigning this to the default, as I'm no longer actively working on this.
Comment 26 Jeffrey Yasskin 2010-02-17 20:44:15 PST
LLVM can be built as a shared library after http://llvm.org/viewvc/llvm-project?view=rev&revision=96559. I'm going to have a redhat distro guy look it over to see if I can improve anything and try to give llvm-config a flag to select the shared library before marking this as fixed.
Comment 27 Justin Hibbits 2012-02-08 20:49:10 PST
Commenting in here, but can open a new specific ticket if that's better.  Although LLVM can be built with shared libraries, with ELF/PowerPC it fails to link at runtime because of non-PIC references to PPCCompilationCallbackC in PPCJITInfo.cpp.  I don't know how to fix this because it's in inline asm and not a separate file.