Bugzilla – Bug 540
Need to link all tools with thread libraries
Last modified: 2010-02-22 12:47:52 CST
In order to support a multithreaded JIT, the LLVM build process must also pull in the pthreads library. This means that there must be autoconf detection for pthreads and all that sort of thing. I struggled for a couple hours just trying to get LLVM to be able to "AutoRegen.sh" on my Debian system, and I was unable to get it working. In Reid's mailing list post, he made it sound like it wouldn't be challenging for him to do it.
I can help here, but I'm unsure what changes you're requesting. I don't think the "all tools" mention in the summary line is valid because clearly not "all tools" need to be multi-threaded. Probably, only the JIT (lli) needs to be multi-threaded. That being the case, I can set up a configure variable containing the -lwhatever option that should be added to the link line and that variable can be used in lli's makefile. *HOWEVER*, I'm really not sure this is wise unless there is consensus that lli should always be a multi-threaded program. Certainly, there's enough problems with multi-threading that this could break many tests. I'd like some other folks to chime in on this before we make this change.
Well, to be more specific, anything that pulls in "ExecutionEngine.h" needs to link the thread library, since the JIT lock is defined in this header file. If the only thing that includes it is the JIT, then that is the only thing that needs it. Well, to be clear, it isn't that lli itself is multithreaded. My patch explicitly makes sure that the JIT executes sequentially. The issue is that currently, LLVM *cannot* execute multithreaded applications. I agree: I don't want to break anyone, but I think there is not much risk in a patch that adds a lock. It seemed to me that Chris thought that the JIT should support threads. I'll try and dig up a mailing list post, and move this discussion there. This reminds me: I need to update my patch...
We need some consensus on whether we link tools that use ExecutionEngine.h (e.g. lli) with libpthread or not. This is the only issue standing in the way of bug 418 and bug 540. Its fine with me. What say others?
Personally I would like to defer that decision to the tool builder. For example, lli should link to pthreads by default (to work with threaded programs), but it should be possible to use the JIT without it. Given your foray into weak thread symbols, this should be very possible. -Chris
Here's what I've done to resolve this situation: 1. Added a configure script test to ensure -lpthread gets added to the end of the link line for all normally built LLVM tools. 2. Written a Mutex class in lib/System that observes the weak symbols and won't call them if they aren't defined (-lpthread isn't linked in). 3. Designed the Mutex class so that additional threading packages can be used if libpthread is not found (no support currently). This means that, by default, all LLVM tools will link against libpthread if it is found at configure time. If not, the Mutex stuff becomes a bunch of no-ops. Similarly, even if libpthread is found, the Mutex class will revert to no-ops on two conditions: (a) the program is not linked with -lpthread and (b) the pthread interface functions all have weak linkage. In the case where (a) holds but (b) does not hold (GCC 4.0.1/Linux at least) you will get link errors. This is the reason why all tools must link against libpthread if it is found. For Win32, there are no alternate threading tools that make sense so we don't run into this issue. The Mutex class on Win32 will be implemented by jeffc using Win32 Critical Sections after I commit the inital version of the code.
Resolved by these patches: http://mail.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20050711/027015.html http://mail.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20050711/027016.html http://mail.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20050711/027017.html