Bugzilla – Bug 351
Consolidate Operating System Abstraction Layer
Last modified: 2005-12-22 14:11:42
You need to log in before you can comment on or make changes to this bug.
Although LLVM purposely doesn't use much of an operating system's functionality, it does use some and its current implementation is somewhat Unix and Linux specific. The purpose of this is to create a "System" module that will hide all operating system dependencies from LLVM. Specifically this will involve: 1. include/llvm/System A set of header files that provide C++ abstractions like Path, File, Directory, Shared Object (Dynamic Link Library), Process, Mutex, Guard, Thread (maybe), and abstractions for other operating system objects used by LLVM. Only those abstractions actually used by LLVM will be included. 2. lib/System Implementation of the abstractions. The design will be based on the operating system abstraction layer in XPS. This design enforces that (a) there will be zero visibility of operating system specific header files through the interface (STL header excepted), (b) the abstractions will be provided in C++ classes that allocate no memory so they are suitable for quick allocation on the stack, (c) there are no polymorphic methods, (d) implementation of operating system specific methods is in operating system specific sub-directories, (e) methods throw no exceptions and are declared with throw() to enforce this, (f) operating system error handling will be coalesced into a very lightweight ErrorCode object that can assist with interpretation of the error. Most of these requirements are aimed at making the abstraction layer very light weight but providing complete sheilding of LLVM from the operating system. 3. The initial implementation will be for Linux. A second implementation based on the Apache Portable Runtime is planned. Chris volunteered to implement the win32 implementation. We'll need Mac OS X and Solaris implementations as well which I will plagarize from the Linux implementation because of similarities. However, someone with those platforms will have to actually compile and implement the abstractions for those platforms because I don't have them (yet!) 4. Conversion of LLVM won't happen until all needed platforms pass a battery of tests to ensure that the System library functions correctly on each platform.
A couple other notes on this: 5. After conversion, it will be *illegal* in LLVM to #include any header files like unistd.h, windows.h, stdio.h or sys/fcntl.h, etc. If you need functionality from the operating system, you must provide it as an abstraction (or extension to an existing abstraction) in the System library. 6. The System library and header files will #include NOTHING from LLVM. System is used by LLVM but does not use LLVM itself. This must be a very clear and definite boundary.
In fact, what I would like to see is libsystem depend only on the OS (it is standalone and depends on no other libraries), libvmcore depends only on libsystem, and libsupport would depend on both of them. If libsupport gets too empty, it can be merged into the vmcore library. -Chris
Agreed.
While we're at it, there are a number of abstract data types (SetVector, ilist, etc. etc.) in the Support library. We should move these to an ADT (abstract data type) library after doing the System/Support split. It might just be that Support goes away leaving us with System and ADT :) Reid.
This work has been started. There is now an include/llvm/System and lib/System directories that provide a skeletal implementation of the library. Please consider using the library as you develop new code. Old code will be migrated to lib/System as time permits.
This is making a lot of progress now. Do you know what specific things are required before we can close this bug? -Chris
This bug can only be closed when the entie LLVM source base has been converted to use the lib/System interfaces. This means that SysUtils.{h,cpp} and FileUtils.{h,cpp} go away and none of LLVM #ncludes any system specific header files. We're a long way from that. My current work only supports what llvmc and llvm-ld need. I've also converted the JIT's memory allocation. But there are many, many other uses of native calls in LLVM. Tracking them down, abstracting what's really intended and implementing them in a portable way still needs to be done. My guess: we're about 10-15% of the way there.
Ok, sounds good. Thanks Reid!
This is currently started but not finished. We're about 25% done. There are more abstractions to write and usage of the abstractions needs to replace existing (non-portable) functions in lib/Support. Scheduled for 1.5
*** Bug 350 has been marked as a duplicate of this bug. ***
This is currently about 90% done. Only three tools remain that are unportable: llvm-stub, llvm-ld, and bugpoint. The remaining issues have been sent out for comment via email. This should get wrapped up in the next week or so.
Here are the remaining non-portable headers that LLVM includes as of this date. lib/Debugger/UnixLocalInferiorProcess.cpp:39:#include <unistd.h> lib/Debugger/UnixLocalInferiorProcess.cpp:40:#include <sys/types.h> lib/Debugger/UnixLocalInferiorProcess.cpp:41:#include <sys/wait.h> lib/ExecutionEngine/JIT/Intercept.cpp:52:#include <sys/stat.h> lib/Support/SlowOperationInformer.cpp:18:#include <signal.h> lib/Support/SlowOperationInformer.cpp:19:#include <unistd.h> tools/bugpoint/OptimizerDriver.cpp:27:#include <unistd.h> tools/bugpoint/OptimizerDriver.cpp:28:#include <sys/types.h> tools/bugpoint/OptimizerDriver.cpp:29:#include <sys/wait.h> tools/llvm-stub/unistd.h:20:#include <unistd.h> tools/llvm-stub/unistd.h:24:#include <process.h> tools/llvm-stub/unistd.h:25:#include <io.h> tools/llvm-stub/llvm-stub.c:23:#include <stdio.h> tools/llvm-stub/llvm-stub.c:24:#include <stdlib.h> tools/llvm-stub/llvm-stub.c:25:#include <string.h> tools/llvm-stub/llvm-stub.c:30:#include <unistd.h> tools/llvm-stub/llvm-stub.c:34:#include <process.h> tools/llvm-stub/llvm-stub.c:35:#include <io.h>
SlowOperationInformer has been made platform independent with these patches: http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20051219/030315.html http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20051219/030318.html http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20051219/030317.html http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20051219/030316.html http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20051219/030319.html
Created an attachment (id=304) [details] A proposed patch for making bugpoint portable This patch removes use of fork/exec from bugpoint and replaces it with the sys::Program::ExecuteAndWait. The change requires new options, -as-child and -child-output, to signal bugpoint that it is being run as a child. In that mode, it processes input running several passes and creates output. The parent picks up the result code to decide what to do.
Decision: llvm-db will not be ported to Windows. The forthcoming support for DWARF and COFF debugging info in LLVM will make it possible to debug LLVM generated programs with the native debugger for a given platform. This will eventually eliminate the need for llvm-db so porting it isn't worth the effort at this point.
bugpoint has been made platform independent with these patches: http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20051219/030355.html http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20051219/030356.html
Decisions: (1) llvm-stub is already portable (via ifdefs) and there is no point moving that minute amount of code into lib/System. Consequently, it will be left as ifdef'd. (2) The ifdef'd code in the JIT to resolve symbols in libc_nonshared.a will also be ignored for much the same reason. Consequently, this bug is finished. New portability bugs should receive their own bug number from here on out.