LLVM Bugzilla is read-only and represents the historical archive of all LLVM issues filled before November 26, 2021. Use github to submit LLVM bugs

Bug 34944 - LTO: linkonce_odr functions are deserialized multiple times from the same source
Summary: LTO: linkonce_odr functions are deserialized multiple times from the same source
Status: CONFIRMED
Alias: None
Product: libraries
Classification: Unclassified
Component: Bitcode Reader (show other bugs)
Version: trunk
Hardware: PC All
: P enhancement
Assignee: Adrian Prantl
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-10-13 16:38 PDT by Adrian Prantl
Modified: 2017-10-17 17:50 PDT (History)
8 users (show)

See Also:
Fixed By Commit(s):


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Adrian Prantl 2017-10-13 16:38:32 PDT
After r314699 (which runs the Verifier as part of the UpgradeDebugInfo) we noticed that during a (full)LTO debug build of clang we were getting lots of "DICompileUnit not listed in llvm.dbg.cu" warnings.

======================================================
The following example has been reduced from llvm-ar:

$ cat a.cpp
class Error {};
template <typename HandlerTs>
void handleAllErrors(HandlerTs Handlers) {}
inline void consumeError(Error Err) {
  handleAllErrors( []() {});
}
void ArchiveMemberHeader() 
{
  consumeError(Error());
}

$ cat b.cpp
class Error {};
template <typename HandlerTs>
void handleAllErrors( HandlerTs  Handlers) {}
inline void consumeError(Error Err) {
  handleAllErrors( []() {});
}
int main(int argc, char **argv) {
  consumeError(Error());
}

$ clang++ -x c++ -std=c++14 -fPIC  -flto -g -fno-exceptions -fno-rtti -c -o a.o -c a.cpp
$ $R/clang++ -x c++ -std=c++14 -fPIC  -flto -g -fno-exceptions -fno-rtti -c -o b.o -c b.cpp
$ $R/llvm-lto a.o b.o
DICompileUnit not listed in llvm.dbg.cu
!35 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus, file: !25, producer: "clang version 6.0.0 (trunk 315470) (llvm/trunk 315472)", isOptimized: true, runtimeVersion: 0, emissionKind: FullDebug, enums: !2)
llvm-lto: warning loading file 'b.o': ignoring invalid debug info in b.o
...
======================================================

Curious! After loading this into the debugger, I noticed that the linkonce_odr function _Z12consumeError5Error, which is present in both files, is deserialized in a most peculiar way: The BitcodeReader has a DeferredFunctionInfo map. The BitcodeReader stays alive between reading module A and module B. Module A is deserialized as usual. When module b is deserialized, the BitcodeReader object is reused and when it comes to materializing _Z12consumeError5Error it is found in the DeferredFunctionInfo, which points to the other module, A. The BitcodeReader then deserializes the function from module A(!) into module B(!), together with all the debug info metadata hanging off it, including the DICompileUnit that we are already familiar with from the warning.

This behavior seems really silly: The BitcodeReader is smart enough to know that it already found this function in another module and deserializes it from there only to later delete the function when it gets uniqued at module merge time. I'm guessing the the BitcodeReader's behavior is more a side-effect of the implementation than an intentional behavior, but I wonder if we could just not make it materialize linkonce_odr functions that were already visited, if we know that we are doing LTO. I guess this optimization would also benefit ThinLTO.
Comment 1 Hans Wennborg 2017-10-16 12:42:10 PDT
We're seeing this in Chromium builds too fwiv.
Comment 2 Adrian Prantl 2017-10-17 10:59:53 PDT
It looks like my analysis was premature. The BitcodeReader actually does get destroyed after parsing a module. What causes this symptom is that when materializing the debug info for the linkonce_odr function in the second module, we make a call to DIBuilder::buildODRType, which uses the LLVMContext to look up whether we already built a unique type with the same UID and this returns the type from the first module (which pulls in the DICompileUnit from the first module as well).

There is still a performance improvement to be made by not materializing linkonce_odr functions more than once, but at least we are deserializing them from the correct module now.
Comment 3 Adrian Prantl 2017-10-17 13:11:43 PDT
Why does a DICompositeType pull in a DICompileUnit?

class Error {};
template <typename HandlerTs>
void handleAllErrors( HandlerTs  Handlers) {}
inline void consumeError(Error Err) {
  handleAllErrors( []() {});
}

Because the concrete type of the template parameter "HandlerTS" (the type of the lambda) has a scope of "(void)consumeError(Error)":

!21 = distinct !DISubprogram(name: "consumeError", linkageName: "_Z12consumeError5Error", scope: !1, file: !1, line: 4, type: !22, isLocal: false, isDefinition: true, scopeLine: 4, flags: DIFlagPrototyped, isOptimized: false, unit: !0, variables: !2)
!29 = distinct !DISubprogram(name: "handleAllErrors<(lambda at ar.ii:5:20)>", linkageName: "_Z15handleAllErrorsIZ12consumeError5ErrorEUlvE_EvT_", scope: !1, file: !1, line: 3, type: !30, isLocal: false, isDefinition: true, scopeLine: 3, flags: DIFlagPrototyped, isOptimized: false, unit: !0, templateParams: !33, variables: !2)
!30 = !DISubroutineType(types: !31)
!31 = !{null, !32}
!32 = distinct !DICompositeType(tag: DW_TAG_class_type, scope: !21, file: !1, line: 5, size: 8, elements: !2, identifier: "_ZTSZ12consumeError5ErrorEUlvE_")
!33 = !{!34}
!34 = !DITemplateTypeParameter(name: "HandlerTs", type: !32)
Comment 4 Adrian Prantl 2017-10-17 17:50:37 PDT
Since this is far less critical than expected I opted to just teach the Verifier about this situation in r316049.