Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LTO: linkonce_odr functions are deserialized multiple times from the same source #34292

Open
adrian-prantl opened this issue Oct 13, 2017 · 5 comments
Assignees
Labels
bugzilla Issues migrated from bugzilla confirmed Verified by a second party llvm:bitcode

Comments

@adrian-prantl
Copy link
Collaborator

Bugzilla Link 34944
Version trunk
OS All
CC @adrian-prantl,@dwblaikie,@dexonsmith,@zmodem,@joker-eph,@pcc

Extended Description

After r314699 (which runs the Verifier as part of the UpgradeDebugInfo) we noticed that during a (full)LTO debug build of clang we were getting lots of "DICompileUnit not listed in llvm.dbg.cu" warnings.

======================================================
The following example has been reduced from llvm-ar:

$ cat a.cpp
class Error {};
template
void handleAllErrors(HandlerTs Handlers) {}
inline void consumeError(Error Err) {
handleAllErrors( {});
}
void ArchiveMemberHeader()
{
consumeError(Error());
}

$ cat b.cpp
class Error {};
template
void handleAllErrors( HandlerTs Handlers) {}
inline void consumeError(Error Err) {
handleAllErrors( {});
}
int main(int argc, char **argv) {
consumeError(Error());
}

$ clang++ -x c++ -std=c++14 -fPIC -flto -g -fno-exceptions -fno-rtti -c -o a.o -c a.cpp
$ $R/clang++ -x c++ -std=c++14 -fPIC -flto -g -fno-exceptions -fno-rtti -c -o b.o -c b.cpp
$ $R/llvm-lto a.o b.o
DICompileUnit not listed in llvm.dbg.cu
!​35 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus, file: !​25, producer: "clang version 6.0.0 (trunk 315470) (llvm/trunk 315472)", isOptimized: true, runtimeVersion: 0, emissionKind: FullDebug, enums: !​2)
llvm-lto: warning loading file 'b.o': ignoring invalid debug info in b.o
...

Curious! After loading this into the debugger, I noticed that the linkonce_odr function _Z12consumeError5Error, which is present in both files, is deserialized in a most peculiar way: The BitcodeReader has a DeferredFunctionInfo map. The BitcodeReader stays alive between reading module A and module B. Module A is deserialized as usual. When module b is deserialized, the BitcodeReader object is reused and when it comes to materializing _Z12consumeError5Error it is found in the DeferredFunctionInfo, which points to the other module, A. The BitcodeReader then deserializes the function from module A(!) into module B(!), together with all the debug info metadata hanging off it, including the DICompileUnit that we are already familiar with from the warning.

This behavior seems really silly: The BitcodeReader is smart enough to know that it already found this function in another module and deserializes it from there only to later delete the function when it gets uniqued at module merge time. I'm guessing the the BitcodeReader's behavior is more a side-effect of the implementation than an intentional behavior, but I wonder if we could just not make it materialize linkonce_odr functions that were already visited, if we know that we are doing LTO. I guess this optimization would also benefit ThinLTO.

@adrian-prantl
Copy link
Collaborator Author

assigned to @adrian-prantl

@zmodem
Copy link
Collaborator

zmodem commented Oct 16, 2017

We're seeing this in Chromium builds too fwiv.

@adrian-prantl
Copy link
Collaborator Author

It looks like my analysis was premature. The BitcodeReader actually does get destroyed after parsing a module. What causes this symptom is that when materializing the debug info for the linkonce_odr function in the second module, we make a call to DIBuilder::buildODRType, which uses the LLVMContext to look up whether we already built a unique type with the same UID and this returns the type from the first module (which pulls in the DICompileUnit from the first module as well).

There is still a performance improvement to be made by not materializing linkonce_odr functions more than once, but at least we are deserializing them from the correct module now.

@adrian-prantl
Copy link
Collaborator Author

Why does a DICompositeType pull in a DICompileUnit?

class Error {};
template
void handleAllErrors( HandlerTs Handlers) {}
inline void consumeError(Error Err) {
handleAllErrors( {});
}

Because the concrete type of the template parameter "HandlerTS" (the type of the lambda) has a scope of "(void)consumeError(Error)":

!​21 = distinct !DISubprogram(name: "consumeError", linkageName: "_Z12consumeError5Error", scope: !​1, file: !​1, line: 4, type: !​22, isLocal: false, isDefinition: true, scopeLine: 4, flags: DIFlagPrototyped, isOptimized: false, unit: !​0, variables: !​2)
!​29 = distinct !DISubprogram(name: "handleAllErrors<(lambda at ar.ii:5:20)>", linkageName: "Z15handleAllErrorsIZ12consumeError5ErrorEUlvE_EvT", scope: !​1, file: !​1, line: 3, type: !​30, isLocal: false, isDefinition: true, scopeLine: 3, flags: DIFlagPrototyped, isOptimized: false, unit: !​0, templateParams: !​33, variables: !​2)
!​30 = !DISubroutineType(types: !​31)
!​31 = !{null, !​32}
!​32 = distinct !DICompositeType(tag: DW_TAG_class_type, scope: !​21, file: !​1, line: 5, size: 8, elements: !​2, identifier: "ZTSZ12consumeError5ErrorEUlvE")
!​33 = !{#34}
!​34 = !DITemplateTypeParameter(name: "HandlerTs", type: !​32)

@adrian-prantl
Copy link
Collaborator Author

Since this is far less critical than expected I opted to just teach the Verifier about this situation in r316049.

@llvmbot llvmbot transferred this issue from llvm/llvm-bugzilla-archive Dec 10, 2021
@llvmbot llvmbot added the confirmed Verified by a second party label Jan 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugzilla Issues migrated from bugzilla confirmed Verified by a second party llvm:bitcode
Projects
None yet
Development

No branches or pull requests

3 participants