-
Notifications
You must be signed in to change notification settings - Fork 12.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
"Mismatched function data" warnings when collecting profiles for clang #32849
Comments
assigned to @vedantk |
We've noticed similar issue in Chrome: https://bugs.chromium.org/p/chromium/issues/detail?id=764484 We verified the following:
However, we are getting the warning + coverage report doesn't show anything for the files we are interested in. It's worth to mention that the target is fairly large (CSS parser that is linked with lots of Blink stuff). |
llvm-cov's diagnostic for this problem is a bit better now (r313853). If you pass "-dump" along with your regular llvm-cov invocation, you should see messages like: hash-mismatch: No profile record found for 'main' with hash = 0xA I've added some explanations of what these mismatches mean in the doxygen comments for the CoverageMapping class. Hopefully that will make it easier to figure out which object file contribute profile records with unexpected hashes. I've also just built ToT clang, run "clang -help", merged the raw profile, and invoked llvm-cov. I get no function mismatches. Merging in profiles from a few additional clang invocations is also no issue. We get mismatches only after merging in profiles from other tools (llvm-tblgen, llc, FileCheck, etc). To avoid this problem, we could name profiles in a way that makes it hard to merge together profiles for different binaries. E.g, that means teaching FileCheck to write to "FileCheck_%m.profraw", llc to "llc_%m.profraw", etc. But before doing that, we should narrow down the cause of the mismatches. |
Thanks for clarifications, Vedant! We'll take a closer look using the ToT revision and "-dump" option. Will post an update after that. |
Forwarding a comment from Chromium bug tracker: I have built ToT llvm and clang and used them to build my fuzzer and generate the coverage report, and the issue remains. After using the -dump option with llvm-cov, it complains that 148 functions have mismatched data and then prints 148 lines similar to the one below: hash-mismatch: No profile record found for '_ZN7logging11CheckGEImplEiiPKc' with hash = 0x0 In all of these cases, the hash was 0x0 and the function name was some kind of mangled name (like the one above) beginning with the prefix "_ZN" |
The root cause seems to be a 0 hash (see https://bugs.chromium.org/p/chromium/issues/detail?id=764484#c14) Also Jonathan came up with a small reproducer: printf '#include \n void h() {std::cout << 1 << std::endl;} int main() {h();h();h();return 0;}' > test.cc; clang++ -fprofile-instr-generate -fcoverage-mapping test.cc -o test; LLVM_PROFILE_FILE=test.profdata ./test >/dev/null; llvm-profdata show test.profdata -all-functions The output of which is: Counters: llvm-cov doesn't complain though |
It's OK for functions to have a hash equal to 0. The question is, why does the profile record found for '_ZN7logging11CheckGEImplEiiPKc' not have a hash equal to 0? |
Vedant, it looks like we've been seeing that issue even with fresh clang revision because we had some dynamic libraries linked in, instead of using static linking. We cannot reproduce it anymore since we started explicitly enable static linking for our targets. Thanks a lot for your help! |
I'm glad you are unblocked but am worried that there are issues using dylibs. Please do let me know if you see more instances of suspicious hash mismatches, and I'll try and set aside more time to reduce the issue. |
@vedantk, this is most definitely an issue, even now in 2023. The gentleman who said he saw the problem go away when doing static linking is only covering up the problem. In our project, we cannot use static linking, nor would we want to. I'm happy to help you diagnose the problem if I can. I'm simply trying to use the llvm-cov tool, which works great for static linking, but is having this problem with dynamic linking of all sorts... not just dylib on Mac. I'm on Windows, using DLL, and it also happens. I am using the latest versions of everything, using clang++ compiler, lld-link linker, and of course llvm-cov to process coverage instrumentation. Please advise on this, since it has been 6 years since the last communication here. |
@vedantk I should also say that the report for coverage of my DLL is missing the subdirectories of files in the report. There is a main src directory in the DLL, and then several subdirectories off of it. The report only incudes the classes in the main directory, and none of the ones in the subdirectories. |
I am getting this warning in Rust, with
If that matters, this is on a Apple M3 Pro with Sonoma 14.1 (23B2073). Edit: it does not seem to happen on Linux (Debian testing) Note: If By expanding the macros, I was able to further reduce it to: // src/lib.rs
fn f() {
static __CALLSITE: ::tracing::callsite::DefaultCallsite = {
static META: ::tracing::Metadata<'static> = ::tracing_core::metadata::Metadata::new(
"",
"",
::tracing::Level::DEBUG,
None,
None,
None,
::tracing_core::field::FieldSet::new(
&[""],
::tracing_core::callsite::Identifier(&__CALLSITE),
),
::tracing::metadata::Kind::EVENT,
);
::tracing::callsite::DefaultCallsite::new(&META)
};
} And // tests/test.rs
#[test]
pub fn f() {
// warning: 2 functions have mismatched data
// let _ = ::tracing::Level::DEBUG <= ::tracing::level_filters::LevelFilter::current();
// warning: 1 functions have mismatched data
::tracing::level_filters::LevelFilter::current();
// warning: 1 functions have mismatched data
// let _ = ::tracing::Level::DEBUG <= ::tracing::Level::DEBUG;
} The minimum working example for #[inline(always)]
pub fn current() -> Self {
// standard indent
// non-standard indent
Self::OFF
} |
Hello, |
I've described my issue further on llvm-discourse |
To others with this issue coming here from Google, |
Hey folks, it appears that we've also hit the same issue. I've described my situation in https://discourse.llvm.org/t/source-based-coverage-results-degradation-between-clang-14-and-clang-18/84502/4, and it took a while to connect the dots and find this report. In our case using What makes things worse for us is that a lot of the coverage machinery is burried under layers of Bazel logic, so it's not straighforward to find this bug in the first place, because the only thing you see is a confusingly wrong coverage report. I don't really know much about LLVM, but is there something I can help with to make progress and resolve this issue? It appears that the last attempt to do something about it was #97574 and it didn't exactly work out, given that it's been a while, I'm just looking for ways to make progress on this. |
Extended Description
After building some llvm tools with -fcoverage-mapping -fprofile-instr-generate, running the tools, and merging the resulting raw profiles, there are "N functions have mismatched data" warnings.
Investigate the source of these warnings, and improve the diagnostic to be more helpful/precise.
The text was updated successfully, but these errors were encountered: