[DebugInfo@O2] Filtering out-of-scope variables drops many legitimate locations #47435

jmorse · 2020-11-05T22:08:39Z


Bugzilla Link	48091
Version	trunk
OS	Linux
Blocks	#30616
CC	@dwblaikie,@jdm

Extended Description

The VarLoc based LiveDebugValues implementation has a filter in the 'join' method, to cease propagating variable locations that have gone out of scope. I believe the aim of this is to prevent LiveDebugValues needlessly computing the location of every variable for every instruction, even those that are only in scope for a few instructions.

Clearly this is effective at moderating compile times, however it also seems to damage location coverage too. Specifically: if you have, say, an X86 CMOV64 instruction:
CMP64rr $rbx, $rcx, implicit-def $eflags
$rbx = CMOV64rr $rbx (tied-def 0), $rcx, 7, implicit $eflags, debug-location !1

Then occasionally the X86 backend will implement this with control flow:
CMP64rr $rbx, $rcx, implicit-def $eflags
JCC_1 %bb.2, 7, debug-location !1

bb.1:
$rbx = COPY $rcx, debug-location !1

bb.2:
...

Unfortunately, this effectively becomes a notch filter for variables that are in scope for DILocation !1. Any variables that aren't in scope in bb.1 will be dropped, and subsequently cause the same variables to be dropped from bb.2 onwards. That means any LLVM-IR select instruction can potentially lead to neighbouring variable locations being dropped, depending on their scopes. In terms of impact, here's llvm-locstats for a clang-3.4 build with and without the out-of-scope filter. With filter:

=================================================
cov% samples percentage(~)

0% 765406 22%
(0%,10%) 45179 1%
[10%,20%) 51699 1%
[20%,30%) 52044 1%
[30%,40%) 46905 1%
[40%,50%) 48292 1%
[50%,60%) 61342 1%
[60%,70%) 58315 1%
[70%,80%) 69848 2%
[80%,90%) 81937 2%
[90%,100%) 101384 2%
100% 2032034 59%

-the number of debug variables processed: 3414385
-PC ranges covered: 61%

-total availability: 64%

Without:

=================================================
cov% samples percentage(~)

0% 765357 22%
(0%,10%) 45168 1%
[10%,20%) 51584 1%
[20%,30%) 51911 1%
[30%,40%) 46569 1%
[40%,50%) 47978 1%
[50%,60%) 60356 1%
[60%,70%) 53071 1%
[70%,80%) 56018 1%
[80%,90%) 79131 2%
[90%,100%) 103664 3%
100% `2059075` 60%

-the number of debug variables processed: 3419882
-PC ranges covered: 61%

-total availability: 65%

Importantly, there are an additional ~27k variables in the 100% range bucket, mostly moved up from the 60-99% ranges. Presumably these are long lived variables that have their lifetimes artificially interrupted by the scope filter. A similar amount of improvement occurs for InstrRefBasedLDV.

IMO, this is eminently solvable in InstrRefBasedLDV -- as it deals with things in terms of transfer functions, we should be able to invent transfer functions between isolated segments of particular scopes using dominance information.

The text was updated successfully, but these errors were encountered:

dwblaikie · 2020-11-05T22:37:18Z

Apologies for the naive interjection here, but do you have an example of the changes in the final DWARF? A small example would be great.

jmorse · 2020-11-06T19:40:28Z

Reproducer
I should probably know better than to open bugs without a reproducer by now; attaching some LLVM-IR. This is a function extracted from clang-3.4 and then llvm-reduce'd, so the code is nonsensical, has a lot of of undef in it, and synthesized debug-info metadata. Still, the point is:

There are three sibling lexical blocks in the metadata,
The "while.body" block is made up of instructions in block !101,
and has a variable location specified in it,
The intervening "loopexit" block has only instructions from block !102,
The "cleanup" block has instructions from !101 and !100
The variable location in "while.body" is not propagated through "loopexit" to
cover the in-scope instruction in ""cleanup"

Here's the assembly produced:
0: xor %eax,%eax
2: test %ecx,%ecx
4: jne 1a
6: nopw %cs:0x0(%rax,%rax,1)
d:
10: test %al,%al
12: jne 1a
14: mov (%rax),%ecx
16: test %ecx,%ecx
18: je 10
1a: retq

(Again, pretty meaningless). Here's the DWARF in LLVM as it stands today, for the lexical block with the variable:
DW_TAG_lexical_block
DW_AT_ranges (0x00000000
[0x0000000000000002, 0x0000000000000006)
[0x0000000000000014, 0x000000000000001a))

DW_TAG_variable
  DW_AT_location      (0x00000000:  
     [0x0000000000000002, 0x0000000000000006): DW_OP_reg2 RCX
     [0x0000000000000016, 0x000000000000001a): DW_OP_reg2 RCX)
  DW_AT_name  ("myVar")

And if you disable this block of code, from line 1650 to 1665 in

llvm-project/llvm/lib/CodeGen/LiveDebugValues/VarLocBasedImpl.cpp

Line 1650 in 16dccf7

// Filter out DBG_VALUES that are out of scope.

DW_TAG_lexical_block
DW_AT_ranges (0x00000000
[0x0000000000000002, 0x0000000000000006)
[0x0000000000000014, 0x000000000000001a))

DW_TAG_variable
  DW_AT_location      (0x00000000:  
     [0x0000000000000002, 0x0000000000000006): DW_OP_reg2 RCX
     [0x0000000000000014, 0x000000000000001a): DW_OP_reg2 RCX)
  DW_AT_name  ("myVar")

The variable location covers addresses 14->16 (the load / mov (%rax)), where it didn't before. The underlying reason for this is that we don't track that variable location across the instructions at 0x10 and 0x12. (The location can be found from 0x16 onwards due to some tail-duplication shenanigans).

I tried and failed to make a C reproducer to nicely demonstrate this; I also haven't examined the un-reduced IR to see whether it'd actually be observable / a problem for a developer. Probably the best test for whether this is a real problem would be finding the largest change in coverage caused by the scope filter and seeing if it's something a developer would be annoyed by. (I probably won't get to look at this for a while).

jmorse · 2020-11-06T19:42:21Z

The underlying reason for this is that we don't track that variable location across the instructions at 0x10 and 0x12.

(... which corresponds to the "loopexit" block in the LLVM-IR, for which all the instructions are outside the variables scope... )

jmorse · 2022-01-21T12:47:05Z

NB: some of this is fixed by https://reviews.llvm.org/D117877 , although not in the general case.

adrian-prantl mentioned this issue Dec 5, 2016

Umbrella: debug info for optimized code #30616

Open

llvmbot transferred this issue from llvm/llvm-bugzilla-archive Dec 10, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DebugInfo@O2] Filtering out-of-scope variables drops many legitimate locations #47435

[DebugInfo@O2] Filtering out-of-scope variables drops many legitimate locations #47435

jmorse commented Nov 5, 2020

dwblaikie commented Nov 5, 2020

jmorse commented Nov 6, 2020

jmorse commented Nov 6, 2020

jmorse commented Jan 21, 2022

[DebugInfo@O2] Filtering out-of-scope variables drops many legitimate locations #47435

[DebugInfo@O2] Filtering out-of-scope variables drops many legitimate locations #47435

Comments

jmorse commented Nov 5, 2020

Extended Description

================================================= cov% samples percentage(~)

0% 765406 22% (0%,10%) 45179 1% [10%,20%) 51699 1% [20%,30%) 52044 1% [30%,40%) 46905 1% [40%,50%) 48292 1% [50%,60%) 61342 1% [60%,70%) 58315 1% [70%,80%) 69848 2% [80%,90%) 81937 2% [90%,100%) 101384 2% 100% 2032034 59%

-the number of debug variables processed: 3414385 -PC ranges covered: 61%

-total availability: 64%

================================================= cov% samples percentage(~)

0% 765357 22% (0%,10%) 45168 1% [10%,20%) 51584 1% [20%,30%) 51911 1% [30%,40%) 46569 1% [40%,50%) 47978 1% [50%,60%) 60356 1% [60%,70%) 53071 1% [70%,80%) 56018 1% [80%,90%) 79131 2% [90%,100%) 103664 3% 100% 2059075 60%

-the number of debug variables processed: 3419882 -PC ranges covered: 61%

-total availability: 65%

dwblaikie commented Nov 5, 2020

jmorse commented Nov 6, 2020

jmorse commented Nov 6, 2020

jmorse commented Jan 21, 2022

=================================================
cov% samples percentage(~)

0% 765406 22%
(0%,10%) 45179 1%
[10%,20%) 51699 1%
[20%,30%) 52044 1%
[30%,40%) 46905 1%
[40%,50%) 48292 1%
[50%,60%) 61342 1%
[60%,70%) 58315 1%
[70%,80%) 69848 2%
[80%,90%) 81937 2%
[90%,100%) 101384 2%
100% 2032034 59%

-the number of debug variables processed: 3414385
-PC ranges covered: 61%

=================================================
cov% samples percentage(~)

0% 765357 22%
(0%,10%) 45168 1%
[10%,20%) 51584 1%
[20%,30%) 51911 1%
[30%,40%) 46569 1%
[40%,50%) 47978 1%
[50%,60%) 60356 1%
[60%,70%) 53071 1%
[70%,80%) 56018 1%
[80%,90%) 79131 2%
[90%,100%) 103664 3%
100% `2059075` 60%

-the number of debug variables processed: 3419882
-PC ranges covered: 61%