Switching the machine verifier ON by default (lib/CodeGen/Passes.cpp then change the default value of the cl::opt related to the machine verifier) leads to a bunch of machine verifier errors for make check: LLVM :: CodeGen/X86/cleanuppad-inalloca.ll LLVM :: CodeGen/X86/cmpxchg-clobber-flags.ll LLVM :: CodeGen/X86/cxx_tlscc64.ll LLVM :: CodeGen/X86/darwin-tls.ll LLVM :: CodeGen/X86/fast-isel-nontemporal.ll LLVM :: CodeGen/X86/i386-tlscall-fastregalloc.ll LLVM :: CodeGen/X86/implicit-null-check.ll LLVM :: CodeGen/X86/inalloca-ctor.ll LLVM :: CodeGen/X86/inalloca-invoke.ll LLVM :: CodeGen/X86/inalloca-regparm.ll LLVM :: CodeGen/X86/inalloca-stdcall.ll LLVM :: CodeGen/X86/inalloca.ll LLVM :: CodeGen/X86/machine-sink-and-implicit-null-checks.ll LLVM :: CodeGen/X86/musttail-indirect.ll LLVM :: CodeGen/X86/musttail-thiscall.ll LLVM :: CodeGen/X86/musttail-varargs.ll LLVM :: CodeGen/X86/peephole-na-phys-copy-folding.ll LLVM :: CodeGen/X86/pku.ll LLVM :: CodeGen/X86/rtm.ll LLVM :: CodeGen/X86/scheduler-backtracking.ll LLVM :: CodeGen/X86/shrink-wrap-chkstk.ll LLVM :: CodeGen/X86/sibcall-2.ll LLVM :: CodeGen/X86/sibcall.ll LLVM :: CodeGen/X86/stack-folding-fp-avx1.ll LLVM :: CodeGen/X86/stack-folding-fp-sse42.ll LLVM :: CodeGen/X86/tls-models.ll LLVM :: CodeGen/X86/tlv-1.ll LLVM :: CodeGen/X86/tlv-2.ll LLVM :: CodeGen/X86/win_coreclr_chkstk.ll LLVM :: CodeGen/X86/x32-function_pointer-1.ll LLVM :: CodeGen/X86/x86-64-stack-and-frame-ptr.ll LLVM :: CodeGen/X86/x86-interrupt_vzeroupper.ll LLVM :: CodeGen/X86/x86-shrink-wrapping.ll This may be latent bugs.
CodeGen/X86/cleanuppad-inalloca.ll is a fun one. The inalloca attribute says that the callee will make the alloca, but the caller will clean it. We end up with the machine verifier complaining about a destroy frame before a setup frame and there is currently no way to tell the verifier that those bits are actually allocated by the callee. I think the proper fix is to add to the stackadjust down instruction an argument with the number of bytes that are push by the callee, similar to what we do with stackadjust up and the number of bytes pop by the callee. That would take a bit of effort because currently the only requirement for the frame setup and frame destroy instructions is to haveone immediate operand. Thus, we need to update all the backends and lowering to make this two.
Fix test/CodeGen/X86/cmpxchg-clobber-flags.ll in r267623. (+ some preparatory patches in r267621 and r267622.) We now generate better code, because we do not try to push/pop RAX if it is not live. The problem was that we were saving a dead register and the verifier requires to have an undef flag to mark that we know we are reading an undef value.
Fixed CodeGen/X86/cxx_tlscc64.ll in r267797. We need to glue the callseq_end to the tlscall, otherwise the target independent code sets a dead flag on the result register whereas it is not dead. Indeed, the target independent code walks the chain of glue to find the users of physical registers.
Fixed CodeGen/X86/fast-isel-nontemporal.ll in r267806. We were using the wrong register class for source of the non-temporal store: FR32 instead of FR128. This did not cause any problem because contain the same registers behind the scene.
Partially fixed test/CodeGen/X86/implicit-null-check.ll in r267817 and r268327. The problems were respectively that we were missing to update the liveness properply and that we failed to mark FAULTING_LOAD_OP as potential branch instructions. We still have machine verifier failures on that tests though.
Fixed the final problem with implicit-null-cheks.ll in r268412. When I fixed the liveness update, I did not take into account that load instructions might have implicit-defs as well.
Fixed x86-interrupt_vzeroupper.ll in r268983. This one was pretty bad because we basically ended up having the AVX512 registers as callee saved registers whereas the target was not using AVX512!!
(In reply to comment #7) > Fixed x86-interrupt_vzeroupper.ll in r268983. This one was pretty bad > because we basically ended up having the AVX512 registers as callee saved > registers whereas the target was not using AVX512!! I think this fixes PR26409 as well?
(In reply to comment #8) > (In reply to comment #7) > > Fixed x86-interrupt_vzeroupper.ll in r268983. This one was pretty bad > > because we basically ended up having the AVX512 registers as callee saved > > registers whereas the target was not using AVX512!! > > I think this fixes PR26409 as well? That may indeed be the case. Could you double check, please?
Here is where we are now: Failing Tests (10): LLVM :: CodeGen/X86/inalloca-ctor.ll LLVM :: CodeGen/X86/local_stack_symbol_ordering.ll LLVM :: CodeGen/X86/pku.ll LLVM :: CodeGen/X86/rtm.ll LLVM :: CodeGen/X86/scheduler-backtracking.ll LLVM :: CodeGen/X86/sibcall-2.ll LLVM :: CodeGen/X86/sibcall.ll LLVM :: CodeGen/X86/stack-folding-fp-avx1.ll LLVM :: CodeGen/X86/stack-folding-fp-sse42.ll LLVM :: CodeGen/X86/win_coreclr_chkstk.ll (Plus the inalloca tests that I analyzed previously but did not fix.)
(In reply to comment #10) > LLVM :: CodeGen/X86/stack-folding-fp-avx1.ll > LLVM :: CodeGen/X86/stack-folding-fp-sse42.ll A candidate fix for these is at: https://reviews.llvm.org/D23276
The stack folding tests are now fixed. The remaining issues are: Failing Tests (16): LLVM :: CodeGen/X86/cleanuppad-inalloca.ll LLVM :: CodeGen/X86/inalloca-ctor.ll LLVM :: CodeGen/X86/inalloca-invoke.ll LLVM :: CodeGen/X86/inalloca-regparm.ll LLVM :: CodeGen/X86/inalloca-stdcall.ll LLVM :: CodeGen/X86/inalloca.ll LLVM :: CodeGen/X86/local_stack_symbol_ordering.ll LLVM :: CodeGen/X86/musttail-indirect.ll LLVM :: CodeGen/X86/musttail-thiscall.ll LLVM :: CodeGen/X86/rtm.ll LLVM :: CodeGen/X86/scheduler-backtracking.ll LLVM :: CodeGen/X86/shrink-wrap-chkstk.ll LLVM :: CodeGen/X86/sibcall-2.ll LLVM :: CodeGen/X86/sibcall.ll LLVM :: CodeGen/X86/sjlj-eh.ll LLVM :: CodeGen/X86/win_coreclr_chkstk.ll
Current failing tests: LLVM :: CodeGen/X86/branchfolding-undef.mir LLVM :: CodeGen/X86/cleanuppad-inalloca.ll LLVM :: CodeGen/X86/inalloca-ctor.ll LLVM :: CodeGen/X86/inalloca-invoke.ll LLVM :: CodeGen/X86/inalloca-regparm.ll LLVM :: CodeGen/X86/inalloca-stdcall.ll LLVM :: CodeGen/X86/inalloca.ll LLVM :: CodeGen/X86/local_stack_symbol_ordering.ll LLVM :: CodeGen/X86/musttail-indirect.ll LLVM :: CodeGen/X86/musttail-thiscall.ll LLVM :: CodeGen/X86/rtm.ll LLVM :: CodeGen/X86/scheduler-backtracking.ll LLVM :: CodeGen/X86/shrink-wrap-chkstk.ll LLVM :: CodeGen/X86/sibcall-2.ll LLVM :: CodeGen/X86/sibcall.ll LLVM :: CodeGen/X86/sjlj-eh.ll LLVM :: CodeGen/X86/sse-regcall.ll LLVM :: CodeGen/X86/win_coreclr_chkstk.ll LLVM :: CodeGen/X86/xray-multiplerets-in-blocks.mir
Failing Tests (19): LLVM :: CodeGen/X86/branchfolding-undef.mir LLVM :: CodeGen/X86/cleanuppad-inalloca.ll LLVM :: CodeGen/X86/fast-isel-select-sse.ll LLVM :: CodeGen/X86/inalloca-ctor.ll LLVM :: CodeGen/X86/inalloca-invoke.ll LLVM :: CodeGen/X86/inalloca-regparm.ll LLVM :: CodeGen/X86/inalloca-stdcall.ll LLVM :: CodeGen/X86/inalloca.ll LLVM :: CodeGen/X86/local_stack_symbol_ordering.ll LLVM :: CodeGen/X86/musttail-indirect.ll LLVM :: CodeGen/X86/musttail-thiscall.ll LLVM :: CodeGen/X86/rtm.ll LLVM :: CodeGen/X86/scheduler-backtracking.ll LLVM :: CodeGen/X86/shrink-wrap-chkstk.ll LLVM :: CodeGen/X86/sibcall-2.ll LLVM :: CodeGen/X86/sibcall.ll LLVM :: CodeGen/X86/sjlj-eh.ll LLVM :: CodeGen/X86/win_coreclr_chkstk.ll LLVM :: CodeGen/X86/xray-multiplerets-in-blocks.mir
> LLVM :: CodeGen/X86/fast-isel-select-sse.ll Candidate Fix: https://reviews.llvm.org/D31200
Fixed fast-isel-select-sse.ll by rL298805. Failing Tests (18): LLVM :: CodeGen/X86/branchfolding-undef.mir LLVM :: CodeGen/X86/cleanuppad-inalloca.ll LLVM :: CodeGen/X86/inalloca-ctor.ll LLVM :: CodeGen/X86/inalloca-invoke.ll LLVM :: CodeGen/X86/inalloca-regparm.ll LLVM :: CodeGen/X86/inalloca-stdcall.ll LLVM :: CodeGen/X86/inalloca.ll LLVM :: CodeGen/X86/local_stack_symbol_ordering.ll LLVM :: CodeGen/X86/musttail-indirect.ll LLVM :: CodeGen/X86/musttail-thiscall.ll LLVM :: CodeGen/X86/rtm.ll LLVM :: CodeGen/X86/scheduler-backtracking.ll LLVM :: CodeGen/X86/shrink-wrap-chkstk.ll LLVM :: CodeGen/X86/sibcall-2.ll LLVM :: CodeGen/X86/sibcall.ll LLVM :: CodeGen/X86/sjlj-eh.ll LLVM :: CodeGen/X86/win_coreclr_chkstk.ll LLVM :: CodeGen/X86/xray-multiplerets-in-blocks.mir
For the inalloca tests, we have a candidate fix by Serge Pavlov: https://reviews.llvm.org/D32394 Reviewers needed!
Serge's alloca fix landed at rL302527. Failing Tests (9): LLVM :: CodeGen/X86/branchfolding-undef.mir LLVM :: CodeGen/X86/local_stack_symbol_ordering.ll LLVM :: CodeGen/X86/rtm.ll LLVM :: CodeGen/X86/scheduler-backtracking.ll LLVM :: CodeGen/X86/sibcall-2.ll LLVM :: CodeGen/X86/sibcall.ll LLVM :: CodeGen/X86/sjlj-eh.ll LLVM :: CodeGen/X86/win_coreclr_chkstk.ll LLVM :: CodeGen/X86/xray-multiplerets-in-blocks.mir
CC'ing Igor as he added the GlobalISel tests recently Current failing tests: LLVM :: CodeGen/X86/GlobalISel/ext.ll LLVM :: CodeGen/X86/GlobalISel/select-trunc.mir LLVM :: CodeGen/X86/GlobalISel/trunc.ll LLVM :: CodeGen/X86/O0-pipeline.ll LLVM :: CodeGen/X86/branchfolding-undef.mir LLVM :: CodeGen/X86/local_stack_symbol_ordering.ll LLVM :: CodeGen/X86/scheduler-backtracking.ll LLVM :: CodeGen/X86/sibcall-2.ll LLVM :: CodeGen/X86/sibcall.ll LLVM :: CodeGen/X86/sjlj-eh.ll LLVM :: CodeGen/X86/win_coreclr_chkstk.ll LLVM :: CodeGen/X86/xray-multiplerets-in-blocks.mir LLVM :: DebugInfo/MIR/X86/bit-piece-dh.mir
GlobalISel tests fixed in r303502. Thanks for pointing it out to me!
Thanks Igor. Failing tests: LLVM :: CodeGen/X86/O0-pipeline.ll LLVM :: CodeGen/X86/branchfolding-undef.mir LLVM :: CodeGen/X86/local_stack_symbol_ordering.ll LLVM :: CodeGen/X86/scheduler-backtracking.ll LLVM :: CodeGen/X86/sibcall-2.ll LLVM :: CodeGen/X86/sibcall.ll LLVM :: CodeGen/X86/sjlj-eh.ll LLVM :: CodeGen/X86/win_coreclr_chkstk.ll LLVM :: CodeGen/X86/xray-multiplerets-in-blocks.mir LLVM :: DebugInfo/MIR/X86/bit-piece-dh.mir
r304320 enabled the machine verifier by default with EXPENSIVE_CHECKS. X86 is one of the targets excluded from this by overriding TargetMachine::isMachineVerifierClean() to return false. Please remove the override when the target at least passes all lit tests without machine verifier failure.
Updated list, as it's been a while: Failing Tests (14): LLVM :: CodeGen/Generic/zero-probability.mir LLVM :: CodeGen/MIR/X86/diexpr-win32.mir LLVM :: CodeGen/X86/O0-pipeline.ll LLVM :: CodeGen/X86/avx512-regcall-NoMask.ll LLVM :: CodeGen/X86/branchfolding-undef.mir LLVM :: CodeGen/X86/local_stack_symbol_ordering.ll LLVM :: CodeGen/X86/scheduler-backtracking.ll LLVM :: CodeGen/X86/sibcall-2.ll LLVM :: CodeGen/X86/sibcall.ll LLVM :: CodeGen/X86/sjlj-eh.ll LLVM :: CodeGen/X86/win_coreclr_chkstk.ll LLVM :: CodeGen/X86/xray-multiplerets-in-blocks.mir LLVM :: DebugInfo/MIR/X86/bit-piece-dh.mir LLVM :: DebugInfo/MIR/X86/empty-inline.mir
Failing Tests (24): LLVM :: CodeGen/Generic/zero-probability.mir LLVM :: CodeGen/X86/O0-pipeline.ll LLVM :: CodeGen/X86/O3-pipeline.ll LLVM :: CodeGen/X86/PR37310.mir LLVM :: CodeGen/X86/avx512-regcall-NoMask.ll LLVM :: CodeGen/X86/branch_instruction_and_target_split_perf_nops.mir LLVM :: CodeGen/X86/branchfolding-undef.mir LLVM :: CodeGen/X86/fentry-insertion.ll LLVM :: CodeGen/X86/icall-branch-funnel.ll LLVM :: CodeGen/X86/indirect-branch-tracking.ll LLVM :: CodeGen/X86/local_stack_symbol_ordering.ll LLVM :: CodeGen/X86/machine-outliner-disubprogram.ll LLVM :: CodeGen/X86/machine-outliner-tailcalls.ll LLVM :: CodeGen/X86/patchable-prologue.ll LLVM :: CodeGen/X86/retpoline-regparm.ll LLVM :: CodeGen/X86/retpoline.ll LLVM :: CodeGen/X86/scheduler-backtracking.ll LLVM :: CodeGen/X86/shadow-stack.ll LLVM :: CodeGen/X86/sibcall-2.ll LLVM :: CodeGen/X86/sibcall.ll LLVM :: CodeGen/X86/sjlj-eh.ll LLVM :: CodeGen/X86/win_coreclr_chkstk.ll LLVM :: DebugInfo/Generic/linear-dbg-value.ll LLVM :: DebugInfo/X86/live-debug-vars-discard-invalid.mir
@jpaquette Any idea whats behind the machine outliner test failures? e.g. CodeGen\X86\machine-outliner-disubprogram.ll # After Machine Outliner # Machine code for function f1: NoPHIs, TracksLiveness, NoVRegs Frame Objects: fi#-1: size=8, align=16, fixed, at location [SP-8] fi#0: size=4, align=4, at location [SP-12] fi#1: size=4, align=4, at location [SP-20] fi#2: size=4, align=4, at location [SP-16] bb.0.entry: frame-setup PUSH64r killed $rbp, implicit-def $rsp, implicit $rsp CFI_INSTRUCTION def_cfa_offset 16 CFI_INSTRUCTION offset $rbp, -16 $rbp = frame-setup MOV64rr $rsp CFI_INSTRUCTION def_cfa_register $rbp $rsp = frame-setup SUB64ri8 $rsp, 12, implicit-def dead $eflags CALL64pcrel32 @OUTLINED_FUNCTION_1, implicit $rsp, implicit $ssp $rsp = frame-destroy ADD64ri8 $rsp, 12, implicit-def dead $eflags, debug-location !46; test.c:26:3 $rbp = frame-destroy POP64r implicit-def $rsp, implicit $rsp, debug-location !46; test.c:26:3 RETQ implicit $eax, debug-location !46; test.c:26:3 # End machine code for function f1. *** Bad machine code: Using an undefined physical register *** - function: f1 - basic block: %bb.0 entry (0x1b5eec123b8) - instruction: RETQ implicit $eax, debug-location !46; test.c:26:3 - operand 0: implicit $eax LLVM ERROR: Found 1 machine code errors. error: command failed with exit status: 1
Possible fix: https://reviews.llvm.org/D49295
Possible fix for the tail call outliner issue as well: https://reviews.llvm.org/D49299
I think Francis' fixes should resolve the outliner failures once he lands them.
Failing Tests (28): LLVM :: CodeGen/Generic/zero-probability.mir LLVM :: CodeGen/X86/O0-pipeline.ll LLVM :: CodeGen/X86/O3-pipeline.ll LLVM :: CodeGen/X86/PR37310.mir LLVM :: CodeGen/X86/avx512-regcall-NoMask.ll LLVM :: CodeGen/X86/branch_instruction_and_target_split_perf_nops.mir LLVM :: CodeGen/X86/branchfolding-undef.mir LLVM :: CodeGen/X86/code-model-elf-memset.ll LLVM :: CodeGen/X86/code-model-elf.ll LLVM :: CodeGen/X86/fentry-insertion.ll LLVM :: CodeGen/X86/icall-branch-funnel.ll LLVM :: CodeGen/X86/indirect-branch-tracking.ll LLVM :: CodeGen/X86/large-pic-string.ll LLVM :: CodeGen/X86/patchable-prologue.ll LLVM :: CodeGen/X86/pr38795.ll LLVM :: CodeGen/X86/retpoline-regparm.ll LLVM :: CodeGen/X86/retpoline.ll LLVM :: CodeGen/X86/scheduler-backtracking.ll LLVM :: CodeGen/X86/sibcall-2.ll LLVM :: CodeGen/X86/sibcall.ll LLVM :: CodeGen/X86/sjlj-eh.ll LLVM :: CodeGen/X86/speculative-load-hardening-indirect.ll LLVM :: CodeGen/X86/win_coreclr_chkstk.ll LLVM :: DebugInfo/Generic/linear-dbg-value.ll LLVM :: DebugInfo/X86/live-debug-vars-discard-invalid.mir LLVM :: LTO/X86/codemodel-2.ll LLVM :: ThinLTO/X86/cfi-devirt.ll LLVM :: ThinLTO/X86/devirt-after-icp.ll
After r345197 I see only 20 failures: Failing Tests (20): LLVM :: CodeGen/X86/O0-pipeline.ll LLVM :: CodeGen/X86/O3-pipeline.ll LLVM :: CodeGen/X86/PR37310.mir LLVM :: CodeGen/X86/avx512-regcall-NoMask.ll LLVM :: CodeGen/X86/branch_instruction_and_target_split_perf_nops.mir LLVM :: CodeGen/X86/code-model-elf-memset.ll LLVM :: CodeGen/X86/code-model-elf.ll LLVM :: CodeGen/X86/fentry-insertion.ll LLVM :: CodeGen/X86/icall-branch-funnel.ll LLVM :: CodeGen/X86/indirect-branch-tracking.ll LLVM :: CodeGen/X86/large-pic-string.ll LLVM :: CodeGen/X86/memcmp-mergeexpand.ll LLVM :: CodeGen/X86/patchable-prologue.ll LLVM :: CodeGen/X86/pr38795.ll LLVM :: CodeGen/X86/retpoline-regparm.ll LLVM :: CodeGen/X86/retpoline.ll LLVM :: CodeGen/X86/scheduler-backtracking.ll LLVM :: CodeGen/X86/sjlj-eh.ll LLVM :: CodeGen/X86/speculative-load-hardening-indirect.ll LLVM :: CodeGen/X86/win_coreclr_chkstk.ll And I have a fix for retpoline in flight: https://reviews.llvm.org/D53653 Now that we're down to such a small number, maybe we should explicitly annotate just the failing tests as -verify-machineinstrs=0, and mark the x86 target as machine verifier clean to prevent regressions?
(In reply to Reid Kleckner from comment #30) > After r345197 I see only 20 failures: > > .... > > And I have a fix for retpoline in flight: > https://reviews.llvm.org/D53653 > > Now that we're down to such a small number, maybe we should explicitly > annotate just the failing tests as -verify-machineinstrs=0, and mark the x86 > target as machine verifier clean to prevent regressions? Yup, that would make sense after D53653 has landed, I'd vote to keep this bug open after that though to track these remaining issues.
+1 for Reid's suggestion (we should also be careful about tests in DebugInfo/X86 which have some MIR tests) * r345266 should fix PR37310.mir. * contacted original author about branch_instruction_and_target_split_perf_nops.mir
I filed bugs/patches for the existing issues: LLVM :: CodeGen/X86/avx512-regcall-NoMask.ll: https://bugs.llvm.org/show_bug.cgi?id=39437 LLVM :: CodeGen/X86/branch_instruction_and_target_split_perf_nops.mir: https://reviews.llvm.org/D53767 LLVM :: CodeGen/X86/icall-branch-funnel.ll: https://bugs.llvm.org/show_bug.cgi?id=39436 LLVM :: CodeGen/X86/indirect-branch-tracking.ll: https://bugs.llvm.org/show_bug.cgi?id=39439 LLVM :: CodeGen/X86/pr38795.ll: https://bugs.llvm.org/show_bug.cgi?id=39440 LLVM :: CodeGen/X86/retpoline-regparm.ll: https://reviews.llvm.org/D53653, https://bugs.llvm.org/show_bug.cgi?id=38391 LLVM :: CodeGen/X86/retpoline.ll: https://reviews.llvm.org/D53653, https://bugs.llvm.org/show_bug.cgi?id=38391 LLVM :: CodeGen/X86/scheduler-backtracking.ll: https://bugs.llvm.org/show_bug.cgi?id=39452 LLVM :: CodeGen/X86/sjlj-eh.ll: https://bugs.llvm.org/show_bug.cgi?id=39439 LLVM :: CodeGen/X86/speculative-load-hardening-indirect.ll: https://bugs.llvm.org/show_bug.cgi?id=39451 LLVM :: CodeGen/X86/win_coreclr_chkstk.ll: https://bugs.llvm.org/show_bug.cgi?id=38376 We should be ready to turn it on by default.
Committed the switch as https://reviews.llvm.org/rL345513. Will keep an eye on the bots. Feel free to revert if I'm not reactive enough.
(In reply to Francis Visoiu Mistrih from comment #35) > Committed the switch as https://reviews.llvm.org/rL345513. Will keep an eye > on the bots. Feel free to revert if I'm not reactive enough. I've put isMachineVerifierClean back at rL345528 to fix some ThinLTO test failures - I've left your test files changes though. Please can you take a look? http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/13629/steps/test-check-all/logs/stdio
(In reply to Simon Pilgrim from comment #36) > Please can you take a look? > > http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/ > builds/13629/steps/test-check-all/logs/stdio Thanks! * Clang fix: r345591 * LLVM fix + re-enable: r345593
(In reply to Francis Visoiu Mistrih from comment #37) > Thanks! > > * Clang fix: r345591 > * LLVM fix + re-enable: r345593 Marking fixed. Separately, I was wondering if maybe the machine verifier should be enabled somehow in +asserts builds, or controlled by some llc flag that's always set when lit calls llc, so that we get earlier detection of issues.