LLVM 20.0.0git
Macros | Functions | Variables
ARMLowOverheadLoops.cpp File Reference

Finalize v8.1-m low-overhead loops by converting the associated pseudo instructions into machine operations. More...

#include "ARM.h"
#include "ARMBaseInstrInfo.h"
#include "ARMBaseRegisterInfo.h"
#include "ARMBasicBlockInfo.h"
#include "ARMSubtarget.h"
#include "MVETailPredUtils.h"
#include "Thumb2InstrInfo.h"
#include "llvm/ADT/SetOperations.h"
#include "llvm/ADT/SetVector.h"
#include "llvm/CodeGen/LivePhysRegs.h"
#include "llvm/CodeGen/MachineFrameInfo.h"
#include "llvm/CodeGen/MachineFunctionPass.h"
#include "llvm/CodeGen/MachineLoopInfo.h"
#include "llvm/CodeGen/MachineLoopUtils.h"
#include "llvm/CodeGen/MachineRegisterInfo.h"
#include "llvm/CodeGen/Passes.h"
#include "llvm/CodeGen/ReachingDefAnalysis.h"
#include "llvm/MC/MCInstrDesc.h"

Go to the source code of this file.

Macros

#define DEBUG_TYPE   "arm-low-overhead-loops"
 
#define ARM_LOW_OVERHEAD_LOOPS_NAME   "ARM Low Overhead Loops pass"
 

Functions

static bool isVectorPredicated (MachineInstr *MI)
 
static bool isVectorPredicate (MachineInstr *MI)
 
static bool hasVPRUse (MachineInstr &MI)
 
static bool isDomainMVE (MachineInstr *MI)
 
static int getVecSize (const MachineInstr &MI)
 
static bool shouldInspect (MachineInstr &MI)
 
static bool isHorizontalReduction (const MachineInstr &MI)
 
 INITIALIZE_PASS (ARMLowOverheadLoops, DEBUG_TYPE, ARM_LOW_OVERHEAD_LOOPS_NAME, false, false) static bool TryRemove(MachineInstr *MI
 
 if (!RDA.isSafeToRemove(MI, Uses, Ignore)) return false
 
 if (WontCorruptITs(Uses, RDA))
 
static bool isRegInClass (const MachineOperand &MO, const TargetRegisterClass *Class)
 
static bool retainsPreviousHalfElement (const MachineInstr &MI)
 
static bool producesDoubleWidthResult (const MachineInstr &MI)
 
static bool canGenerateNonZeros (const MachineInstr &MI)
 
static bool producesFalseLanesZero (MachineInstr &MI, const TargetRegisterClass *QPRs, const ReachingDefAnalysis &RDA, InstSet &FalseLanesZero)
 
static bool ValidateMVEStore (MachineInstr *MI, MachineLoop *ML)
 

Variables

static cl::opt< boolDisableTailPredication ("arm-loloops-disable-tailpred", cl::Hidden, cl::desc("Disable tail-predication in the ARM LowOverheadLoop pass"), cl::init(false))
 
static cl::opt< boolDisableOmitDLS ("arm-disable-omit-dls", cl::Hidden, cl::desc("Disable omitting 'dls lr, lr' instructions"), cl::init(false))
 
ReachingDefAnalysisRDA
 
ReachingDefAnalysis InstSet & ToRemove
 
ReachingDefAnalysis InstSet InstSet & Ignore
 
SmallPtrSet< MachineInstr *, 2 > Uses
 
return false
 

Detailed Description

Finalize v8.1-m low-overhead loops by converting the associated pseudo instructions into machine operations.

The expectation is that the loop contains three pseudo instructions:

In addition to this, we also look for the presence of the VCTP instruction, which determines whether we can generated the tail-predicated low-overhead loop form.

Assumptions and Dependencies: Low-overhead loops are constructed and executed using a setup instruction: DLS, WLS, DLSTP or WLSTP and an instruction that loops back: LE or LETP. WLS(TP) and LE(TP) are branching instructions with a (large) limited range but fixed polarity: WLS can only branch forwards and LE can only branch backwards. These restrictions mean that this pass is dependent upon block layout and block sizes, which is why it's the last pass to run. The same is true for ConstantIslands, but this pass does not increase the size of the basic blocks, nor does it change the CFG. Instructions are mainly removed during the transform and pseudo instructions are replaced by real ones. In some cases, when we have to revert to a 'normal' loop, we have to introduce multiple instructions for a single pseudo (see RevertWhile and RevertLoopEnd). To handle this situation, t2WhileLoopStartLR and t2LoopEnd are defined to be as large as this maximum sequence of replacement instructions.

A note on VPR.P0 (the lane mask): VPT, VCMP, VPNOT and VCTP won't overwrite VPR.P0 when they update it in a "VPT Active" context (which includes low-overhead loops and vpt blocks). They will simply "and" the result of their calculation with the current value of VPR.P0. You can think of it like this:

/// if VPT active:    ; Between a DLSTP/LETP, or for predicated instrs
///   VPR.P0 &= Value
/// else
///   VPR.P0 = Value
/// 

When we're inside the low-overhead loop (between DLSTP and LETP), we always fall in the "VPT active" case, so we can consider that all VPR writes by one of those instruction is actually a "and".

Definition in file ARMLowOverheadLoops.cpp.

Macro Definition Documentation

◆ ARM_LOW_OVERHEAD_LOOPS_NAME

#define ARM_LOW_OVERHEAD_LOOPS_NAME   "ARM Low Overhead Loops pass"

Definition at line 76 of file ARMLowOverheadLoops.cpp.

◆ DEBUG_TYPE

#define DEBUG_TYPE   "arm-low-overhead-loops"

Definition at line 75 of file ARMLowOverheadLoops.cpp.

Function Documentation

◆ canGenerateNonZeros()

static bool canGenerateNonZeros ( const MachineInstr MI)
static

Definition at line 836 of file ARMLowOverheadLoops.cpp.

References MI, and producesDoubleWidthResult().

Referenced by producesFalseLanesZero().

◆ getVecSize()

static int getVecSize ( const MachineInstr MI)
static

◆ hasVPRUse()

static bool hasVPRUse ( MachineInstr MI)
static

Definition at line 97 of file ARMLowOverheadLoops.cpp.

References MI.

Referenced by shouldInspect().

◆ if() [1/2]

if ( !RDA.  isSafeToRemoveMI, Uses, Ignore)

◆ if() [2/2]

if ( WontCorruptITs(Uses, RDA )

◆ INITIALIZE_PASS()

INITIALIZE_PASS ( ARMLowOverheadLoops  ,
DEBUG_TYPE  ,
ARM_LOW_OVERHEAD_LOOPS_NAME  ,
false  ,
false   
)

◆ isDomainMVE()

static bool isDomainMVE ( MachineInstr MI)
static

Definition at line 101 of file ARMLowOverheadLoops.cpp.

References llvm::ARMII::DomainMask, llvm::ARMII::DomainMVE, and MI.

Referenced by shouldInspect().

◆ isHorizontalReduction()

static bool isHorizontalReduction ( const MachineInstr MI)
static

◆ isRegInClass()

static bool isRegInClass ( const MachineOperand MO,
const TargetRegisterClass Class 
)
static

◆ isVectorPredicate()

static bool isVectorPredicate ( MachineInstr MI)
static

Definition at line 93 of file ARMLowOverheadLoops.cpp.

References MI.

Referenced by shouldInspect().

◆ isVectorPredicated()

static bool isVectorPredicated ( MachineInstr MI)
static

Definition at line 88 of file ARMLowOverheadLoops.cpp.

References llvm::findFirstVPTPredOperandIdx(), and MI.

Referenced by producesFalseLanesZero().

◆ producesDoubleWidthResult()

static bool producesDoubleWidthResult ( const MachineInstr MI)
static

◆ producesFalseLanesZero()

static bool producesFalseLanesZero ( MachineInstr MI,
const TargetRegisterClass QPRs,
const ReachingDefAnalysis RDA,
InstSet &  FalseLanesZero 
)
static

◆ retainsPreviousHalfElement()

static bool retainsPreviousHalfElement ( const MachineInstr MI)
static

◆ shouldInspect()

static bool shouldInspect ( MachineInstr MI)
static

Definition at line 112 of file ARMLowOverheadLoops.cpp.

References hasVPRUse(), isDomainMVE(), isVectorPredicate(), and MI.

◆ ValidateMVEStore()

static bool ValidateMVEStore ( MachineInstr MI,
MachineLoop ML 
)
static

Variable Documentation

◆ DisableOmitDLS

cl::opt< bool > DisableOmitDLS("arm-disable-omit-dls", cl::Hidden, cl::desc("Disable omitting 'dls lr, lr' instructions"), cl::init(false)) ( "arm-disable-omit-dls"  ,
cl::Hidden  ,
cl::desc("Disable omitting 'dls lr, lr' instructions")  ,
cl::init(false)   
)
static

◆ DisableTailPredication

cl::opt< bool > DisableTailPredication("arm-loloops-disable-tailpred", cl::Hidden, cl::desc("Disable tail-predication in the ARM LowOverheadLoop pass"), cl::init(false)) ( "arm-loloops-disable-tailpred"  ,
cl::Hidden  ,
cl::desc("Disable tail-predication in the ARM LowOverheadLoop pass")  ,
cl::init(false)   
)
static

◆ false

return false

Definition at line 598 of file ARMLowOverheadLoops.cpp.

◆ Ignore

ReachingDefAnalysis InstSet InstSet& Ignore

◆ RDA

Definition at line 533 of file ARMLowOverheadLoops.cpp.

Referenced by if(), and producesFalseLanesZero().

◆ ToRemove

ReachingDefAnalysis InstSet& ToRemove
Examples
/home/buildbot/as-worker-4/publish-doxygen-docs/llvm-project/llvm/include/llvm/Transforms/Utils/Local.h.

Definition at line 534 of file ARMLowOverheadLoops.cpp.

Referenced by llvm::AA::PointerInfo::State::addAccess(), analyzeAndPruneOutputBlocks(), llvm::LoopVectorizationCostModel::calculateRegisterUsage(), checkAndReplaceCmp(), checkAndReplaceCondition(), checkAndReplaceMinMax(), llvm::LoopAccessInfoManager::clear(), llvm::EliminateDuplicatePHINodes(), EliminateDuplicatePHINodesNaiveImpl(), EliminateDuplicatePHINodesSetBasedImpl(), llvm::objcopy::elf::RelocSectionWithSymtabBase< DynamicSymbolTableSection >::finalize(), findBasePointer(), llvm::objcopy::coff::handleArgs(), if(), llvm::ReachingDefAnalysis::isSafeToRemove(), legalizeAndOptimizeInductions(), llvm::SPIRV::RequirementHandler::removeCapabilityIf(), llvm::objcopy::elf::Section::removeSectionReferences(), llvm::objcopy::elf::SymbolTableSection::removeSectionReferences(), llvm::objcopy::elf::GroupSection::removeSectionReferences(), llvm::objcopy::elf::DynamicRelocationSection::removeSectionReferences(), llvm::objcopy::elf::Object::removeSections(), llvm::objcopy::coff::Object::removeSections(), llvm::objcopy::wasm::Object::removeSections(), llvm::objcopy::macho::SymbolTable::removeSymbols(), llvm::objcopy::elf::Object::removeSymbols(), llvm::objcopy::elf::SymbolTableSection::removeSymbols(), llvm::objcopy::elf::RelocationSection::removeSymbols(), llvm::objcopy::elf::GroupSection::removeSymbols(), llvm::objcopy::coff::Object::removeSymbols(), removeUbsanTraps(), removeUnreachableFunctions(), replaceSubOverflowUses(), SPIRVStripConvergentIntrinsics::runOnFunction(), sinkMinMaxInBB(), llvm::JumpThreadingPass::threadGuard(), llvm::DbgValueHistoryMap::trimLocationRanges(), and tryToSimplifyOverflowMath().

◆ Uses

Definition at line 578 of file ARMLowOverheadLoops.cpp.

Referenced by if().