LLVM
12.0.0git
|
Finalize v8.1-m low-overhead loops by converting the associated pseudo instructions into machine operations. More...
#include "ARM.h"
#include "ARMBaseInstrInfo.h"
#include "ARMBaseRegisterInfo.h"
#include "ARMBasicBlockInfo.h"
#include "ARMSubtarget.h"
#include "MVETailPredUtils.h"
#include "Thumb2InstrInfo.h"
#include "llvm/ADT/SetOperations.h"
#include "llvm/ADT/SmallSet.h"
#include "llvm/CodeGen/LivePhysRegs.h"
#include "llvm/CodeGen/MachineFunctionPass.h"
#include "llvm/CodeGen/MachineLoopInfo.h"
#include "llvm/CodeGen/MachineLoopUtils.h"
#include "llvm/CodeGen/MachineRegisterInfo.h"
#include "llvm/CodeGen/Passes.h"
#include "llvm/CodeGen/ReachingDefAnalysis.h"
#include "llvm/MC/MCInstrDesc.h"
Go to the source code of this file.
Macros | |
#define | DEBUG_TYPE "arm-low-overhead-loops" |
#define | ARM_LOW_OVERHEAD_LOOPS_NAME "ARM Low Overhead Loops pass" |
Functions | |
static bool | isVectorPredicated (MachineInstr *MI) |
static bool | isVectorPredicate (MachineInstr *MI) |
static bool | hasVPRUse (MachineInstr &MI) |
static bool | isDomainMVE (MachineInstr *MI) |
static bool | shouldInspect (MachineInstr &MI) |
static bool | isDo (MachineInstr *MI) |
INITIALIZE_PASS (ARMLowOverheadLoops, DEBUG_TYPE, ARM_LOW_OVERHEAD_LOOPS_NAME, false, false) static bool TryRemove(MachineInstr *MI | |
if (!RDA.isSafeToRemove(MI, Uses, Ignore)) return false | |
if (WontCorruptITs(Uses, RDA)) | |
static bool | isRegInClass (const MachineOperand &MO, const TargetRegisterClass *Class) |
static bool | retainsPreviousHalfElement (const MachineInstr &MI) |
static bool | producesDoubleWidthResult (const MachineInstr &MI) |
static bool | isHorizontalReduction (const MachineInstr &MI) |
static bool | canGenerateNonZeros (const MachineInstr &MI) |
static bool | producesFalseLanesZero (MachineInstr &MI, const TargetRegisterClass *QPRs, const ReachingDefAnalysis &RDA, InstSet &FalseLanesZero) |
Variables | |
static cl::opt< bool > | DisableTailPredication ("arm-loloops-disable-tailpred", cl::Hidden, cl::desc("Disable tail-predication in the ARM LowOverheadLoop pass"), cl::init(false)) |
ReachingDefAnalysis & | RDA |
ReachingDefAnalysis InstSet & | ToRemove |
ReachingDefAnalysis InstSet InstSet & | Ignore |
SmallPtrSet< MachineInstr *, 2 > | Uses |
return | false |
Finalize v8.1-m low-overhead loops by converting the associated pseudo instructions into machine operations.
The expectation is that the loop contains three pseudo instructions:
In addition to this, we also look for the presence of the VCTP instruction, which determines whether we can generated the tail-predicated low-overhead loop form.
Assumptions and Dependencies: Low-overhead loops are constructed and executed using a setup instruction: DLS, WLS, DLSTP or WLSTP and an instruction that loops back: LE or LETP. WLS(TP) and LE(TP) are branching instructions with a (large) limited range but fixed polarity: WLS can only branch forwards and LE can only branch backwards. These restrictions mean that this pass is dependent upon block layout and block sizes, which is why it's the last pass to run. The same is true for ConstantIslands, but this pass does not increase the size of the basic blocks, nor does it change the CFG. Instructions are mainly removed during the transform and pseudo instructions are replaced by real ones. In some cases, when we have to revert to a 'normal' loop, we have to introduce multiple instructions for a single pseudo (see RevertWhile and RevertLoopEnd). To handle this situation, t2WhileLoopStart and t2LoopEnd are defined to be as large as this maximum sequence of replacement instructions.
A note on VPR.P0 (the lane mask): VPT, VCMP, VPNOT and VCTP won't overwrite VPR.P0 when they update it in a "VPT Active" context (which includes low-overhead loops and vpt blocks). They will simply "and" the result of their calculation with the current value of VPR.P0. You can think of it like this:
/// if VPT active: ; Between a DLSTP/LETP, or for predicated instrs /// VPR.P0 &= Value /// else /// VPR.P0 = Value ///
When we're inside the low-overhead loop (between DLSTP and LETP), we always fall in the "VPT active" case, so we can consider that all VPR writes by one of those instruction is actually a "and".
Definition in file ARMLowOverheadLoops.cpp.
Definition at line 75 of file ARMLowOverheadLoops.cpp.
#define DEBUG_TYPE "arm-low-overhead-loops" |
Definition at line 74 of file ARMLowOverheadLoops.cpp.
|
static |
Definition at line 836 of file ARMLowOverheadLoops.cpp.
References MI, and producesDoubleWidthResult().
Referenced by producesFalseLanesZero().
|
static |
Definition at line 91 of file ARMLowOverheadLoops.cpp.
References MI.
Referenced by shouldInspect().
if | ( | !RDA. | isSafeToRemoveMI, Uses, Ignore | ) |
Definition at line 588 of file ARMLowOverheadLoops.cpp.
References llvm::SmallPtrSetImpl< PtrType >::begin(), llvm::ReachingDefAnalysis::collectKilledOperands(), llvm::dbgs(), llvm::RegState::Dead, llvm::SmallPtrSetImpl< PtrType >::end(), LLVM_DEBUG, MI, RDA, ToRemove, and Uses.
INITIALIZE_PASS | ( | ARMLowOverheadLoops | , |
DEBUG_TYPE | , | ||
ARM_LOW_OVERHEAD_LOOPS_NAME | , | ||
false | , | ||
false | |||
) |
|
static |
Definition at line 104 of file ARMLowOverheadLoops.cpp.
References MI.
|
static |
Definition at line 95 of file ARMLowOverheadLoops.cpp.
References llvm::ARMII::DomainMask, llvm::ARMII::DomainMVE, and MI.
Referenced by shouldInspect().
|
static |
Definition at line 826 of file ARMLowOverheadLoops.cpp.
References llvm::ARMII::HorizontalReduction, MI, and llvm::MCInstrDesc::TSFlags.
Referenced by producesFalseLanesZero().
|
static |
Definition at line 803 of file ARMLowOverheadLoops.cpp.
References llvm::MachineOperand::getReg(), and llvm::MachineOperand::isReg().
Referenced by producesFalseLanesZero().
|
static |
Definition at line 87 of file ARMLowOverheadLoops.cpp.
References MI.
Referenced by shouldInspect().
|
static |
Definition at line 82 of file ARMLowOverheadLoops.cpp.
References llvm::findFirstVPTPredOperandIdx(), and MI.
Referenced by producesFalseLanesZero().
|
static |
Definition at line 820 of file ARMLowOverheadLoops.cpp.
References llvm::ARMII::DoubleWidthResult, MI, and llvm::MCInstrDesc::TSFlags.
Referenced by canGenerateNonZeros().
|
static |
Definition at line 865 of file ARMLowOverheadLoops.cpp.
References canGenerateNonZeros(), llvm::dbgs(), llvm::tgtok::Def, llvm::AArch64::FalseLanesZero, llvm::ReachingDefAnalysis::getGlobalReachingDefs(), llvm::MachineOperand::getReg(), isHorizontalReduction(), llvm::ARM_MC::isPredicated(), llvm::MachineOperand::isReg(), isRegInClass(), llvm::MachineOperand::isUse(), isVectorPredicated(), LLVM_DEBUG, MI, and RDA.
|
static |
Definition at line 811 of file ARMLowOverheadLoops.cpp.
References MI, llvm::ARMII::RetainsPreviousHalfElement, and llvm::MCInstrDesc::TSFlags.
|
static |
Definition at line 100 of file ARMLowOverheadLoops.cpp.
References hasVPRUse(), isDomainMVE(), isVectorPredicate(), and MI.
|
static |
return false |
Definition at line 604 of file ARMLowOverheadLoops.cpp.
ReachingDefAnalysis InstSet InstSet& Ignore |
Definition at line 541 of file ARMLowOverheadLoops.cpp.
Referenced by llvm::ReachingDefAnalysis::isSafeToDefRegAt(), and llvm::ReachingDefAnalysis::isSafeToRemove().
ReachingDefAnalysis& RDA |
Definition at line 540 of file ARMLowOverheadLoops.cpp.
Referenced by if(), and producesFalseLanesZero().
ReachingDefAnalysis InstSet& ToRemove |
Definition at line 540 of file ARMLowOverheadLoops.cpp.
Referenced by llvm::LoopVectorizationCostModel::calculateRegisterUsage(), if(), llvm::ReachingDefAnalysis::isSafeToRemove(), llvm::JumpThreadingPass::processBlock(), llvm::JumpThreadingPass::threadGuard(), and llvm::DbgValueHistoryMap::trimLocationRanges().
SmallPtrSet<MachineInstr *, 2> Uses |
Definition at line 584 of file ARMLowOverheadLoops.cpp.
Referenced by llvm::CombinerHelper::applyCombineExtendingLoads(), ClearKillFlags(), llvm::ReachingDefAnalysis::collectKilledOperands(), computeLiveOuts(), llvm::rdf::Liveness::computePhiInfo(), llvm::PeelingModuloScheduleExpander::CreateLCSSAExitingBlock(), llvm::OpenMPIRBuilder::createParallel(), llvm::X86RegisterInfo::findDeadCallerSavedReg(), findHoistingInsertPosAndDeps(), findUses(), llvm::rdf::Liveness::getAllReachedUses(), llvm::ReachingDefAnalysis::getGlobalUses(), llvm::ReachingDefAnalysis::getLiveInUses(), llvm::ReachingDefAnalysis::getReachingLocalUses(), llvm::mca::InstructionBase::getUses(), if(), llvm::ReachingDefAnalysis::isSafeToDefRegAt(), parseOperands(), llvm::PeelSingleBlockLoop(), propagateLocalCopies(), relocationViaAlloca(), llvm::SelectionDAG::ReplaceAllUsesOfValuesWith(), llvm::ReplaceableMetadataImpl::replaceAllUsesWith(), llvm::ReplaceableMetadataImpl::resolveAllUses(), and TrackDefUses().