LLVM
15.0.0git
|
VPlan models a candidate for vectorization, encoding various decisions take to produce efficient output IR, including which branches, basic-blocks and output IR instructions to generate, and their cost. More...
#include "Transforms/Vectorize/VPlan.h"
Public Member Functions | |
VPlan (VPBlockBase *Entry=nullptr) | |
~VPlan () | |
void | prepareToExecute (Value *TripCount, Value *VectorTripCount, Value *CanonicalIVStartValue, VPTransformState &State) |
Prepare the plan for execution, setting up the required live-in values. More... | |
void | execute (struct VPTransformState *State) |
Generate the IR code for this VPlan. More... | |
VPBlockBase * | getEntry () |
const VPBlockBase * | getEntry () const |
VPBlockBase * | setEntry (VPBlockBase *Block) |
VPValue * | getOrCreateTripCount () |
The trip count of the original loop. More... | |
VPValue * | getOrCreateBackedgeTakenCount () |
The backedge taken count of the original loop. More... | |
VPValue & | getVectorTripCount () |
The vector trip count. More... | |
void | disableValue2VPValue () |
Mark the plan to indicate that using Value2VPValue is not safe any longer, because it may be stale. More... | |
void | addVF (ElementCount VF) |
bool | hasVF (ElementCount VF) |
const std::string & | getName () const |
void | setName (const Twine &newName) |
VPValue * | getOrAddExternalDef (Value *V) |
Get the existing or add a new external definition for V . More... | |
void | addVPValue (Value *V) |
void | addVPValue (Value *V, VPValue *VPV) |
VPValue * | getVPValue (Value *V, bool OverrideAllowed=false) |
Returns the VPValue for V . More... | |
VPValue * | getOrAddVPValue (Value *V, bool OverrideAllowed=false) |
Gets the VPValue or adds a new one (if none exists yet) for V . More... | |
void | removeVPValueFor (Value *V) |
void | print (raw_ostream &O) const |
Print this VPlan to O . More... | |
void | printDOT (raw_ostream &O) const |
Print this VPlan in DOT format to O . More... | |
LLVM_DUMP_METHOD void | dump () const |
Dump the plan to stderr (for debugging). More... | |
iterator_range< mapped_iterator< Use *, std::function< VPValue *(Value *)> > > | mapToVPValues (User::op_range Operands) |
Returns a range mapping the values the range Operands to their corresponding VPValues. More... | |
bool | isUniformAfterVectorization (VPValue *VPV) const |
Returns true if VPV is uniform after vectorization. More... | |
VPRegionBlock * | getVectorLoopRegion () |
Returns the VPRegionBlock of the vector loop. More... | |
const VPRegionBlock * | getVectorLoopRegion () const |
VPCanonicalIVPHIRecipe * | getCanonicalIV () |
Returns the canonical induction recipe of the vector loop. More... | |
void | addLiveOut (PHINode *PN, VPValue *V) |
void | clearLiveOuts () |
void | removeLiveOut (PHINode *PN) |
const MapVector< PHINode *, VPLiveOut * > & | getLiveOuts () const |
Friends | |
class | VPlanPrinter |
class | VPSlotTracker |
VPlan models a candidate for vectorization, encoding various decisions take to produce efficient output IR, including which branches, basic-blocks and output IR instructions to generate, and their cost.
VPlan holds a Hierarchical-CFG of VPBasicBlocks and VPRegionBlocks rooted at an Entry VPBlock.
|
inline |
|
inline |
Definition at line 2466 of file VPlan.h.
References clearLiveOuts().
|
inline |
Definition at line 2523 of file VPlan.h.
References llvm::SetVector< T, SmallVector< T, N >, SmallDenseSet< T, N > >::insert().
|
inline |
Definition at line 2539 of file VPlan.h.
References assert(), and llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::count().
Referenced by getOrAddVPValue().
Definition at line 2549 of file VPlan.h.
References assert(), and llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::count().
|
inline |
Definition at line 2631 of file VPlan.h.
Referenced by llvm::InnerLoopVectorizer::fixVectorizedLoop(), and ~VPlan().
|
inline |
LLVM_DUMP_METHOD void VPlan::dump | ( | ) | const |
Dump the plan to stderr (for debugging).
Definition at line 720 of file VPlan.cpp.
References llvm::dbgs(), and print().
void VPlan::execute | ( | struct VPTransformState * | State | ) |
Generate the IR code for this VPlan.
Generate the code inside the preheader and body of the vectorized loop.
Assumes a single pre-header basic-block was created for this. Introduce additional basic-blocks as needed, and fill them all.
Definition at line 598 of file VPlan.cpp.
Referenced by llvm::LoopVectorizationPlanner::executePlan().
|
inline |
Returns the canonical induction recipe of the vector loop.
Definition at line 2620 of file VPlan.h.
References llvm::VPBasicBlock::begin(), llvm::VPBasicBlock::empty(), llvm::VPBlockBase::getEntryBasicBlock(), llvm::VPBlockBase::getSingleSuccessor(), and getVectorLoopRegion().
Referenced by llvm::VPlanTransforms::optimizeInductions(), and llvm::VPlanTransforms::removeRedundantCanonicalIVs().
|
inline |
Definition at line 2493 of file VPlan.h.
Referenced by llvm::InnerLoopVectorizer::collectPoisonGeneratingRecipes(), llvm::VPlanPrinter::dump(), llvm::InnerLoopVectorizer::fixNonInductionPHIs(), llvm::vputils::getOrCreateVPValueForSCEVExpr(), getVectorLoopRegion(), llvm::VPlanTransforms::mergeReplicateRegions(), llvm::VPlanTransforms::removeDeadRecipes(), llvm::VPlanTransforms::removeRedundantExpandSCEVRecipes(), llvm::VPBlockBase::setPlan(), llvm::VPlanTransforms::sinkScalarOperands(), and llvm::VPlanVerifier::verifyPlanIsValid().
|
inline |
|
inline |
Definition at line 2527 of file VPlan.h.
Referenced by llvm::VPlanPrinter::dump().
Get the existing or add a new external definition for V
.
Definition at line 2532 of file VPlan.h.
References I, and llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::insert().
Referenced by llvm::vputils::getOrCreateVPValueForSCEVExpr(), and llvm::LoopVectorizePass::processLoop().
Gets the VPValue or adds a new one (if none exists yet) for V
.
OverrideAllowed
can be used to disable checking whether it is safe to query VPValues using IR Values.
Definition at line 2569 of file VPlan.h.
References addVPValue(), assert(), llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::count(), and getVPValue().
Referenced by addCanonicalIVRecipes(), addUsersInExitBlock(), and mapToVPValues().
|
inline |
|
inline |
|
inline |
Returns the VPRegionBlock of the vector loop.
Definition at line 2612 of file VPlan.h.
References getEntry().
Referenced by addCanonicalIVRecipes(), llvm::VPlanHCFGBuilder::buildHierarchicalCFG(), llvm::LoopVectorizationPlanner::executePlan(), llvm::InnerLoopVectorizer::fixCrossIterationPHIs(), llvm::InnerLoopVectorizer::fixVectorizedLoop(), getCanonicalIV(), llvm::VPlanTransforms::optimizeInductions(), llvm::LoopVectorizePass::processLoop(), llvm::VPlanTransforms::removeRedundantCanonicalIVs(), llvm::VPlanTransforms::removeRedundantInductionCasts(), and llvm::VPInterleavedAccessInfo::VPInterleavedAccessInfo().
|
inline |
Definition at line 2615 of file VPlan.h.
References getEntry().
|
inline |
The vector trip count.
Definition at line 2517 of file VPlan.h.
Referenced by addCanonicalIVRecipes().
Returns the VPValue for V
.
OverrideAllowed
can be used to disable checking whether it is safe to query VPValues using IR Values.
Definition at line 2558 of file VPlan.h.
References assert(), and llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::count().
Referenced by getOrAddVPValue(), and llvm::InnerLoopVectorizer::truncateToMinimalBitwidths().
|
inline |
Definition at line 2525 of file VPlan.h.
References llvm::SetVector< T, SmallVector< T, N >, SmallDenseSet< T, N > >::count().
Referenced by llvm::VPlanTransforms::optimizeInductions().
|
inline |
Returns true if VPV
is uniform after vectorization.
Definition at line 2606 of file VPlan.h.
References llvm::VPValue::getDef().
Referenced by llvm::VPLiveOut::fixPhi().
|
inline |
Returns a range mapping the values the range Operands
to their corresponding VPValues.
Definition at line 2598 of file VPlan.h.
References function, getOrAddVPValue(), llvm::map_range(), and Operands.
void VPlan::prepareToExecute | ( | Value * | TripCount, |
Value * | VectorTripCount, | ||
Value * | CanonicalIVStartValue, | ||
VPTransformState & | State | ||
) |
Prepare the plan for execution, setting up the required live-in values.
Definition at line 526 of file VPlan.cpp.
References llvm::all_of(), llvm::VPBasicBlock::appendRecipe(), assert(), llvm::VPBasicBlock::back(), llvm::VPInstruction::BranchOnCond, llvm::VPInstruction::BranchOnCount, llvm::VPTransformState::Builder, llvm::VPTransformState::CFG, llvm::IRBuilderBase::CreateSub(), llvm::IRBuilderBase::CreateVectorSplat(), llvm::ConstantInt::get(), llvm::VPBlockBase::getExitingBasicBlock(), llvm::LinearPolySize< LeafTy >::getKnownMinValue(), llvm::BasicBlock::getTerminator(), llvm::IRBuilderBase::getTrue(), llvm::Value::getType(), llvm::ElementCount::isScalar(), IV, llvm::VPTransformState::CFGState::PrevBB, llvm::VPTransformState::set(), llvm::M68kBeads::Term, llvm::VPTransformState::UF, and llvm::VPTransformState::VF.
Referenced by llvm::LoopVectorizationPlanner::executePlan().
LLVM_DUMP_METHOD void VPlan::print | ( | raw_ostream & | O | ) | const |
Print this VPlan to O
.
Definition at line 678 of file VPlan.cpp.
References llvm::depth_first(), and llvm::RISCVFenceField::O.
Referenced by llvm::operator<<().
LLVM_DUMP_METHOD void VPlan::printDOT | ( | raw_ostream & | O | ) | const |
|
inline |
Definition at line 2637 of file VPlan.h.
References llvm::MapVector< KeyT, ValueT, MapType, VectorType >::erase().
Referenced by llvm::InnerLoopVectorizer::fixFirstOrderRecurrence(), llvm::InnerLoopVectorizer::fixReduction(), and llvm::InnerLoopVectorizer::fixupIVUsers().
|
inline |
Definition at line 2578 of file VPlan.h.
References assert(), and llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::erase().
|
inline |
Definition at line 2496 of file VPlan.h.
Referenced by llvm::VPlanHCFGBuilder::buildHierarchicalCFG().
Definition at line 2529 of file VPlan.h.
References llvm::Twine::str().
Referenced by llvm::VPlanHCFGBuilder::buildHierarchicalCFG().
|
friend |
|
friend |