LLVM 20.0.0git
Public Member Functions | Static Public Member Functions | Friends | List of all members
llvm::VPlan Class Reference

VPlan models a candidate for vectorization, encoding various decisions take to produce efficient output IR, including which branches, basic-blocks and output IR instructions to generate, and their cost. More...

#include "Transforms/Vectorize/VPlan.h"

Public Member Functions

 VPlan (VPBasicBlock *Preheader, VPValue *TC, VPBasicBlock *Entry)
 Construct a VPlan with original preheader Preheader, trip count TC and Entry to the plan.
 
 VPlan (VPBasicBlock *Preheader, VPBasicBlock *Entry)
 Construct a VPlan with original preheader Preheader and Entry to the plan.
 
 ~VPlan ()
 
void prepareToExecute (Value *TripCount, Value *VectorTripCount, Value *CanonicalIVStartValue, VPTransformState &State)
 Prepare the plan for execution, setting up the required live-in values.
 
void execute (VPTransformState *State)
 Generate the IR code for this VPlan.
 
InstructionCost cost (ElementCount VF, VPCostContext &Ctx)
 Return the cost of this plan.
 
VPBasicBlockgetEntry ()
 
const VPBasicBlockgetEntry () const
 
VPValuegetTripCount () const
 The trip count of the original loop.
 
void resetTripCount (VPValue *NewTripCount)
 Resets the trip count for the VPlan.
 
VPValuegetOrCreateBackedgeTakenCount ()
 The backedge taken count of the original loop.
 
VPValuegetVectorTripCount ()
 The vector trip count.
 
VPValuegetVFxUF ()
 Returns VF * UF of the vector loop region.
 
void addVF (ElementCount VF)
 
void setVF (ElementCount VF)
 
bool hasVF (ElementCount VF)
 
bool hasScalableVF ()
 
iterator_range< SmallSetVector< ElementCount, 2 >::iterator > vectorFactors () const
 Returns an iterator range over all VFs of the plan.
 
bool hasScalarVFOnly () const
 
bool hasUF (unsigned UF) const
 
void setUF (unsigned UF)
 
std::string getName () const
 Return a string with the name of the plan and the applicable VFs and UFs.
 
void setName (const Twine &newName)
 
VPValuegetOrAddLiveIn (Value *V)
 Gets the live-in VPValue for V or adds a new live-in (if none exists yet) for V.
 
VPValuegetLiveIn (Value *V) const
 Return the live-in VPValue for V, if there is one or nullptr otherwise.
 
void printLiveIns (raw_ostream &O) const
 Print the live-ins of this VPlan to O.
 
void print (raw_ostream &O) const
 Print this VPlan to O.
 
void printDOT (raw_ostream &O) const
 Print this VPlan in DOT format to O.
 
LLVM_DUMP_METHOD void dump () const
 Dump the plan to stderr (for debugging).
 
VPRegionBlockgetVectorLoopRegion ()
 Returns the VPRegionBlock of the vector loop.
 
const VPRegionBlockgetVectorLoopRegion () const
 
VPCanonicalIVPHIRecipegetCanonicalIV ()
 Returns the canonical induction recipe of the vector loop.
 
void addLiveOut (PHINode *PN, VPValue *V)
 
const MapVector< PHINode *, VPLiveOut * > & getLiveOuts () const
 
VPValuegetSCEVExpansion (const SCEV *S) const
 
void addSCEVExpansion (const SCEV *S, VPValue *V)
 
VPBasicBlockgetPreheader ()
 
const VPBasicBlockgetPreheader () const
 
VPlanduplicate ()
 Clone the current VPlan, update all VPValues of the new VPlan and cloned recipes to refer to the clones, and return it.
 

Static Public Member Functions

static VPlanPtr createInitialVPlan (const SCEV *TripCount, ScalarEvolution &PSE, bool RequiresScalarEpilogueCheck, bool TailFolded, Loop *TheLoop)
 Create initial VPlan, having an "entry" VPBasicBlock (wrapping original scalar pre-header ) which contains SCEV expansions that need to happen before the CFG is modified; a VPBasicBlock for the vector pre-header, followed by a region for the vector loop, followed by the middle VPBasicBlock.
 

Friends

class VPlanPrinter
 
class VPSlotTracker
 

Detailed Description

VPlan models a candidate for vectorization, encoding various decisions take to produce efficient output IR, including which branches, basic-blocks and output IR instructions to generate, and their cost.

VPlan holds a Hierarchical-CFG of VPBasicBlocks and VPRegionBlocks rooted at an Entry VPBasicBlock.

Definition at line 3274 of file VPlan.h.

Constructor & Destructor Documentation

◆ VPlan() [1/2]

llvm::VPlan::VPlan ( VPBasicBlock Preheader,
VPValue TC,
VPBasicBlock Entry 
)
inline

Construct a VPlan with original preheader Preheader, trip count TC and Entry to the plan.

At the moment, Preheader and Entry need to be disconnected, as the bypass blocks between them are not yet modeled in VPlan.

Definition at line 3334 of file VPlan.h.

◆ VPlan() [2/2]

llvm::VPlan::VPlan ( VPBasicBlock Preheader,
VPBasicBlock Entry 
)
inline

Construct a VPlan with original preheader Preheader and Entry to the plan.

At the moment, Preheader and Entry need to be disconnected, as the bypass blocks between them are not yet modeled in VPlan.

Definition at line 3343 of file VPlan.h.

References assert(), llvm::VPBlockBase::getNumPredecessors(), llvm::VPBlockBase::getNumSuccessors(), and llvm::VPBlockBase::setPlan().

◆ ~VPlan()

VPlan::~VPlan ( )

Member Function Documentation

◆ addLiveOut()

void VPlan::addLiveOut ( PHINode PN,
VPValue V 
)

Definition at line 1187 of file VPlan.cpp.

References assert().

Referenced by addLiveOutsForFirstOrderRecurrences(), and addUsersInExitBlock().

◆ addSCEVExpansion()

void llvm::VPlan::addSCEVExpansion ( const SCEV S,
VPValue V 
)
inline

◆ addVF()

void llvm::VPlan::addVF ( ElementCount  VF)
inline

Definition at line 3406 of file VPlan.h.

References llvm::SetVector< T, Vector, Set, N >::insert().

◆ cost()

InstructionCost VPlan::cost ( ElementCount  VF,
VPCostContext Ctx 
)

Return the cost of this plan.

Definition at line 1091 of file VPlan.cpp.

References llvm::VPTransformState::VF.

◆ createInitialVPlan()

VPlanPtr VPlan::createInitialVPlan ( const SCEV TripCount,
ScalarEvolution PSE,
bool  RequiresScalarEpilogueCheck,
bool  TailFolded,
Loop TheLoop 
)
static

Create initial VPlan, having an "entry" VPBasicBlock (wrapping original scalar pre-header ) which contains SCEV expansions that need to happen before the CFG is modified; a VPBasicBlock for the vector pre-header, followed by a region for the vector loop, followed by the middle VPBasicBlock.

If a check is needed to guard executing the scalar epilogue loop, it will be added to the middle block, together with VPBasicBlocks for the scalar preheader and exit blocks.

Definition at line 863 of file VPlan.cpp.

References llvm::VPInstruction::BranchOnCond, llvm::VPTransformState::Builder, llvm::VPBlockUtils::connectBlocks(), llvm::Type::getContext(), llvm::Type::getInt1Ty(), llvm::LoopBase< BlockT, LoopT >::getLoopLatch(), llvm::LoopBase< BlockT, LoopT >::getLoopPreheader(), getOrAddLiveIn(), llvm::vputils::getOrCreateVPValueForSCEVExpr(), llvm::BasicBlock::getTerminator(), getTripCount(), llvm::ConstantInt::getTrue(), llvm::SCEV::getType(), llvm::LoopBase< BlockT, LoopT >::getUniqueExitBlock(), getVectorTripCount(), llvm::CmpInst::ICMP_EQ, llvm::VPBlockUtils::insertBlockAfter(), and llvm::VPTransformState::Plan.

◆ dump()

LLVM_DUMP_METHOD void VPlan::dump ( ) const

Dump the plan to stderr (for debugging).

Definition at line 1184 of file VPlan.cpp.

References llvm::dbgs(), and llvm::print().

Referenced by llvm::LoopVectorizationPlanner::executePlan().

◆ duplicate()

VPlan * VPlan::duplicate ( )

Clone the current VPlan, update all VPValues of the new VPlan and cloned recipes to refer to the clones, and return it.

Definition at line 1230 of file VPlan.cpp.

References _, assert(), llvm::VPBasicBlock::clone(), cloneFrom(), llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::contains(), Name, and remapOperands().

Referenced by llvm::LoopVectorizePass::processLoop().

◆ execute()

void VPlan::execute ( VPTransformState State)

Generate the IR code for this VPlan.

Generate the code inside the preheader and body of the vectorized loop.

Assumes a single pre-header basic-block was created for this. Introduce additional basic-blocks as needed, and fill them all.

Definition at line 981 of file VPlan.cpp.

References llvm::GenericDomTreeUpdater< DerivedT, DomTreeT, PostDomTreeT >::applyUpdates(), assert(), llvm::Block, llvm::VPTransformState::Builder, llvm::VPTransformState::CFG, llvm::DominatorTreeBase< BasicBlock, false >::Delete, llvm::VPTransformState::CFGState::DTU, llvm::Instruction::eraseFromParent(), llvm::VPTransformState::CFGState::ExitBB, llvm::GenericDomTreeUpdater< DerivedT, DomTreeT, PostDomTreeT >::flush(), GEP, llvm::VPTransformState::get(), llvm::BasicBlock::getContext(), llvm::GenericDomTreeUpdater< DerivedT, DomTreeT, PostDomTreeT >::getDomTree(), llvm::VPBlockBase::getExitingBasicBlock(), llvm::ilist_node_with_parent< NodeTy, ParentTy, Options >::getPrevNode(), llvm::BasicBlock::getSingleSuccessor(), llvm::VPBlockBase::getSuccessors(), llvm::BasicBlock::getTerminator(), llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::isScalable(), llvm::Instruction::moveBefore(), llvm::VPTransformState::CFGState::PrevBB, llvm::VPTransformState::CFGState::PrevVPBB, replaceVPBBWithIRVPBB(), llvm::IRBuilderBase::SetInsertPoint(), llvm::VPTransformState::UF, llvm::DominatorTreeBase< NodeT, IsPostDom >::verify(), llvm::VPTransformState::VF, llvm::vp_depth_first_shallow(), and llvm::VPTransformState::CFGState::VPBB2IRBB.

Referenced by llvm::LoopVectorizationPlanner::executePlan().

◆ getCanonicalIV()

VPCanonicalIVPHIRecipe * llvm::VPlan::getCanonicalIV ( )
inline

◆ getEntry() [1/2]

VPBasicBlock * llvm::VPlan::getEntry ( )
inline

◆ getEntry() [2/2]

const VPBasicBlock * llvm::VPlan::getEntry ( ) const
inline

Definition at line 3377 of file VPlan.h.

◆ getLiveIn()

VPValue * llvm::VPlan::getLiveIn ( Value V) const
inline

Return the live-in VPValue for V, if there is one or nullptr otherwise.

Definition at line 3459 of file VPlan.h.

References llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::lookup().

◆ getLiveOuts()

const MapVector< PHINode *, VPLiveOut * > & llvm::VPlan::getLiveOuts ( ) const
inline

Definition at line 3495 of file VPlan.h.

Referenced by llvm::InnerLoopVectorizer::fixVectorizedLoop().

◆ getName()

std::string VPlan::getName ( ) const

Return a string with the name of the plan and the applicable VFs and UFs.

Definition at line 1154 of file VPlan.cpp.

References llvm::drop_begin(), Name, llvm::VPTransformState::UF, and llvm::VPTransformState::VF.

Referenced by llvm::VPlanPrinter::dump().

◆ getOrAddLiveIn()

VPValue * llvm::VPlan::getOrAddLiveIn ( Value V)
inline

◆ getOrCreateBackedgeTakenCount()

VPValue * llvm::VPlan::getOrCreateBackedgeTakenCount ( )
inline

The backedge taken count of the original loop.

Definition at line 3394 of file VPlan.h.

Referenced by llvm::VPRecipeBuilder::createHeaderMask(), and llvm::vputils::isHeaderMask().

◆ getPreheader() [1/2]

VPBasicBlock * llvm::VPlan::getPreheader ( )
inline

◆ getPreheader() [2/2]

const VPBasicBlock * llvm::VPlan::getPreheader ( ) const
inline

Definition at line 3510 of file VPlan.h.

◆ getSCEVExpansion()

VPValue * llvm::VPlan::getSCEVExpansion ( const SCEV S) const
inline

◆ getTripCount()

VPValue * llvm::VPlan::getTripCount ( ) const
inline

◆ getVectorLoopRegion() [1/2]

VPRegionBlock * llvm::VPlan::getVectorLoopRegion ( )
inline

◆ getVectorLoopRegion() [2/2]

const VPRegionBlock * llvm::VPlan::getVectorLoopRegion ( ) const
inline

Definition at line 3479 of file VPlan.h.

References getEntry().

◆ getVectorTripCount()

VPValue & llvm::VPlan::getVectorTripCount ( )
inline

The vector trip count.

Definition at line 3401 of file VPlan.h.

Referenced by addCanonicalIVRecipes(), and createInitialVPlan().

◆ getVFxUF()

VPValue & llvm::VPlan::getVFxUF ( )
inline

Returns VF * UF of the vector loop region.

Definition at line 3404 of file VPlan.h.

Referenced by addCanonicalIVRecipes().

◆ hasScalableVF()

bool llvm::VPlan::hasScalableVF ( )
inline

◆ hasScalarVFOnly()

bool llvm::VPlan::hasScalarVFOnly ( ) const
inline

Definition at line 3425 of file VPlan.h.

References llvm::SetVector< T, Vector, Set, N >::size().

Referenced by sinkScalarOperands().

◆ hasUF()

bool llvm::VPlan::hasUF ( unsigned  UF) const
inline

◆ hasVF()

bool llvm::VPlan::hasVF ( ElementCount  VF)
inline

◆ prepareToExecute()

void VPlan::prepareToExecute ( Value TripCount,
Value VectorTripCount,
Value CanonicalIVStartValue,
VPTransformState State 
)

◆ print()

LLVM_DUMP_METHOD void VPlan::print ( raw_ostream O) const

Print this VPlan to O.

Definition at line 1128 of file VPlan.cpp.

References llvm::Block, getName(), and llvm::vp_depth_first_shallow().

Referenced by llvm::operator<<(), and llvm::LoopVectorizationPlanner::printPlans().

◆ printDOT()

LLVM_DUMP_METHOD void VPlan::printDOT ( raw_ostream O) const

Print this VPlan in DOT format to O.

Definition at line 1178 of file VPlan.cpp.

References llvm::Metadata::dump(), and Printer.

Referenced by llvm::LoopVectorizationPlanner::printPlans().

◆ printLiveIns()

void VPlan::printLiveIns ( raw_ostream O) const

Print the live-ins of this VPlan to O.

Definition at line 1098 of file VPlan.cpp.

Referenced by llvm::VPlanPrinter::dump().

◆ resetTripCount()

void llvm::VPlan::resetTripCount ( VPValue NewTripCount)
inline

Resets the trip count for the VPlan.

The caller must make sure all uses of the original trip count have been replaced.

Definition at line 3387 of file VPlan.h.

References assert(), and llvm::VPValue::getNumUsers().

Referenced by llvm::LoopVectorizePass::processLoop().

◆ setName()

void llvm::VPlan::setName ( const Twine newName)
inline

◆ setUF()

void llvm::VPlan::setUF ( unsigned  UF)
inline

◆ setVF()

void llvm::VPlan::setVF ( ElementCount  VF)
inline

◆ vectorFactors()

iterator_range< SmallSetVector< ElementCount, 2 >::iterator > llvm::VPlan::vectorFactors ( ) const
inline

Returns an iterator range over all VFs of the plan.

Definition at line 3421 of file VPlan.h.

References llvm::SetVector< T, Vector, Set, N >::begin(), and llvm::SetVector< T, Vector, Set, N >::end().

Referenced by llvm::LoopVectorizationPlanner::computeBestVF().

Friends And Related Function Documentation

◆ VPlanPrinter

friend class VPlanPrinter
friend

Definition at line 3275 of file VPlan.h.

◆ VPSlotTracker

friend class VPSlotTracker
friend

Definition at line 3276 of file VPlan.h.


The documentation for this class was generated from the following files: