VPlan models a candidate for vectorization, encoding various decisions take to produce efficient output IR, including which branches, basic-blocks and output IR instructions to generate, and their cost. More...

#include "Transforms/Vectorize/VPlan.h"

Public Member Functions
	VPlan (Loop *L)
	Construct a VPlan for `L`.

	VPlan (BasicBlock ScalarHeaderBB, VPValue TC)
	Construct a VPlan with a new VPBasicBlock as entry, a VPIRBasicBlock wrapping `ScalarHeaderBB` and a trip count of `TC`.

	~VPlan ()

void	setEntry (VPBasicBlock *VPBB)

void	prepareToExecute (Value TripCount, Value VectorTripCount, VPTransformState &State)
	Prepare the plan for execution, setting up the required live-in values.

void	execute (VPTransformState *State)
	Generate the IR code for this VPlan.

InstructionCost	cost (ElementCount VF, VPCostContext &Ctx)
	Return the cost of this plan.

VPBasicBlock *	getEntry ()

const VPBasicBlock *	getEntry () const

VPBasicBlock *	getVectorPreheader ()
	Returns the preheader of the vector loop region, if one exists, or null otherwise.

VPRegionBlock *	getVectorLoopRegion ()
	Returns the VPRegionBlock of the vector loop.

const VPRegionBlock *	getVectorLoopRegion () const

const VPBasicBlock *	getMiddleBlock () const
	Returns the 'middle' block of the plan, that is the block that selects whether to execute the scalar tail loop or the exit block from the loop latch.

VPBasicBlock *	getMiddleBlock ()

VPBasicBlock *	getScalarPreheader () const
	Return the VPBasicBlock for the preheader of the scalar loop.

VPIRBasicBlock *	getScalarHeader () const
	Return the VPIRBasicBlock wrapping the header of the scalar loop.

auto	getExitBlocks ()
	Return an iterator range over the VPIRBasicBlock wrapping the exit blocks of the VPlan, that is leaf nodes except the scalar header.

VPValue *	getTripCount () const
	The trip count of the original loop.

void	resetTripCount (VPValue *NewTripCount)
	Resets the trip count for the VPlan.

VPValue *	getOrCreateBackedgeTakenCount ()
	The backedge taken count of the original loop.

VPValue &	getVectorTripCount ()
	The vector trip count.

VPValue &	getVF ()
	Returns the VF of the vector loop region.

VPValue &	getVFxUF ()
	Returns VF * UF of the vector loop region.

void	addVF (ElementCount VF)

void	setVF (ElementCount VF)

bool	hasVF (ElementCount VF)

bool	hasScalableVF ()

iterator_range< SmallSetVector< ElementCount, 2 >::iterator >	vectorFactors () const
	Returns an iterator range over all VFs of the plan.

bool	hasScalarVFOnly () const

bool	hasUF (unsigned UF) const

unsigned	getUF () const

void	setUF (unsigned UF)

std::string	getName () const
	Return a string with the name of the plan and the applicable VFs and UFs.

void	setName (const Twine &newName)

VPValue *	getOrAddLiveIn (Value *V)
	Gets the live-in VPValue for `V` or adds a new live-in (if none exists yet) for `V`.

VPValue *	getLiveIn (Value *V) const
	Return the live-in VPValue for `V`, if there is one or nullptr otherwise.

void	printLiveIns (raw_ostream &O) const
	Print the live-ins of this VPlan to `O`.

void	print (raw_ostream &O) const
	Print this VPlan to `O`.

void	printDOT (raw_ostream &O) const
	Print this VPlan in DOT format to `O`.

LLVM_DUMP_METHOD void	dump () const
	Dump the plan to stderr (for debugging).

VPCanonicalIVPHIRecipe *	getCanonicalIV ()
	Returns the canonical induction recipe of the vector loop.

VPValue *	getSCEVExpansion (const SCEV *S) const

void	addSCEVExpansion (const SCEV S, VPValue V)

VPlan *	duplicate ()
	Clone the current VPlan, update all VPValues of the new VPlan and cloned recipes to refer to the clones, and return it.

VPBasicBlock *	createVPBasicBlock (const Twine &Name, VPRecipeBase *Recipe=nullptr)
	Create a new VPBasicBlock with `Name` and containing `Recipe` if present.

VPRegionBlock *	createVPRegionBlock (VPBlockBase Entry, VPBlockBase Exiting, const std::string &Name="", bool IsReplicator=false)
	Create a new VPRegionBlock with `Entry`, `Exiting` and `Name`.

VPRegionBlock *	createVPRegionBlock (const std::string &Name="", bool IsReplicator=false)
	Create a new VPRegionBlock with `Name` and entry and exiting blocks set to nullptr.

VPIRBasicBlock *	createEmptyVPIRBasicBlock (BasicBlock *IRBB)
	Create a VPIRBasicBlock wrapping `IRBB`, but do not create VPIRInstructions wrapping the instructions in t`IRBB`.

VPIRBasicBlock *	createVPIRBasicBlock (BasicBlock *IRBB)
	Create a VPIRBasicBlock from `IRBB` containing VPIRInstructions for all instructions in `IRBB`, except its terminator which is managed by the successors of the block in VPlan.

Static Public Member Functions
static VPlanPtr	createInitialVPlan (Type InductionTy, PredicatedScalarEvolution &PSE, bool RequiresScalarEpilogueCheck, bool TailFolded, Loop TheLoop)
	Create initial VPlan, having an "entry" VPBasicBlock (wrapping original scalar pre-header) which contains SCEV expansions that need to happen before the CFG is modified (when executing a VPlan for the epilogue vector loop, the original entry needs to be replaced by a new one); a VPBasicBlock for the vector pre-header, followed by a region for the vector loop, followed by the middle VPBasicBlock.

Friends
class	VPlanPrinter

class	VPSlotTracker

Detailed Description

VPlan models a candidate for vectorization, encoding various decisions take to produce efficient output IR, including which branches, basic-blocks and output IR instructions to generate, and their cost.

VPlan holds a Hierarchical-CFG of VPBasicBlocks and VPRegionBlocks rooted at an Entry VPBasicBlock.

Definition at line 3804 of file VPlan.h.

Constructor & Destructor Documentation

◆ VPlan() [1/2]

VPlan::VPlan ( Loop * L )

Construct a VPlan for L.

This will create VPIRBasicBlocks wrapping the original preheader and scalar header of L, to be used as entry and scalar header blocks of the new VPlan.

Definition at line 817 of file VPlan.cpp.

◆ VPlan() [2/2]

llvm::VPlan::VPlan	(	BasicBlock *	ScalarHeaderBB,
		VPValue *	TC
	)

inline

Construct a VPlan with a new VPBasicBlock as entry, a VPIRBasicBlock wrapping ScalarHeaderBB and a trip count of TC.

Definition at line 3879 of file VPlan.h.

References createVPBasicBlock(), createVPIRBasicBlock(), and setEntry().

◆ ~VPlan()

VPlan::~VPlan ( )

Definition at line 822 of file VPlan.cpp.

References I.

Member Function Documentation

◆ addSCEVExpansion()

void llvm::VPlan::addSCEVExpansion	(	const SCEV *	S,
		VPValue *	V
	)

inline

Definition at line 4075 of file VPlan.h.

References assert(), and llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::contains().

Referenced by llvm::vputils::getOrCreateVPValueForSCEVExpr().

◆ addVF()

void llvm::VPlan::addVF ( ElementCount VF )

inline

Definition at line 3987 of file VPlan.h.

References llvm::SetVector< T, Vector, Set, N >::insert().

◆ cost()

InstructionCost VPlan::cost	(	ElementCount	VF,
		VPCostContext &	Ctx
	)

Return the cost of this plan.

Definition at line 1046 of file VPlan.cpp.

References llvm::VPTransformState::VF.

◆ createEmptyVPIRBasicBlock()

VPIRBasicBlock * VPlan::createEmptyVPIRBasicBlock ( BasicBlock * IRBB )

Create a VPIRBasicBlock wrapping IRBB, but do not create VPIRInstructions wrapping the instructions in tIRBB.

The returned block is owned by the VPlan and deleted once the VPlan is destroyed.

Definition at line 1246 of file VPlan.cpp.

◆ createInitialVPlan()

VPlanPtr VPlan::createInitialVPlan	(	Type *	InductionTy,
		PredicatedScalarEvolution &	PSE,
		bool	RequiresScalarEpilogueCheck,
		bool	TailFolded,
		Loop *	TheLoop
	)

static

Create initial VPlan, having an "entry" VPBasicBlock (wrapping original scalar pre-header) which contains SCEV expansions that need to happen before the CFG is modified (when executing a VPlan for the epilogue vector loop, the original entry needs to be replaced by a new one); a VPBasicBlock for the vector pre-header, followed by a region for the vector loop, followed by the middle VPBasicBlock.

If a check is needed to guard executing the scalar epilogue loop, it will be added to the middle block, together with VPBasicBlocks for the scalar preheader and exit blocks. InductionTy is the type of the canonical induction and used for related values, like the trip count expression.

Definition at line 845 of file VPlan.cpp.

References assert(), llvm::VPInstruction::BranchOnCond, llvm::VPTransformState::Builder, llvm::VPBlockUtils::connectBlocks(), createVPBasicBlock(), createVPIRBasicBlock(), createVPRegionBlock(), llvm::Type::getContext(), getEntry(), llvm::Type::getInt1Ty(), llvm::LoopBase< BlockT, LoopT >::getLoopLatch(), getOrAddLiveIn(), llvm::vputils::getOrCreateVPValueForSCEVExpr(), getScalarHeader(), llvm::PredicatedScalarEvolution::getSE(), llvm::PredicatedScalarEvolution::getSymbolicMaxBackedgeTakenCount(), llvm::BasicBlock::getTerminator(), getTripCount(), llvm::ScalarEvolution::getTripCountFromExitCount(), llvm::ConstantInt::getTrue(), llvm::SCEV::getType(), llvm::LoopBase< BlockT, LoopT >::getUniqueLatchExitBlock(), getVectorTripCount(), llvm::CmpInst::ICMP_EQ, llvm::VPBlockUtils::insertBlockAfter(), and llvm::VPTransformState::Plan.

◆ createVPBasicBlock()

VPBasicBlock * llvm::VPlan::createVPBasicBlock	(	const Twine &	Name,
		VPRecipeBase *	Recipe = `nullptr`
	)

inline

Create a new VPBasicBlock with Name and containing Recipe if present.

The returned block is owned by the VPlan and deleted once the VPlan is destroyed.

Definition at line 4087 of file VPlan.h.

References Name, and llvm::SmallVectorTemplateBase< T, bool >::push_back().

Referenced by createInitialVPlan(), createReplicateRegion(), llvm::VPlanTransforms::handleUncountableEarlyExit(), and VPlan().

◆ createVPIRBasicBlock()

VPIRBasicBlock * VPlan::createVPIRBasicBlock ( BasicBlock * IRBB )

Create a VPIRBasicBlock from IRBB containing VPIRInstructions for all instructions in IRBB, except its terminator which is managed by the successors of the block in VPlan.

The returned block is owned by the VPlan and deleted once the VPlan is destroyed.

Definition at line 1252 of file VPlan.cpp.

References llvm::BasicBlock::begin(), llvm::ilist_node_impl< OptionsT >::getIterator(), llvm::BasicBlock::getTerminator(), I, and llvm::make_range().

Referenced by createInitialVPlan(), llvm::EpilogueVectorizerEpilogueLoop::emitMinimumVectorEpilogueIterCountCheck(), llvm::VPlanTransforms::handleUncountableEarlyExit(), llvm::InnerLoopVectorizer::introduceCheckBlockInVPlan(), replaceVPBBWithIRVPBB(), and VPlan().

◆ createVPRegionBlock() [1/2]

VPRegionBlock * llvm::VPlan::createVPRegionBlock	(	const std::string &	Name = `""`,
		bool	IsReplicator = `false`
	)

inline

Create a new VPRegionBlock with Name and entry and exiting blocks set to nullptr.

If IsReplicator is true, the region is a replicate region. The returned block is owned by the VPlan and deleted once the VPlan is destroyed.

Definition at line 4109 of file VPlan.h.

References Name, and llvm::SmallVectorTemplateBase< T, bool >::push_back().

◆ createVPRegionBlock() [2/2]

VPRegionBlock * llvm::VPlan::createVPRegionBlock	(	VPBlockBase *	Entry,
		VPBlockBase *	Exiting,
		const std::string &	Name = `""`,
		bool	IsReplicator = `false`
	)

inline

Create a new VPRegionBlock with Entry, Exiting and Name.

If IsReplicator is true, the region is a replicate region. The returned block is owned by the VPlan and deleted once the VPlan is destroyed.

Definition at line 4097 of file VPlan.h.

References Name, and llvm::SmallVectorTemplateBase< T, bool >::push_back().

Referenced by createInitialVPlan(), and createReplicateRegion().

◆ dump()

LLVM_DUMP_METHOD void VPlan::dump ( ) const

Dump the plan to stderr (for debugging).

Definition at line 1151 of file VPlan.cpp.

References llvm::dbgs(), and llvm::print().

◆ duplicate()

VPlan * VPlan::duplicate ( )

Clone the current VPlan, update all VPValues of the new VPlan and cloned recipes to refer to the clones, and return it.

Definition at line 1192 of file VPlan.cpp.

References assert(), cloneFrom(), llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::contains(), llvm::find_if(), I, Name, remapOperands(), llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::size(), llvm::VPTransformState::VF, and llvm::vp_depth_first_shallow().

Referenced by llvm::LoopVectorizePass::processLoop().

◆ execute()

void VPlan::execute ( VPTransformState * State )

Generate the IR code for this VPlan.

Generate the code inside the preheader and body of the vectorized loop.

Assumes a single pre-header basic-block was created for this. Introduce additional basic-blocks as needed, and fill them all.

Definition at line 956 of file VPlan.cpp.

Referenced by llvm::LoopVectorizationPlanner::executePlan().

◆ getCanonicalIV()

VPCanonicalIVPHIRecipe * llvm::VPlan::getCanonicalIV ( )

inline

Returns the canonical induction recipe of the vector loop.

Definition at line 4062 of file VPlan.h.

References llvm::VPBasicBlock::begin(), llvm::VPBasicBlock::empty(), llvm::VPBlockBase::getEntryBasicBlock(), llvm::VPBlockBase::getSingleSuccessor(), and getVectorLoopRegion().

Referenced by llvm::VPlanTransforms::addActiveLaneMask(), addExitUsersForFirstOrderRecurrences(), addScalarResumePhis(), addVPLaneMaskPhiAndUpdateExitBranch(), collectAllHeaderMasks(), llvm::VPRecipeBuilder::createHeaderMask(), createScalarIVSteps(), llvm::VPlanTransforms::optimize(), llvm::VPlanTransforms::optimizeForVFAndUF(), preparePlanForMainVectorLoop(), removeRedundantCanonicalIVs(), transformRecipestoEVLRecipes(), llvm::VPlanTransforms::truncateToMinimalBitwidths(), llvm::VPlanTransforms::tryAddExplicitVectorLength(), and willGenerateVectors().

◆ getEntry() [1/2]

VPBasicBlock * llvm::VPlan::getEntry ( )

inline

◆ getEntry() [2/2]

const VPBasicBlock * llvm::VPlan::getEntry ( ) const

inline

Definition at line 3918 of file VPlan.h.

◆ getExitBlocks()

auto llvm::VPlan::getExitBlocks ( )

inline

Return an iterator range over the VPIRBasicBlock wrapping the exit blocks of the VPlan, that is leaf nodes except the scalar header.

Defined in VPlanHCFG, as the definition of the type needs access to the definitions of VPBlockShallowTraversalWrapper.

Definition at line 309 of file VPlanCFG.h.

References llvm::VPBlockBase::getNumSuccessors(), getScalarHeader(), getVectorLoopRegion(), llvm::make_filter_range(), and llvm::vp_depth_first_shallow().

Referenced by collectUsersInExitBlocks().

◆ getLiveIn()

VPValue * llvm::VPlan::getLiveIn ( Value * V ) const

inline

Return the live-in VPValue for V, if there is one or nullptr otherwise.

Definition at line 4045 of file VPlan.h.

References llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::lookup().

◆ getMiddleBlock() [1/2]

VPBasicBlock * llvm::VPlan::getMiddleBlock ( )

inline

Definition at line 3939 of file VPlan.h.

References getScalarPreheader().

◆ getMiddleBlock() [2/2]

const VPBasicBlock * llvm::VPlan::getMiddleBlock ( ) const

inline

Returns the 'middle' block of the plan, that is the block that selects whether to execute the scalar tail loop or the exit block from the loop latch.

Definition at line 3936 of file VPlan.h.

References getScalarPreheader().

Referenced by addExitUsersForFirstOrderRecurrences(), addScalarResumePhis(), addUsersInExitBlocks(), collectUsersInExitBlocks(), llvm::InnerLoopVectorizer::createVectorLoopSkeleton(), llvm::LoopVectorizationPlanner::executePlan(), and llvm::VPlanTransforms::handleUncountableEarlyExit().

◆ getName()

std::string VPlan::getName ( ) const

Return a string with the name of the plan and the applicable VFs and UFs.

Definition at line 1121 of file VPlan.cpp.

References llvm::drop_begin(), Name, and llvm::VPTransformState::VF.

Referenced by llvm::VPlanPrinter::dump().

◆ getOrAddLiveIn()

VPValue * llvm::VPlan::getOrAddLiveIn ( Value * V )

inline

Gets the live-in VPValue for V or adds a new live-in (if none exists yet) for V.

Definition at line 4028 of file VPlan.h.

References assert(), llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::count(), llvm::VPValue::isLiveIn(), and llvm::SmallVectorTemplateBase< T, bool >::push_back().

Referenced by addCanonicalIVRecipes(), addExitUsersForFirstOrderRecurrences(), addScalarResumePhis(), addUsersInExitBlocks(), createInitialVPlan(), llvm::VPlanTransforms::createInterleaveGroups(), llvm::vputils::getOrCreateVPValueForSCEVExpr(), llvm::VPRecipeBuilder::getVPValueOrAddLiveIn(), legalizeAndOptimizeInductions(), llvm::VPlanTransforms::optimizeForVFAndUF(), preparePlanForEpilogueVectorLoop(), transformRecipestoEVLRecipes(), and llvm::VPlanTransforms::tryAddExplicitVectorLength().

◆ getOrCreateBackedgeTakenCount()

VPValue * llvm::VPlan::getOrCreateBackedgeTakenCount ( )

inline

The backedge taken count of the original loop.

Definition at line 3972 of file VPlan.h.

Referenced by llvm::VPRecipeBuilder::createHeaderMask(), and llvm::vputils::isHeaderMask().

◆ getScalarHeader()

VPIRBasicBlock * llvm::VPlan::getScalarHeader ( ) const

inline

Return the VPIRBasicBlock wrapping the header of the scalar loop.

Definition at line 3949 of file VPlan.h.

Referenced by addScalarResumePhis(), createInitialVPlan(), getExitBlocks(), getScalarPreheader(), and preparePlanForMainVectorLoop().

◆ getScalarPreheader()

VPBasicBlock * llvm::VPlan::getScalarPreheader ( ) const

inline

Return the VPBasicBlock for the preheader of the scalar loop.

Definition at line 3944 of file VPlan.h.

References getScalarHeader().

Referenced by addExitUsersForFirstOrderRecurrences(), addScalarResumePhis(), llvm::InnerLoopVectorizer::createVectorLoopSkeleton(), getMiddleBlock(), llvm::InnerLoopVectorizer::introduceCheckBlockInVPlan(), and preparePlanForMainVectorLoop().

◆ getSCEVExpansion()

VPValue * llvm::VPlan::getSCEVExpansion ( const SCEV * S ) const

inline

Definition at line 4071 of file VPlan.h.

References llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::lookup().

Referenced by llvm::InnerLoopVectorizer::fixupIVUsers(), and llvm::vputils::getOrCreateVPValueForSCEVExpr().

◆ getTripCount()

VPValue * llvm::VPlan::getTripCount ( ) const

inline

The trip count of the original loop.

Definition at line 3958 of file VPlan.h.

References assert().

Referenced by llvm::VPlanTransforms::addActiveLaneMask(), addVPLaneMaskPhiAndUpdateExitBranch(), createInitialVPlan(), llvm::LoopVectorizationPlanner::executePlan(), llvm::vputils::isHeaderMask(), llvm::VPlanTransforms::optimizeForVFAndUF(), preparePlanForEpilogueVectorLoop(), llvm::LoopVectorizationPlanner::selectEpilogueVectorizationFactor(), and llvm::VPlanTransforms::tryAddExplicitVectorLength().

◆ getUF()

unsigned llvm::VPlan::getUF ( ) const

inline

Definition at line 4010 of file VPlan.h.

References assert(), and llvm::SetVector< T, Vector, Set, N >::size().

Referenced by llvm::VPWidenPointerInductionRecipe::execute(), llvm::LoopVectorizationPlanner::executePlan(), and llvm::VPTransformState::setDebugLocFrom().

◆ getVectorLoopRegion() [1/2]

VPRegionBlock * VPlan::getVectorLoopRegion ( )

Returns the VPRegionBlock of the vector loop.

Definition at line 1052 of file VPlan.cpp.

References B, and llvm::vp_depth_first_shallow().

Referenced by addCanonicalIVRecipes(), addExitUsersForFirstOrderRecurrences(), addScalarResumePhis(), addVPLaneMaskPhiAndUpdateExitBranch(), llvm::VPlanTransforms::adjustFixedOrderRecurrences(), llvm::VPlanTransforms::clearReductionWrapFlags(), collectAllHeaderMasks(), llvm::collectEphemeralRecipesForVPlan(), llvm::VPRecipeBuilder::createHeaderMask(), createScalarIVSteps(), llvm::LoopVectorizationPlanner::executePlan(), llvm::InnerLoopVectorizer::fixVectorizedLoop(), getCanonicalIV(), getExitBlocks(), getVectorPreheader(), llvm::VPlanTransforms::handleUncountableEarlyExit(), isDefinedInsideLoopRegions(), legalizeAndOptimizeInductions(), licm(), llvm::VPlanTransforms::optimizeForVFAndUF(), planContainsAdditionalSimplifications(), preparePlanForEpilogueVectorLoop(), preparePlanForMainVectorLoop(), removeRedundantCanonicalIVs(), removeRedundantInductionCasts(), llvm::InnerLoopVectorizer::scalarizeInstruction(), llvm::VPlanTransforms::truncateToMinimalBitwidths(), llvm::VPlanTransforms::tryAddExplicitVectorLength(), llvm::VPlanTransforms::unrollByUF(), llvm::VPInterleavedAccessInfo::VPInterleavedAccessInfo(), and willGenerateVectors().

◆ getVectorLoopRegion() [2/2]

const VPRegionBlock * VPlan::getVectorLoopRegion ( ) const

Definition at line 1060 of file VPlan.cpp.

References B, and llvm::vp_depth_first_shallow().

◆ getVectorPreheader()

VPBasicBlock * llvm::VPlan::getVectorPreheader ( )

inline

Returns the preheader of the vector loop region, if one exists, or null otherwise.

Definition at line 3922 of file VPlan.h.

References llvm::VPBlockBase::getSinglePredecessor(), and getVectorLoopRegion().

Referenced by addVPLaneMaskPhiAndUpdateExitBranch(), llvm::VPTransformState::get(), licm(), and llvm::VPlanTransforms::truncateToMinimalBitwidths().

◆ getVectorTripCount()

VPValue & llvm::VPlan::getVectorTripCount ( )

inline

The vector trip count.

Definition at line 3979 of file VPlan.h.

Referenced by addCanonicalIVRecipes(), addScalarResumePhis(), createInitialVPlan(), and preparePlanForMainVectorLoop().

◆ getVF()

VPValue & llvm::VPlan::getVF ( )

inline

Returns the VF of the vector loop region.

Definition at line 3982 of file VPlan.h.

Referenced by createWidenInductionRecipes(), and transformRecipestoEVLRecipes().

◆ getVFxUF()

VPValue & llvm::VPlan::getVFxUF ( )

inline

Returns VF * UF of the vector loop region.

Definition at line 3985 of file VPlan.h.

Referenced by addCanonicalIVRecipes().

◆ hasScalableVF()

bool llvm::VPlan::hasScalableVF ( )

inline

Definition at line 3996 of file VPlan.h.

References llvm::any_of(), and llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::isScalable().

Referenced by legalizeAndOptimizeInductions().

◆ hasScalarVFOnly()

bool llvm::VPlan::hasScalarVFOnly ( ) const

inline

Definition at line 4006 of file VPlan.h.

References llvm::SetVector< T, Vector, Set, N >::size().

Referenced by sinkScalarOperands().

◆ hasUF()

bool llvm::VPlan::hasUF ( unsigned UF ) const

inline

Definition at line 4008 of file VPlan.h.

References llvm::SetVector< T, Vector, Set, N >::contains(), and llvm::SetVector< T, Vector, Set, N >::empty().

Referenced by llvm::LoopVectorizationPlanner::executePlan(), llvm::VPlanTransforms::optimizeForVFAndUF(), and setUF().

◆ hasVF()

bool llvm::VPlan::hasVF ( ElementCount VF )

inline

Definition at line 3995 of file VPlan.h.

References llvm::SetVector< T, Vector, Set, N >::count().

Referenced by llvm::LoopVectorizationPlanner::executePlan(), llvm::LoopVectorizationPlanner::getPlanFor(), legalizeAndOptimizeInductions(), llvm::VPlanTransforms::optimizeForVFAndUF(), and setVF().

◆ prepareToExecute()

void VPlan::prepareToExecute	(	Value *	TripCount,
		Value *	VectorTripCount,
		VPTransformState &	State
	)

Prepare the plan for execution, setting up the required live-in values.

Definition at line 924 of file VPlan.cpp.

References assert(), llvm::VPTransformState::Builder, llvm::VPTransformState::CFG, llvm::IRBuilderBase::CreateMul(), llvm::createStepForVF(), llvm::IRBuilderBase::CreateSub(), llvm::getRuntimeVF(), llvm::BasicBlock::getTerminator(), llvm::Value::getType(), llvm::VPTransformState::CFGState::PrevBB, and llvm::VPTransformState::VF.

Referenced by llvm::LoopVectorizationPlanner::executePlan().

◆ print()

LLVM_DUMP_METHOD void VPlan::print ( raw_ostream & O ) const

Print this VPlan to O.

Definition at line 1104 of file VPlan.cpp.

References llvm::Block, and getName().

Referenced by llvm::operator<<(), and llvm::LoopVectorizationPlanner::printPlans().

◆ printDOT()

LLVM_DUMP_METHOD void VPlan::printDOT ( raw_ostream & O ) const

Print this VPlan in DOT format to O.

Definition at line 1145 of file VPlan.cpp.

References llvm::Metadata::dump(), and Printer.

Referenced by llvm::LoopVectorizationPlanner::printPlans().

◆ printLiveIns()

void VPlan::printLiveIns ( raw_ostream & O ) const

Print the live-ins of this VPlan to O.

Definition at line 1068 of file VPlan.cpp.

References llvm::VPTransformState::VF.

Referenced by llvm::VPlanPrinter::dump().

◆ resetTripCount()

void llvm::VPlan::resetTripCount ( VPValue * NewTripCount )

inline

Resets the trip count for the VPlan.

The caller must make sure all uses of the original trip count have been replaced.

Definition at line 3965 of file VPlan.h.

References assert(), and llvm::VPValue::getNumUsers().

Referenced by preparePlanForEpilogueVectorLoop().

◆ setEntry()

void llvm::VPlan::setEntry ( VPBasicBlock * VPBB )

inline

Definition at line 3887 of file VPlan.h.

References llvm::VPBlockBase::setPlan().

Referenced by llvm::EpilogueVectorizerEpilogueLoop::emitMinimumVectorEpilogueIterCountCheck(), and VPlan().

◆ setName()

void llvm::VPlan::setName ( const Twine & newName )

inline

Definition at line 4024 of file VPlan.h.

References Name, and llvm::Twine::str().

Referenced by llvm::VPlanHCFGBuilder::buildHierarchicalCFG().

◆ setUF()

void llvm::VPlan::setUF ( unsigned UF )

inline

Definition at line 4015 of file VPlan.h.

References assert(), llvm::SetVector< T, Vector, Set, N >::clear(), hasUF(), and llvm::SetVector< T, Vector, Set, N >::insert().

Referenced by llvm::VPlanTransforms::optimizeForVFAndUF(), llvm::VPlanTransforms::tryAddExplicitVectorLength(), and llvm::VPlanTransforms::unrollByUF().

◆ setVF()

void llvm::VPlan::setVF ( ElementCount VF )

inline

Definition at line 3989 of file VPlan.h.

References assert(), llvm::SetVector< T, Vector, Set, N >::clear(), hasVF(), and llvm::SetVector< T, Vector, Set, N >::insert().

Referenced by llvm::VPlanTransforms::optimizeForVFAndUF().

◆ vectorFactors()

iterator_range< SmallSetVector< ElementCount, 2 >::iterator > llvm::VPlan::vectorFactors ( ) const

inline

Returns an iterator range over all VFs of the plan.

Definition at line 4002 of file VPlan.h.

References llvm::SetVector< T, Vector, Set, N >::begin(), and llvm::SetVector< T, Vector, Set, N >::end().

Referenced by llvm::LoopVectorizationPlanner::computeBestVF().

Friends And Related Function Documentation

◆ VPlanPrinter

friend class VPlanPrinter

friend

Definition at line 3805 of file VPlan.h.

◆ VPSlotTracker

friend class VPSlotTracker

friend

Definition at line 3806 of file VPlan.h.

The documentation for this class was generated from the following files:

lib/Transforms/Vectorize/VPlan.h
lib/Transforms/Vectorize/VPlan.cpp
lib/Transforms/Vectorize/VPlanCFG.h

Public Member Functions

Static Public Member Functions

Friends

Detailed Description

Constructor & Destructor Documentation

◆ VPlan() [1/2]

◆ VPlan() [2/2]

◆ ~VPlan()

Member Function Documentation

◆ addSCEVExpansion()

◆ addVF()

◆ cost()

◆ createEmptyVPIRBasicBlock()

◆ createInitialVPlan()

◆ createVPBasicBlock()

◆ createVPIRBasicBlock()

◆ createVPRegionBlock() [1/2]

◆ createVPRegionBlock() [2/2]

◆ dump()

◆ duplicate()

◆ execute()

◆ getCanonicalIV()

◆ getEntry() [1/2]

◆ getEntry() [2/2]

◆ getExitBlocks()

◆ getLiveIn()

◆ getMiddleBlock() [1/2]

◆ getMiddleBlock() [2/2]

◆ getName()

◆ getOrAddLiveIn()

◆ getOrCreateBackedgeTakenCount()

◆ getScalarHeader()

◆ getScalarPreheader()

◆ getSCEVExpansion()

◆ getTripCount()

◆ getUF()

◆ getVectorLoopRegion() [1/2]

◆ getVectorLoopRegion() [2/2]

◆ getVectorPreheader()

◆ getVectorTripCount()

◆ getVF()

◆ getVFxUF()

◆ hasScalableVF()

◆ hasScalarVFOnly()

◆ hasUF()

◆ hasVF()

◆ prepareToExecute()

◆ print()

◆ printDOT()

◆ printLiveIns()

◆ resetTripCount()

◆ setEntry()

◆ setName()

◆ setUF()

◆ setVF()

◆ vectorFactors()

Friends And Related Function Documentation

◆ VPlanPrinter

◆ VPSlotTracker