llvm::LoopVectorizationPlanner Class Reference

Planner drives the vectorization process after having passed Legality checks. More...

#include "Transforms/Vectorize/LoopVectorizationPlanner.h"

Public Member Functions
	LoopVectorizationPlanner (Loop *L, LoopInfo *LI, const TargetLibraryInfo *TLI, const TargetTransformInfo *TTI, LoopVectorizationLegality *Legal, LoopVectorizationCostModel &CM, InterleavedAccessInfo &IAI, PredicatedScalarEvolution &PSE, const LoopVectorizeHints &Hints, OptimizationRemarkEmitter *ORE)
std::optional< VectorizationFactor >	plan (ElementCount UserVF, unsigned UserIC)
	Plan how to best vectorize, return the best VF and its cost, or std::nullopt if vectorization and interleaving should be avoided up front.
VectorizationFactor	planInVPlanNativePath (ElementCount UserVF)
	Use the VPlan-native path to plan how to best vectorize, return the best VF and its cost.
VPlan &	getBestPlanFor (ElementCount VF) const
	Return the best VPlan for VF.
void	executePlan (ElementCount VF, unsigned UF, VPlan &BestPlan, InnerLoopVectorizer &LB, DominatorTree *DT, bool IsEpilogueVectorization)
	Generate the IR code for the body of the vectorized loop according to the best selected VF, UF and VPlan BestPlan.
void	printPlans (raw_ostream &O)
bool	hasPlanWithVF (ElementCount VF) const
	Look through the existing plans and return true if we have one with all the vectorization factors in question.
bool	requiresTooManyRuntimeChecks () const
	Check if the number of runtime checks exceeds the threshold.

Static Public Member Functions
static bool	getDecisionAndClampRange (const std::function< bool(ElementCount)> &Predicate, VFRange &Range)
	Test a Predicate on a Range of VF's.

Protected Member Functions
void	buildVPlans (ElementCount MinVF, ElementCount MaxVF)
	Build VPlans for power-of-2 VF's between MinVF and MaxVF inclusive, according to the information gathered by Legal when it checked if it is legal to vectorize the loop.

Detailed Description

Planner drives the vectorization process after having passed Legality checks.

Definition at line 253 of file LoopVectorizationPlanner.h.

Constructor & Destructor Documentation

LoopVectorizationPlanner()

llvm::LoopVectorizationPlanner::LoopVectorizationPlanner ( Loop *  L, LoopInfo *  LI, const TargetLibraryInfo *  TLI, const TargetTransformInfo *  TTI, LoopVectorizationLegality *  Legal, LoopVectorizationCostModel &  CM, InterleavedAccessInfo &  IAI, PredicatedScalarEvolution &  PSE, const LoopVectorizeHints &  Hints, OptimizationRemarkEmitter *  ORE  )

inline

Definition at line 287 of file LoopVectorizationPlanner.h.

References llvm::IRSimilarity::Legal.

Member Function Documentation

buildVPlans()

void LoopVectorizationPlanner::buildVPlans ( ElementCount  MinVF, ElementCount  MaxVF  )

protected

Build VPlans for power-of-2 VF's between MinVF and MaxVF inclusive, according to the information gathered by Legal when it checked if it is legal to vectorize the loop.

Build VPlans for the full range of feasible VF's = {MinVF, 2 * MinVF, 4 * MinVF, ..., MaxVF} by repeatedly building a VPlan for a sub-range of VF's starting at a given VF and extending it as much as possible.

Each vectorization decision can potentially shorten this sub-range during buildVPlan().

Definition at line 8054 of file LoopVectorize.cpp.

References llvm::VFRange::End, llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::getWithIncrement(), llvm::details::FixedOrScalableQuantity< ElementCount, unsigned >::isKnownLT(), and llvm::InnerLoopVectorizer::VF.

executePlan()

void LoopVectorizationPlanner::executePlan	(	ElementCount	VF,
		unsigned	UF,
		VPlan &	BestPlan,
		InnerLoopVectorizer &	LB,
		DominatorTree *	DT,
		bool	IsEpilogueVectorization
	)

Generate the IR code for the body of the vectorized loop according to the best selected VF, UF and VPlan BestPlan.

TODO: IsEpilogueVectorization is needed to avoid issues due to epilogue vectorization re-using plans for both the main and epilogue vector loops. It should be removed once the re-use issue has been fixed.

Definition at line 7645 of file LoopVectorize.cpp.

References AddRuntimeUnrollDisableMetaData(), assert(), llvm::InnerLoopVectorizer::Builder, llvm::InnerLoopVectorizer::collectPoisonGeneratingRecipes(), llvm::InnerLoopVectorizer::createVectorizedLoopSkeleton(), llvm::dbgs(), llvm::InnerLoopVectorizer::DT, llvm::SmallVectorBase< Size_T >::empty(), llvm::VPlan::execute(), llvm::InnerLoopVectorizer::fixVectorizedLoop(), llvm::RuntimePointerChecking::getChecks(), llvm::RuntimePointerChecking::getDiffChecks(), llvm::VPBlockBase::getEntryBasicBlock(), llvm::LoopVectorizationLegality::getLAI(), llvm::LoopInfoBase< BlockT, LoopT >::getLoopFor(), llvm::Loop::getLoopID(), llvm::LoopBase< BlockT, LoopT >::getLoopPreheader(), llvm::InnerLoopVectorizer::getOrCreateTripCount(), llvm::InnerLoopVectorizer::getOrCreateVectorTripCount(), llvm::LoopAccessInfo::getRuntimePointerChecking(), llvm::PredicatedScalarEvolution::getSE(), llvm::VPlan::getVectorLoopRegion(), llvm::VPlan::hasUF(), llvm::VPlan::hasVF(), llvm::InnerLoopVectorizer::Legal, llvm::InnerLoopVectorizer::LI, LLVM_DEBUG, LLVMLoopVectorizeFollowupAll, LLVMLoopVectorizeFollowupVectorized, llvm::makeFollowupLoopID(), llvm::VPlanTransforms::optimizeForVFAndUF(), llvm::InnerLoopVectorizer::ORE, llvm::InnerLoopVectorizer::OrigLoop, llvm::VPlan::prepareToExecute(), llvm::InnerLoopVectorizer::printDebugTracesAtEnd(), llvm::InnerLoopVectorizer::printDebugTracesAtStart(), llvm::InnerLoopVectorizer::PSE, and llvm::LoopVectorizeHints::setAlreadyVectorized().

Referenced by llvm::LoopVectorizePass::processLoop(), and processLoopInVPlanNativePath().

getBestPlanFor()

VPlan & LoopVectorizationPlanner::getBestPlanFor

(

ElementCount

VF

)

const

Return the best VPlan for VF.

Definition at line 7598 of file LoopVectorize.cpp.

References assert(), llvm::count_if(), llvm_unreachable, and llvm::InnerLoopVectorizer::VF.

Referenced by llvm::LoopVectorizePass::processLoop(), and processLoopInVPlanNativePath().

getDecisionAndClampRange()

bool LoopVectorizationPlanner::getDecisionAndClampRange ( const std::function< bool(ElementCount)> &  Predicate, VFRange &  Range  )

static

Test a Predicate on a Range of VF's.

Return the value of applying Predicate on Range.Start, possibly decreasing Range.End such that the returned value holds for the entire Range.

Definition at line 8034 of file LoopVectorize.cpp.

References assert(), and llvm::details::FixedOrScalableQuantity< ElementCount, unsigned >::isKnownLT().

Referenced by createWidenInductionRecipes(), llvm::VPRecipeBuilder::handleReplication(), and llvm::VPRecipeBuilder::tryToCreateWidenRecipe().

hasPlanWithVF()

bool llvm::LoopVectorizationPlanner::hasPlanWithVF ( ElementCount  VF ) const

inline

Look through the existing plans and return true if we have one with all the vectorization factors in question.

Definition at line 324 of file LoopVectorizationPlanner.h.

References llvm::any_of().

Referenced by llvm::LoopVectorizationCostModel::selectEpilogueVectorizationFactor().

plan()

std::optional< VectorizationFactor > LoopVectorizationPlanner::plan	(	ElementCount	UserVF,
		unsigned	UserIC
	)

Plan how to best vectorize, return the best VF and its cost, or std::nullopt if vectorization and interleaving should be avoided up front.

Definition at line 7526 of file LoopVectorize.cpp.

References assert(), llvm::dbgs(), llvm::VectorizationFactor::Disabled(), llvm::FixedScalableVFPair::FixedVF, llvm::ElementCount::getFixed(), llvm::LoopBase< BlockT, LoopT >::getHeader(), llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::getKnownMinValue(), llvm::ElementCount::getScalable(), llvm::FixedScalableVFPair::hasVector(), llvm::SmallSet< T, N, C >::insert(), llvm::LoopBase< BlockT, LoopT >::isInnermost(), llvm::details::FixedOrScalableQuantity< ElementCount, unsigned >::isKnownLE(), llvm::isPowerOf2_32(), llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::isScalable(), llvm::ElementCount::isScalar(), llvm::ElementCount::isVector(), llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::isZero(), LLVM_DEBUG, llvm::InnerLoopVectorizer::ORE, llvm::InnerLoopVectorizer::OrigLoop, llvm::reportVectorizationInfo(), llvm::FixedScalableVFPair::ScalableVF, useMaskedInterleavedAccesses(), and llvm::InnerLoopVectorizer::VF.

Referenced by llvm::LoopVectorizePass::processLoop().

planInVPlanNativePath()

VectorizationFactor LoopVectorizationPlanner::planInVPlanNativePath

(

ElementCount

UserVF

)

Use the VPlan-native path to plan how to best vectorize, return the best VF and its cost.

Definition at line 7481 of file LoopVectorize.cpp.

References assert(), llvm::dbgs(), determineVPlanVF(), llvm::VectorizationFactor::Disabled(), llvm::EnableVPlanNativePath, llvm::ElementCount::getFixed(), llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::getFixedValue(), llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::getKnownMinValue(), llvm::TargetTransformInfo::getRegisterBitWidth(), llvm::LoopBase< BlockT, LoopT >::isInnermost(), llvm::isPowerOf2_32(), llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::isScalable(), llvm::ElementCount::isScalar(), llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::isZero(), LLVM_DEBUG, llvm::InnerLoopVectorizer::OrigLoop, llvm::TargetTransformInfo::RGK_FixedWidthVector, llvm::InnerLoopVectorizer::VF, and VPlanBuildStressTest.

Referenced by processLoopInVPlanNativePath().

printPlans()

void LoopVectorizationPlanner::printPlans

(

raw_ostream &

O

)

Definition at line 7747 of file LoopVectorize.cpp.

References PrintVPlansInDotFormat.

requiresTooManyRuntimeChecks()

bool llvm::LoopVectorizationPlanner::requiresTooManyRuntimeChecks

(

)

const

Check if the number of runtime checks exceeds the threshold.

---

The documentation for this class was generated from the following files:

• lib/Transforms/Vectorize/LoopVectorizationPlanner.h
• lib/Transforms/Vectorize/LoopVectorize.cpp