LLVM 17.0.0git
|
Helper class to create VPRecipies from IR instructions. More...
#include "Transforms/Vectorize/VPRecipeBuilder.h"
Public Member Functions | |
VPRecipeBuilder (Loop *OrigLoop, const TargetLibraryInfo *TLI, LoopVectorizationLegality *Legal, LoopVectorizationCostModel &CM, PredicatedScalarEvolution &PSE, VPBuilder &Builder) | |
VPRecipeOrVPValueTy | tryToCreateWidenRecipe (Instruction *Instr, ArrayRef< VPValue * > Operands, VFRange &Range, VPBasicBlock *VPBB, VPlanPtr &Plan) |
Check if an existing VPValue can be used for Instr or a recipe can be create for I withing the given VF Range . | |
void | setRecipe (Instruction *I, VPRecipeBase *R) |
Set the recipe created for given ingredient. | |
VPValue * | createBlockInMask (BasicBlock *BB, VPlanPtr &Plan) |
A helper function that computes the predicate of the block BB, assuming that the header block of the loop is set to True. | |
VPValue * | createEdgeMask (BasicBlock *Src, BasicBlock *Dst, VPlanPtr &Plan) |
A helper function that computes the predicate of the edge between SRC and DST. | |
void | recordRecipeOf (Instruction *I) |
Mark given ingredient for recording its recipe once one is created for it. | |
VPRecipeBase * | getRecipe (Instruction *I) |
Return the recipe created for given ingredient. | |
VPRegionBlock * | createReplicateRegion (VPReplicateRecipe *PredRecipe, VPlanPtr &Plan) |
Create a replicating region for PredRecipe . | |
VPBasicBlock * | handleReplication (Instruction *I, VFRange &Range, VPBasicBlock *VPBB, VPlanPtr &Plan) |
Build a VPReplicationRecipe for I and enclose it within a Region if it is predicated. | |
void | fixHeaderPhis () |
Add the incoming values from the backedge to reduction & first-order recurrence cross-iteration phis. | |
Helper class to create VPRecipies from IR instructions.
Definition at line 27 of file VPRecipeBuilder.h.
|
inline |
Definition at line 110 of file VPRecipeBuilder.h.
References Builder, and llvm::IRSimilarity::Legal.
VPValue * VPRecipeBuilder::createBlockInMask | ( | BasicBlock * | BB, |
VPlanPtr & | Plan | ||
) |
A helper function that computes the predicate of the block BB, assuming that the header block of the loop is set to True.
It returns the entry mask for the block BB.
Definition at line 8090 of file LoopVectorize.cpp.
References llvm::VPInstruction::ActiveLaneMask, assert(), llvm::InnerLoopVectorizer::Builder, llvm::LoopBase< BlockT, LoopT >::contains(), llvm::DataAndControlFlow, llvm::DataWithoutLaneMask, llvm::VPBlockBase::getEntryBasicBlock(), llvm::VPBasicBlock::getFirstNonPhi(), llvm::LoopBase< BlockT, LoopT >::getHeader(), llvm::VPInstruction::ICmpULE, llvm::VPBasicBlock::insert(), IV, llvm::None, llvm::InnerLoopVectorizer::OrigLoop, and llvm::predecessors().
VPValue * VPRecipeBuilder::createEdgeMask | ( | BasicBlock * | Src, |
BasicBlock * | Dst, | ||
VPlanPtr & | Plan | ||
) |
A helper function that computes the predicate of the edge between SRC and DST.
Definition at line 8045 of file LoopVectorize.cpp.
References assert(), llvm::InnerLoopVectorizer::Builder, llvm::BranchInst::getCondition(), llvm::Instruction::getDebugLoc(), llvm::ConstantInt::getFalse(), llvm::BranchInst::getSuccessor(), llvm::Value::getType(), llvm::is_contained(), llvm::BranchInst::isConditional(), llvm::LoopBase< BlockT, LoopT >::isLoopExiting(), llvm::InnerLoopVectorizer::OrigLoop, and llvm::predecessors().
VPRegionBlock * VPRecipeBuilder::createReplicateRegion | ( | VPReplicateRecipe * | PredRecipe, |
VPlanPtr & | Plan | ||
) |
Create a replicating region for PredRecipe
.
Definition at line 8557 of file LoopVectorize.cpp.
References assert(), llvm::VPBlockUtils::connectBlocks(), llvm::Instruction::getOpcodeName(), llvm::Instruction::getParent(), llvm::Value::getType(), llvm::VPRecipeBase::getUnderlyingInstr(), llvm::VPBlockUtils::insertTwoBlocksAfter(), and llvm::Type::isVoidTy().
void VPRecipeBuilder::fixHeaderPhis | ( | ) |
Add the incoming values from the backedge to reduction & first-order recurrence cross-iteration phis.
Definition at line 8465 of file LoopVectorize.cpp.
References llvm::LoopBase< BlockT, LoopT >::getLoopLatch(), llvm::VPDef::getVPSingleValue(), and llvm::InnerLoopVectorizer::OrigLoop.
|
inline |
Return the recipe created for given ingredient.
Definition at line 154 of file VPRecipeBuilder.h.
References assert(), llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::count(), and I.
VPBasicBlock * VPRecipeBuilder::handleReplication | ( | Instruction * | I, |
VFRange & | Range, | ||
VPBasicBlock * | VPBB, | ||
VPlanPtr & | Plan | ||
) |
Build a VPReplicationRecipe for I
and enclose it within a Region if it is predicated.
VPBB
augmented with this new recipe if I
is not predicated, otherwise I
. Range.End may be decreased to ensure same recipe behavior from Range.Start
to Range.End
. Definition at line 8475 of file LoopVectorize.cpp.
References llvm::VPBasicBlock::appendRecipe(), assert(), llvm::VPBlockUtils::connectBlocks(), llvm::dbgs(), llvm::VPBlockUtils::disconnectBlocks(), llvm::LoopVectorizationPlanner::getDecisionAndClampRange(), getIntrinsicID(), llvm::VPBlockBase::getSingleSuccessor(), I, llvm::VPBlockUtils::insertBlockAfter(), LLVM_DEBUG, and llvm::InnerLoopVectorizer::VF.
|
inline |
Mark given ingredient for recording its recipe once one is created for it.
Definition at line 147 of file VPRecipeBuilder.h.
References assert(), llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::count(), and I.
|
inline |
Set the recipe created for given ingredient.
This operation is a no-op for ingredients that were not marked using a nullptr entry in the map.
Definition at line 128 of file VPRecipeBuilder.h.
References assert(), llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::count(), and I.
VPRecipeOrVPValueTy VPRecipeBuilder::tryToCreateWidenRecipe | ( | Instruction * | Instr, |
ArrayRef< VPValue * > | Operands, | ||
VFRange & | Range, | ||
VPBasicBlock * | VPBB, | ||
VPlanPtr & | Plan | ||
) |
Check if an existing VPValue can be used for Instr
or a recipe can be create for I
withing the given VF Range
.
If an existing VPValue can be used or if a recipe can be created, return it. Otherwise return a VPRecipeOrVPValueTy with nullptr.
Definition at line 8594 of file LoopVectorize.cpp.
References assert(), GEP, llvm::LoopVectorizationPlanner::getDecisionAndClampRange(), llvm::LoopBase< BlockT, LoopT >::getHeader(), llvm::PHINode::getIncomingValueForBlock(), llvm::LoopBase< BlockT, LoopT >::getLoopLatch(), llvm::LoopBase< BlockT, LoopT >::getLoopPreheader(), llvm::Instruction::getParent(), llvm::RecurrenceDescriptor::getRecurrenceStartValue(), llvm::PredicatedScalarEvolution::getSCEV(), llvm::PredicatedScalarEvolution::getSE(), llvm::ScalarEvolution::isLoopInvariant(), llvm::ElementCount::isScalar(), llvm::IRSimilarity::Legal, llvm::make_range(), Operands, llvm::InnerLoopVectorizer::OrigLoop, llvm::InnerLoopVectorizer::PSE, SI, and llvm::InnerLoopVectorizer::VF.