|
LLVM 19.0.0git
|
Helper class to create VPRecipies from IR instructions. More...
#include "Transforms/Vectorize/VPRecipeBuilder.h"
Public Member Functions | |
| VPRecipeBuilder (VPlan &Plan, Loop *OrigLoop, const TargetLibraryInfo *TLI, LoopVectorizationLegality *Legal, LoopVectorizationCostModel &CM, PredicatedScalarEvolution &PSE, VPBuilder &Builder) | |
| VPRecipeBase * | tryToCreateWidenRecipe (Instruction *Instr, ArrayRef< VPValue * > Operands, VFRange &Range, VPBasicBlock *VPBB) |
Create and return a widened recipe for I if one can be created within the given VF Range. | |
| void | setRecipe (Instruction *I, VPRecipeBase *R) |
| Set the recipe created for given ingredient. | |
| void | createHeaderMask () |
| Create the mask for the vector loop header block. | |
| void | createBlockInMask (BasicBlock *BB) |
| A helper function that computes the predicate of the block BB, assuming that the header block of the loop is set to True or the loop mask when tail folding. | |
| VPValue * | getBlockInMask (BasicBlock *BB) const |
Returns the entry mask for the block BB. | |
| VPValue * | createEdgeMask (BasicBlock *Src, BasicBlock *Dst) |
| A helper function that computes the predicate of the edge between SRC and DST. | |
| VPValue * | getEdgeMask (BasicBlock *Src, BasicBlock *Dst) const |
| A helper that returns the previously computed predicate of the edge between SRC and DST. | |
| VPRecipeBase * | getRecipe (Instruction *I) |
| Return the recipe created for given ingredient. | |
| VPReplicateRecipe * | handleReplication (Instruction *I, VFRange &Range) |
Build a VPReplicationRecipe for I. | |
| void | fixHeaderPhis () |
| Add the incoming values from the backedge to reduction & first-order recurrence cross-iteration phis. | |
| iterator_range< mapped_iterator< Use *, std::function< VPValue *(Value *)> > > | mapToVPValues (User::op_range Operands) |
Returns a range mapping the values of the range Operands to their corresponding VPValues. | |
| VPValue * | getVPValueOrAddLiveIn (Value *V, VPlan &Plan) |
Helper class to create VPRecipies from IR instructions.
Definition at line 25 of file VPRecipeBuilder.h.
|
inline |
Definition at line 106 of file VPRecipeBuilder.h.
References llvm::IRSimilarity::Legal.
| void VPRecipeBuilder::createBlockInMask | ( | BasicBlock * | BB | ) |
A helper function that computes the predicate of the block BB, assuming that the header block of the loop is set to True or the loop mask when tail folding.
Definition at line 8056 of file LoopVectorize.cpp.
References assert(), llvm::LoopBase< BlockT, LoopT >::contains(), llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::count(), createEdgeMask(), llvm::VPBuilder::createOr(), llvm::LoopBase< BlockT, LoopT >::getHeader(), and llvm::predecessors().
| VPValue * VPRecipeBuilder::createEdgeMask | ( | BasicBlock * | Src, |
| BasicBlock * | Dst | ||
| ) |
A helper function that computes the predicate of the edge between SRC and DST.
Definition at line 7966 of file LoopVectorize.cpp.
References assert(), llvm::VPBuilder::createNot(), llvm::VPBuilder::createSelect(), llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::end(), llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::find(), getBlockInMask(), llvm::BranchInst::getCondition(), llvm::Instruction::getDebugLoc(), llvm::ConstantInt::getFalse(), llvm::VPlan::getOrAddLiveIn(), llvm::BranchInst::getSuccessor(), llvm::Value::getType(), getVPValueOrAddLiveIn(), llvm::is_contained(), llvm::BranchInst::isConditional(), llvm::LoopBase< BlockT, LoopT >::isLoopExiting(), and llvm::predecessors().
Referenced by createBlockInMask().
| void VPRecipeBuilder::createHeaderMask | ( | ) |
Create the mask for the vector loop header block.
Definition at line 8021 of file LoopVectorize.cpp.
References llvm::VPBuilder::createICmp(), llvm::LoopVectorizationCostModel::foldTailByMasking(), llvm::VPlan::getCanonicalIV(), llvm::VPBlockBase::getEntryBasicBlock(), llvm::VPBasicBlock::getFirstNonPhi(), llvm::LoopBase< BlockT, LoopT >::getHeader(), llvm::VPlan::getOrCreateBackedgeTakenCount(), llvm::VPlan::getVectorLoopRegion(), llvm::CmpInst::ICMP_ULE, llvm::VPBasicBlock::insert(), IV, and llvm::VPBuilder::setInsertPoint().
| void VPRecipeBuilder::fixHeaderPhis | ( | ) |
Add the incoming values from the backedge to reduction & first-order recurrence cross-iteration phis.
Definition at line 8394 of file LoopVectorize.cpp.
References llvm::LoopBase< BlockT, LoopT >::getLoopLatch(), getRecipe(), and llvm::VPDef::getVPSingleValue().
| VPValue * VPRecipeBuilder::getBlockInMask | ( | BasicBlock * | BB | ) | const |
Returns the entry mask for the block BB.
Definition at line 8048 of file LoopVectorize.cpp.
References assert(), llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::end(), and llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::find().
Referenced by createEdgeMask(), and handleReplication().
| VPValue * VPRecipeBuilder::getEdgeMask | ( | BasicBlock * | Src, |
| BasicBlock * | Dst | ||
| ) | const |
A helper that returns the previously computed predicate of the edge between SRC and DST.
Definition at line 8010 of file LoopVectorize.cpp.
References assert(), llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::end(), llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::find(), llvm::is_contained(), and llvm::predecessors().
|
inline |
Return the recipe created for given ingredient.
Definition at line 146 of file VPRecipeBuilder.h.
References assert(), llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::count(), and I.
Referenced by fixHeaderPhis().
Definition at line 168 of file VPRecipeBuilder.h.
References llvm::VPlan::getOrAddLiveIn(), I, and llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::lookup().
Referenced by addUsersInExitBlock(), and createEdgeMask().
| VPReplicateRecipe * VPRecipeBuilder::handleReplication | ( | Instruction * | I, |
| VFRange & | Range | ||
| ) |
Build a VPReplicationRecipe for I.
If it is predicated, add the mask as last operand. Range.End may be decreased to ensure same recipe behavior from Range.Start to Range.End.
Definition at line 8404 of file LoopVectorize.cpp.
References llvm::dbgs(), getBlockInMask(), llvm::LoopVectorizationPlanner::getDecisionAndClampRange(), getIntrinsicID(), I, llvm::LoopVectorizationCostModel::isPredicatedInst(), llvm::LoopVectorizationCostModel::isUniformAfterVectorization(), LLVM_DEBUG, and mapToVPValues().
| iterator_range< mapped_iterator< Use *, std::function< VPValue *(Value *)> > > VPRecipeBuilder::mapToVPValues | ( | User::op_range | Operands | ) |
Returns a range mapping the values of the range Operands to their corresponding VPValues.
Definition at line 7955 of file LoopVectorize.cpp.
References I, llvm::map_range(), and Operands.
Referenced by handleReplication().
|
inline |
Set the recipe created for given ingredient.
Definition at line 120 of file VPRecipeBuilder.h.
References assert(), llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::contains(), and I.
| VPRecipeBase * VPRecipeBuilder::tryToCreateWidenRecipe | ( | Instruction * | Instr, |
| ArrayRef< VPValue * > | Operands, | ||
| VFRange & | Range, | ||
| VPBasicBlock * | VPBB | ||
| ) |
Create and return a widened recipe for I if one can be created within the given VF Range.
Definition at line 8461 of file LoopVectorize.cpp.
References assert(), llvm::MapVector< KeyT, ValueT, MapType, VectorType >::find(), GEP, llvm::LoopVectorizationPlanner::getDecisionAndClampRange(), llvm::LoopBase< BlockT, LoopT >::getHeader(), llvm::LoopBase< BlockT, LoopT >::getLoopPreheader(), llvm::Instruction::getOpcode(), llvm::RecurrenceDescriptor::getRecurrenceStartValue(), llvm::LoopVectorizationLegality::getReductionVars(), llvm::Value::getType(), llvm::LoopVectorizationLegality::isFixedOrderRecurrence(), llvm::LoopVectorizationCostModel::isInLoopReduction(), llvm::LoopVectorizationLegality::isReductionVariable(), llvm::ElementCount::isScalar(), llvm::make_range(), Operands, and llvm::LoopVectorizationCostModel::useOrderedReductions().
1.9.6