|
LLVM 17.0.0git
|
#include "Transforms/Vectorize/VPlanTransforms.h"
Static Public Member Functions | |
| static void | VPInstructionsToVPRecipes (VPlanPtr &Plan, function_ref< const InductionDescriptor *(PHINode *)> GetIntOrFpInductionDescriptor, SmallPtrSetImpl< Instruction * > &DeadInstructions, ScalarEvolution &SE, const TargetLibraryInfo &TLI) |
Replaces the VPInstructions in Plan with corresponding widen recipes. | |
| static void | createAndOptimizeReplicateRegions (VPlan &Plan) |
| Wrap predicated VPReplicateRecipes with a mask operand in an if-then region block and remove the mask operand. | |
| static bool | mergeBlocksIntoPredecessors (VPlan &Plan) |
| Remove redundant VPBasicBlocks by merging them into their predecessor if the predecessor has a single successor. | |
| static void | removeRedundantInductionCasts (VPlan &Plan) |
| Remove redundant casts of inductions. | |
| static void | removeRedundantCanonicalIVs (VPlan &Plan) |
| Try to replace VPWidenCanonicalIVRecipes with a widened canonical IV recipe, if it exists. | |
| static void | removeDeadRecipes (VPlan &Plan) |
| static void | optimizeInductions (VPlan &Plan, ScalarEvolution &SE) |
| If any user of a VPWidenIntOrFpInductionRecipe needs scalar values, provide them by building scalar steps off of the canonical scalar IV and update the original IV's users. | |
| static void | removeRedundantExpandSCEVRecipes (VPlan &Plan) |
Remove redundant EpxandSCEVRecipes in Plan's entry block by replacing them with already existing recipes expanding the same SCEV expression. | |
| static void | adjustFixedOrderRecurrences (VPlan &Plan, VPBuilder &Builder) |
| Sink users of fixed-order recurrences after the recipe defining their previous value. | |
| static void | optimizeForVFAndUF (VPlan &Plan, ElementCount BestVF, unsigned BestUF, PredicatedScalarEvolution &PSE) |
Optimize Plan based on BestVF and BestUF. | |
Definition at line 31 of file VPlanTransforms.h.
Sink users of fixed-order recurrences after the recipe defining their previous value.
Then introduce FirstOrderRecurrenceSplice VPInstructions to combine the value from the recurrence phis and previous values. The current implementation assumes all users can be sunk after the previous value, which is enforced by earlier legality checks.
Definition at line 706 of file VPlanTransforms.cpp.
References assert(), Builder, llvm::VPInstruction::FirstOrderRecurrenceSplice, llvm::VPRegionBlock::getEntry(), llvm::VPBlockBase::getEntryBasicBlock(), llvm::VPBasicBlock::getFirstNonPhi(), llvm::ilist_node_impl< OptionsT >::getIterator(), llvm::VPRecipeBase::getParent(), llvm::VPlan::getVectorLoopRegion(), llvm::SmallPtrSetImpl< PtrType >::insert(), llvm::VPBasicBlock::phis(), llvm::SmallVectorTemplateBase< T, bool >::push_back(), llvm::DominatorTreeBase< NodeT, IsPostDom >::recalculate(), and sinkRecurrenceUsersAfterPrevious().
|
static |
Wrap predicated VPReplicateRecipes with a mask operand in an if-then region block and remove the mask operand.
Optimize the created regions by iteratively sinking scalar operands into the region, followed by merging regions until no improvements are remaining.
Definition at line 381 of file VPlanTransforms.cpp.
References addReplicateRegions(), mergeBlocksIntoPredecessors(), mergeReplicateRegionsIntoSuccessors(), and sinkScalarOperands().
Remove redundant VPBasicBlocks by merging them into their predecessor if the predecessor has a single successor.
Definition at line 392 of file VPlanTransforms.cpp.
References llvm::VPBlockUtils::connectBlocks(), llvm::VPBlockUtils::disconnectBlocks(), llvm::SmallVectorBase< Size_T >::empty(), llvm::VPBasicBlock::end(), llvm::VPlan::getEntry(), llvm::make_early_inc_range(), llvm::SmallVectorTemplateBase< T, bool >::push_back(), llvm::to_vector(), and llvm::vp_depth_first_deep().
Referenced by createAndOptimizeReplicateRegions().
|
static |
Optimize Plan based on BestVF and BestUF.
This may restrict the resulting plan to BestVF and BestUF.
Definition at line 573 of file VPlanTransforms.cpp.
References llvm::VPBasicBlock::appendRecipe(), assert(), llvm::VPBasicBlock::back(), llvm::VPInstruction::BranchOnCond, llvm::VPInstruction::BranchOnCount, llvm::CallingConv::C, canSimplifyBranchOnCond(), llvm::createTripCountSCEV(), llvm::VPlan::getCanonicalIV(), llvm::ScalarEvolution::getConstant(), llvm::ScalarEvolution::getContext(), llvm::VPBlockBase::getExitingBasicBlock(), llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::getKnownMinValue(), llvm::VPValue::getLiveInIRValue(), llvm::VPlan::getOrAddExternalDef(), llvm::PredicatedScalarEvolution::getSE(), llvm::VPHeaderPHIRecipe::getStartValue(), llvm::ConstantInt::getTrue(), llvm::SCEV::getType(), llvm::Value::getType(), llvm::VPlan::getVectorLoopRegion(), llvm::VPlan::hasUF(), llvm::VPlan::hasVF(), llvm::CmpInst::ICMP_ULE, llvm::ScalarEvolution::isKnownPredicate(), llvm::SCEV::isZero(), llvm::VPlan::setUF(), and llvm::VPlan::setVF().
Referenced by llvm::LoopVectorizationPlanner::executePlan().
|
static |
If any user of a VPWidenIntOrFpInductionRecipe needs scalar values, provide them by building scalar steps off of the canonical scalar IV and update the original IV's users.
This is an optional optimization to reduce the needs of vector extracts.
Definition at line 501 of file VPlanTransforms.cpp.
References E, llvm::VPlan::getCanonicalIV(), llvm::VPValue::getDefiningRecipe(), llvm::VPBlockBase::getEntryBasicBlock(), llvm::VPBasicBlock::getFirstNonPhi(), llvm::ElementCount::getFixed(), llvm::vputils::getOrCreateVPValueForSCEVExpr(), llvm::VPlan::getVectorLoopRegion(), llvm::VPlan::hasVF(), I, llvm::VPBasicBlock::insert(), llvm::VPCanonicalIVPHIRecipe::isCanonical(), llvm::none_of(), llvm::VPBasicBlock::phis(), ToRemove, and Users.
|
static |
Definition at line 484 of file VPlanTransforms.cpp.
References llvm::any_of(), llvm::VPlan::getEntry(), llvm::make_early_inc_range(), and llvm::reverse().
|
static |
Try to replace VPWidenCanonicalIVRecipes with a widened canonical IV recipe, if it exists.
Definition at line 451 of file VPlanTransforms.cpp.
References llvm::VPRecipeBase::eraseFromParent(), llvm::VPlan::getCanonicalIV(), llvm::VPBlockBase::getEntryBasicBlock(), llvm::VPWidenCanonicalIVRecipe::getScalarType(), llvm::VPlan::getVectorLoopRegion(), llvm::vputils::onlyFirstLaneUsed(), llvm::VPBasicBlock::phis(), llvm::VPValue::replaceAllUsesWith(), and llvm::VPValue::users().
|
static |
Remove redundant EpxandSCEVRecipes in Plan's entry block by replacing them with already existing recipes expanding the same SCEV expression.
Definition at line 547 of file VPlanTransforms.cpp.
References llvm::VPlan::getEntry(), llvm::VPBlockBase::getEntryBasicBlock(), I, llvm::DenseMapBase< DerivedT, KeyT, ValueT, KeyInfoT, BucketT >::insert(), and llvm::make_early_inc_range().
|
static |
Remove redundant casts of inductions.
Such redundant casts are casts of induction variables that can be ignored, because we already proved that the casted phi is equal to the uncasted phi in the vectorized loop. There is no need to vectorize the cast - the same value can be used for both the phi and casts in the vector loop.
Definition at line 419 of file VPlanTransforms.cpp.
References llvm::VPBlockBase::getEntryBasicBlock(), llvm::VPlan::getVectorLoopRegion(), llvm::VPDef::getVPSingleValue(), IV, llvm::VPBasicBlock::phis(), llvm::VPValue::replaceAllUsesWith(), llvm::reverse(), and llvm::VPValue::users().
|
static |
Replaces the VPInstructions in Plan with corresponding widen recipes.
Definition at line 26 of file VPlanTransforms.cpp.
References assert(), llvm::SmallPtrSetImpl< PtrType >::count(), llvm::VPDef::definedValues(), GEP, llvm::getLoadStorePointerOperand(), llvm::VPDef::getNumDefinedValues(), llvm::vputils::getOrCreateVPValueForSCEVExpr(), llvm::VPValue::getUnderlyingValue(), llvm::getVectorIntrinsicIDForCall(), llvm::VPDef::getVPSingleValue(), llvm::VPRecipeBase::insertBefore(), llvm::make_early_inc_range(), llvm::make_range(), llvm::User::operands(), llvm::VPValue::replaceAllUsesWith(), and SI.
1.9.6