LLVM  7.0.0svn
Public Types | Public Member Functions | List of all members
llvm::LoopVectorizationLegality Class Reference

LoopVectorizationLegality checks if it is legal to vectorize a loop, and to what vectorization factor. More...

Public Types

using ReductionList = DenseMap< PHINode *, RecurrenceDescriptor >
 ReductionList contains the reduction descriptors for all of the reductions that were found in the loop. More...
 
using InductionList = MapVector< PHINode *, InductionDescriptor >
 InductionList saves induction variables and maps them to the induction descriptor. More...
 
using RecurrenceSet = SmallPtrSet< const PHINode *, 8 >
 RecurrenceSet contains the phi nodes that are recurrences other than inductions and reductions. More...
 

Public Member Functions

 LoopVectorizationLegality (Loop *L, PredicatedScalarEvolution &PSE, DominatorTree *DT, TargetLibraryInfo *TLI, AliasAnalysis *AA, Function *F, const TargetTransformInfo *TTI, std::function< const LoopAccessInfo &(Loop &)> *GetLAA, LoopInfo *LI, OptimizationRemarkEmitter *ORE, LoopVectorizationRequirements *R, LoopVectorizeHints *H, DemandedBits *DB, AssumptionCache *AC)
 
bool canVectorize ()
 Returns true if it is legal to vectorize this loop. More...
 
PHINodegetPrimaryInduction ()
 Returns the primary induction variable. More...
 
ReductionListgetReductionVars ()
 Returns the reduction variables found in the loop. More...
 
InductionListgetInductionVars ()
 Returns the induction variables found in the loop. More...
 
RecurrenceSetgetFirstOrderRecurrences ()
 Return the first-order recurrences found in the loop. More...
 
DenseMap< Instruction *, Instruction * > & getSinkAfter ()
 Return the set of instructions to sink to handle first-order recurrences. More...
 
TypegetWidestInductionType ()
 Returns the widest induction type. More...
 
bool isInductionPhi (const Value *V)
 Returns True if V is a Phi node of an induction variable in this loop. More...
 
bool isCastedInductionVariable (const Value *V)
 Returns True if V is a cast that is part of an induction def-use chain, and had been proven to be redundant under a runtime guard (in other words, the cast has the same SCEV expression as the induction phi). More...
 
bool isInductionVariable (const Value *V)
 Returns True if V can be considered as an induction variable in this loop. More...
 
bool isReductionVariable (PHINode *PN)
 Returns True if PN is a reduction variable in this loop. More...
 
bool isFirstOrderRecurrence (const PHINode *Phi)
 Returns True if Phi is a first-order recurrence in this loop. More...
 
bool blockNeedsPredication (BasicBlock *BB)
 Return true if the block BB needs to be predicated in order for the loop to be vectorized. More...
 
int isConsecutivePtr (Value *Ptr)
 Check if this pointer is consecutive when vectorizing. More...
 
bool isUniform (Value *V)
 Returns true if the value V is uniform within the loop. More...
 
const RuntimePointerCheckinggetRuntimePointerChecking () const
 Returns the information that we collected about runtime memory check. More...
 
const LoopAccessInfogetLAI () const
 
bool isAccessInterleaved (Instruction *Instr)
 Check if Instr belongs to any interleaved access group. More...
 
const InterleaveGroupgetInterleavedAccessGroup (Instruction *Instr)
 Get the interleaved access group that Instr belongs to. More...
 
bool requiresScalarEpilogue () const
 Returns true if an interleaved group requires a scalar iteration to handle accesses with gaps. More...
 
unsigned getMaxSafeDepDistBytes ()
 
uint64_t getMaxSafeRegisterWidth () const
 
bool hasStride (Value *V)
 
bool isLegalMaskedStore (Type *DataType, Value *Ptr)
 Returns true if the target machine supports masked store operation for the given DataType and kind of access to Ptr. More...
 
bool isLegalMaskedLoad (Type *DataType, Value *Ptr)
 Returns true if the target machine supports masked load operation for the given DataType and kind of access to Ptr. More...
 
bool isLegalMaskedScatter (Type *DataType)
 Returns true if the target machine supports masked scatter operation for the given DataType. More...
 
bool isLegalMaskedGather (Type *DataType)
 Returns true if the target machine supports masked gather operation for the given DataType. More...
 
bool isLegalGatherOrScatter (Value *V)
 Returns true if the target machine can represent V as a masked gather or scatter operation. More...
 
bool isMaskRequired (const Instruction *I)
 Returns true if vector representation of the instruction I requires mask. More...
 
unsigned getNumStores () const
 
unsigned getNumLoads () const
 
unsigned getNumPredStores () const
 
bool isScalarWithPredication (Instruction *I)
 Returns true if I is an instruction that will be scalarized with predication. More...
 
bool memoryInstructionCanBeWidened (Instruction *I, unsigned VF=1)
 Returns true if I is a memory instruction with consecutive memory access that can be widened. More...
 
bool hasFunNoNaNAttr () const
 

Detailed Description

LoopVectorizationLegality checks if it is legal to vectorize a loop, and to what vectorization factor.

This class does not look at the profitability of vectorization, only the legality. This class has two main kinds of checks:

Definition at line 1534 of file LoopVectorize.cpp.

Member Typedef Documentation

◆ InductionList

InductionList saves induction variables and maps them to the induction descriptor.

Definition at line 1553 of file LoopVectorize.cpp.

◆ RecurrenceSet

RecurrenceSet contains the phi nodes that are recurrences other than inductions and reductions.

Definition at line 1557 of file LoopVectorize.cpp.

◆ ReductionList

ReductionList contains the reduction descriptors for all of the reductions that were found in the loop.

Definition at line 1549 of file LoopVectorize.cpp.

Constructor & Destructor Documentation

◆ LoopVectorizationLegality()

llvm::LoopVectorizationLegality::LoopVectorizationLegality ( Loop L,
PredicatedScalarEvolution PSE,
DominatorTree DT,
TargetLibraryInfo TLI,
AliasAnalysis AA,
Function F,
const TargetTransformInfo TTI,
std::function< const LoopAccessInfo &(Loop &)> *  GetLAA,
LoopInfo LI,
OptimizationRemarkEmitter ORE,
LoopVectorizationRequirements *  R,
LoopVectorizeHints *  H,
DemandedBits DB,
AssumptionCache AC 
)
inline

Definition at line 1536 of file LoopVectorize.cpp.

Member Function Documentation

◆ blockNeedsPredication()

bool LoopVectorizationLegality::blockNeedsPredication ( BasicBlock BB)

Return true if the block BB needs to be predicated in order for the loop to be vectorized.

Definition at line 5685 of file LoopVectorize.cpp.

References llvm::AMDGPU::HSAMD::Kernel::Arg::Key::Align, B, llvm::LoopBlocksDFS::beginRPO(), llvm::LoopAccessInfo::blockNeedsPredication(), C, llvm::SmallPtrSetImpl< PtrType >::count(), llvm::dbgs(), DEBUG, DFS(), llvm::InnerLoopVectorizer::DT, llvm::dyn_cast(), E, llvm::MapVector< KeyT, ValueT, MapType, VectorType >::empty(), llvm::LoopBlocksDFS::endRPO(), llvm::SCEVConstant::getAPInt(), llvm::PointerType::getElementType(), llvm::InterleaveGroup::getIndex(), getMemInstAddressSpace(), getMemInstAlignment(), llvm::ScalarEvolution::getMinusSCEV(), llvm::Instruction::getParent(), getPointerOperand(), llvm::getPtrStride(), llvm::PredicatedScalarEvolution::getSE(), llvm::APInt::getSExtValue(), llvm::Value::getType(), I, llvm::SetVector< T, SmallVector< T, N >, SmallDenseSet< T, N > >::insert(), llvm::InterleaveGroup::insertMember(), llvm::HexagonMCInstrInfo::isPredicated(), llvm::InnerLoopVectorizer::LI, llvm::make_range(), llvm::Instruction::mayReadFromMemory(), llvm::Instruction::mayWriteToMemory(), NumberOfStoresToPredicate, llvm::LoopBlocksDFS::perform(), llvm::InnerLoopVectorizer::PSE, llvm::MapVector< KeyT, ValueT, MapType, VectorType >::rbegin(), llvm::SetVector< T, SmallVector< T, N >, SmallDenseSet< T, N > >::remove(), llvm::MapVector< KeyT, ValueT, MapType, VectorType >::rend(), llvm::replaceSymbolicStrideSCEV(), llvm::InterleaveGroup::setInsertPos(), SI, and llvm::AMDGPU::HSAMD::Kernel::Arg::Key::Size.

Referenced by llvm::LoopVectorizationCostModel::collectInstsToScalarize().

◆ canVectorize()

bool LoopVectorizationLegality::canVectorize ( )

◆ getFirstOrderRecurrences()

RecurrenceSet* llvm::LoopVectorizationLegality::getFirstOrderRecurrences ( )
inline

Return the first-order recurrences found in the loop.

Definition at line 1574 of file LoopVectorize.cpp.

◆ getInductionVars()

InductionList* llvm::LoopVectorizationLegality::getInductionVars ( )
inline

◆ getInterleavedAccessGroup()

const InterleaveGroup* llvm::LoopVectorizationLegality::getInterleavedAccessGroup ( Instruction Instr)
inline

◆ getLAI()

const LoopAccessInfo* llvm::LoopVectorizationLegality::getLAI ( ) const
inline

Definition at line 1625 of file LoopVectorize.cpp.

Referenced by llvm::InnerLoopVectorizer::emitMemRuntimeChecks().

◆ getMaxSafeDepDistBytes()

unsigned llvm::LoopVectorizationLegality::getMaxSafeDepDistBytes ( )
inline

◆ getMaxSafeRegisterWidth()

uint64_t llvm::LoopVectorizationLegality::getMaxSafeRegisterWidth ( ) const
inline

Definition at line 1645 of file LoopVectorize.cpp.

Referenced by llvm::LoopVectorizationCostModel::computeMaxVF().

◆ getNumLoads()

unsigned llvm::LoopVectorizationLegality::getNumLoads ( ) const
inline

◆ getNumPredStores()

unsigned llvm::LoopVectorizationLegality::getNumPredStores ( ) const
inline

Definition at line 1693 of file LoopVectorize.cpp.

References I, and llvm::InnerLoopVectorizer::VF.

Referenced by llvm::LoopVectorizationCostModel::computeMaxVF().

◆ getNumStores()

unsigned llvm::LoopVectorizationLegality::getNumStores ( ) const
inline

◆ getPrimaryInduction()

PHINode* llvm::LoopVectorizationLegality::getPrimaryInduction ( )
inline

Returns the primary induction variable.

Definition at line 1565 of file LoopVectorize.cpp.

Referenced by llvm::InnerLoopVectorizer::createVectorizedLoopSkeleton(), and llvm::LoopVectorizationCostModel::isOptimizableIVTruncate().

◆ getReductionVars()

ReductionList* llvm::LoopVectorizationLegality::getReductionVars ( )
inline

◆ getRuntimePointerChecking()

const RuntimePointerChecking* llvm::LoopVectorizationLegality::getRuntimePointerChecking ( ) const
inline

Returns the information that we collected about runtime memory check.

Definition at line 1621 of file LoopVectorize.cpp.

Referenced by llvm::LoopVectorizationCostModel::computeMaxVF(), and llvm::LoopVectorizationCostModel::selectInterleaveCount().

◆ getSinkAfter()

DenseMap<Instruction *, Instruction *>& llvm::LoopVectorizationLegality::getSinkAfter ( )
inline

Return the set of instructions to sink to handle first-order recurrences.

Definition at line 1577 of file LoopVectorize.cpp.

Referenced by llvm::LoopVectorizationPlanner::buildVPlans().

◆ getWidestInductionType()

Type* llvm::LoopVectorizationLegality::getWidestInductionType ( )
inline

Returns the widest induction type.

Definition at line 1580 of file LoopVectorize.cpp.

Referenced by llvm::InnerLoopVectorizer::createVectorizedLoopSkeleton(), and llvm::InnerLoopVectorizer::getOrCreateTripCount().

◆ hasFunNoNaNAttr()

bool llvm::LoopVectorizationLegality::hasFunNoNaNAttr ( ) const
inline

◆ hasStride()

bool llvm::LoopVectorizationLegality::hasStride ( Value V)
inline

◆ isAccessInterleaved()

bool llvm::LoopVectorizationLegality::isAccessInterleaved ( Instruction Instr)
inline

Check if Instr belongs to any interleaved access group.

Definition at line 1628 of file LoopVectorize.cpp.

Referenced by llvm::LoopVectorizationCostModel::getSmallestAndWidestTypes(), and llvm::LoopVectorizationCostModel::setCostBasedWideningDecision().

◆ isCastedInductionVariable()

bool LoopVectorizationLegality::isCastedInductionVariable ( const Value V)

Returns True if V is a cast that is part of an induction def-use chain, and had been proven to be redundant under a runtime guard (in other words, the cast has the same SCEV expression as the induction phi).

Definition at line 5672 of file LoopVectorize.cpp.

References llvm::dyn_cast().

◆ isConsecutivePtr()

int LoopVectorizationLegality::isConsecutivePtr ( Value Ptr)

Check if this pointer is consecutive when vectorizing.

This happens when the last index of the GEP is the induction variable, or that the pointer itself is an induction variable. This check allows us to vectorize A[idx] into a wide load/store. Returns: 0 - Stride is unknown or non-consecutive. 1 - Address is consecutive. -1 - Address is consecutive, and decreasing. NOTE: This method must only be used before modifying the original scalar loop. Do not use after invoking 'createVectorizedLoopSkeleton' (PR34965).

Definition at line 2695 of file LoopVectorize.cpp.

References llvm::getPtrStride(), and llvm::InnerLoopVectorizer::PSE.

Referenced by llvm::createLoopVectorizePass(), isStrideMul(), and llvm::LoopVectorizationCostModel::setCostBasedWideningDecision().

◆ isFirstOrderRecurrence()

bool LoopVectorizationLegality::isFirstOrderRecurrence ( const PHINode Phi)

◆ isInductionPhi()

bool LoopVectorizationLegality::isInductionPhi ( const Value V)

Returns True if V is a Phi node of an induction variable in this loop.

Definition at line 5663 of file LoopVectorize.cpp.

Referenced by llvm::LoopVectorizationCostModel::isOptimizableIVTruncate().

◆ isInductionVariable()

bool LoopVectorizationLegality::isInductionVariable ( const Value V)

Returns True if V can be considered as an induction variable in this loop.

V can be the induction phi, or some redundant cast in the def-use chain of the inducion phi.

Definition at line 5677 of file LoopVectorize.cpp.

Referenced by getAddressAccessSCEV().

◆ isLegalGatherOrScatter()

bool llvm::LoopVectorizationLegality::isLegalGatherOrScatter ( Value V)
inline

Returns true if the target machine can represent V as a masked gather or scatter operation.

Definition at line 1677 of file LoopVectorize.cpp.

References llvm::dyn_cast(), getPointerOperand(), llvm::Value::getType(), llvm::InnerLoopVectorizer::LI, and SI.

Referenced by llvm::LoopVectorizationCostModel::getSmallestAndWidestTypes(), and llvm::LoopVectorizationCostModel::setCostBasedWideningDecision().

◆ isLegalMaskedGather()

bool llvm::LoopVectorizationLegality::isLegalMaskedGather ( Type DataType)
inline

Returns true if the target machine supports masked gather operation for the given DataType.

Definition at line 1671 of file LoopVectorize.cpp.

References llvm::TargetTransformInfo::isLegalMaskedGather().

◆ isLegalMaskedLoad()

bool llvm::LoopVectorizationLegality::isLegalMaskedLoad ( Type DataType,
Value Ptr 
)
inline

Returns true if the target machine supports masked load operation for the given DataType and kind of access to Ptr.

Definition at line 1659 of file LoopVectorize.cpp.

References llvm::TargetTransformInfo::isLegalMaskedLoad().

◆ isLegalMaskedScatter()

bool llvm::LoopVectorizationLegality::isLegalMaskedScatter ( Type DataType)
inline

Returns true if the target machine supports masked scatter operation for the given DataType.

Definition at line 1665 of file LoopVectorize.cpp.

References llvm::TargetTransformInfo::isLegalMaskedScatter().

◆ isLegalMaskedStore()

bool llvm::LoopVectorizationLegality::isLegalMaskedStore ( Type DataType,
Value Ptr 
)
inline

Returns true if the target machine supports masked store operation for the given DataType and kind of access to Ptr.

Definition at line 1653 of file LoopVectorize.cpp.

References llvm::TargetTransformInfo::isLegalMaskedStore().

◆ isMaskRequired()

bool llvm::LoopVectorizationLegality::isMaskRequired ( const Instruction I)
inline

Returns true if vector representation of the instruction I requires mask.

Definition at line 1689 of file LoopVectorize.cpp.

Referenced by llvm::LoopVectorizationPlanner::buildVPlans(), and isStrideMul().

◆ isReductionVariable()

bool llvm::LoopVectorizationLegality::isReductionVariable ( PHINode PN)
inline

◆ isScalarWithPredication()

bool LoopVectorizationLegality::isScalarWithPredication ( Instruction I)

Returns true if I is an instruction that will be scalarized with predication.

Such instructions include conditional stores and instructions that may divide by zero.

Definition at line 5424 of file LoopVectorize.cpp.

References llvm::Instruction::getOpcode(), llvm::Instruction::getParent(), mayDivideByZero(), and llvm::SPII::Store.

Referenced by llvm::LoopVectorizationPlanner::buildVPlans(), llvm::LoopVectorizationCostModel::collectInstsToScalarize(), isStrideMul(), and llvm::LoopVectorizationCostModel::setCostBasedWideningDecision().

◆ isUniform()

bool LoopVectorizationLegality::isUniform ( Value V)

Returns true if the value V is uniform within the loop.

Definition at line 2705 of file LoopVectorize.cpp.

Referenced by llvm::LoopVectorizationCostModel::setCostBasedWideningDecision(), and llvm::InnerLoopVectorizer::vectorizeMemoryInstruction().

◆ memoryInstructionCanBeWidened()

bool LoopVectorizationLegality::memoryInstructionCanBeWidened ( Instruction I,
unsigned  VF = 1 
)

◆ requiresScalarEpilogue()

bool llvm::LoopVectorizationLegality::requiresScalarEpilogue ( ) const
inline

Returns true if an interleaved group requires a scalar iteration to handle accesses with gaps.

Definition at line 1639 of file LoopVectorize.cpp.

Referenced by llvm::InnerLoopVectorizer::emitMinimumIterationCountCheck(), and llvm::InnerLoopVectorizer::getOrCreateVectorTripCount().


The documentation for this class was generated from the following file: