LLVM 20.0.0git
|
Drive the analysis of interleaved memory accesses in the loop. More...
#include "llvm/Analysis/VectorUtils.h"
Public Member Functions | |
InterleavedAccessInfo (PredicatedScalarEvolution &PSE, Loop *L, DominatorTree *DT, LoopInfo *LI, const LoopAccessInfo *LAI) | |
~InterleavedAccessInfo () | |
void | analyzeInterleaving (bool EnableMaskedInterleavedGroup) |
Analyze the interleaved accesses and collect them in interleave groups. | |
bool | invalidateGroups () |
Invalidate groups, e.g., in case all blocks in loop will be predicated contrary to original assumption. | |
bool | isInterleaved (Instruction *Instr) const |
Check if Instr belongs to any interleave group. | |
InterleaveGroup< Instruction > * | getInterleaveGroup (const Instruction *Instr) const |
Get the interleave group that Instr belongs to. | |
iterator_range< SmallPtrSetIterator< llvm::InterleaveGroup< Instruction > * > > | getInterleaveGroups () |
bool | requiresScalarEpilogue () const |
Returns true if an interleaved group that may access memory out-of-bounds requires a scalar epilogue iteration for correctness. | |
void | invalidateGroupsRequiringScalarEpilogue () |
Invalidate groups that require a scalar epilogue (due to gaps). | |
bool | hasGroups () const |
Returns true if we have any interleave groups. | |
Drive the analysis of interleaved memory accesses in the loop.
Use this class to analyze interleaved accesses only when we can vectorize a loop. Otherwise it's meaningless to do analysis as the vectorization on interleaved accesses is unsafe.
The analysis collects interleave groups and records the relationships between the member and the group in a map.
Definition at line 610 of file VectorUtils.h.
|
inline |
Definition at line 612 of file VectorUtils.h.
|
inline |
Definition at line 617 of file VectorUtils.h.
References invalidateGroups().
void InterleavedAccessInfo::analyzeInterleaving | ( | bool | EnableMaskedInterleavedGroup | ) |
Analyze the interleaved accesses and collect them in interleave groups.
Substitute symbolic strides using Strides
. Consider also predicated loads/stores in the analysis if EnableMaskedInterleavedGroup
is true.
Definition at line 1202 of file VectorUtils.cpp.
References A, assert(), B, llvm::SetVector< T, Vector, Set, N >::contains(), llvm::SmallPtrSetImpl< PtrType >::contains(), llvm::dbgs(), llvm::MapVector< KeyT, ValueT, MapType, VectorType >::empty(), llvm::MapVector< KeyT, ValueT, MapType, VectorType >::find(), llvm::SCEVConstant::getAPInt(), llvm::InterleaveGroup< InstTy >::getFactor(), llvm::InterleaveGroup< InstTy >::getIndex(), getInterleaveGroup(), llvm::getLoadStoreAddressSpace(), llvm::getLoadStorePointerOperand(), llvm::getLoadStoreType(), llvm::InterleaveGroup< InstTy >::getMember(), llvm::ScalarEvolution::getMinusSCEV(), llvm::InterleaveGroup< InstTy >::getNumMembers(), llvm::getPtrStride(), llvm::PredicatedScalarEvolution::getSE(), llvm::APInt::getSExtValue(), llvm::LoopAccessInfo::getSymbolicStrides(), llvm::SetVector< T, Vector, Set, N >::insert(), llvm::SmallPtrSetImpl< PtrType >::insert(), llvm::InterleaveGroup< InstTy >::insertMember(), isInterleaved(), llvm::InterleaveGroup< InstTy >::isReverse(), LLVM_DEBUG, llvm::MapVector< KeyT, ValueT, MapType, VectorType >::rbegin(), llvm::SetVector< T, Vector, Set, N >::remove(), llvm::MapVector< KeyT, ValueT, MapType, VectorType >::rend(), and llvm::InterleaveGroup< InstTy >::setInsertPos().
Referenced by llvm::LoopVectorizePass::processLoop().
|
inline |
Get the interleave group that Instr
belongs to.
Definition at line 655 of file VectorUtils.h.
Referenced by analyzeInterleaving(), and llvm::LoopVectorizationCostModel::getInterleavedAccessGroup().
|
inline |
Definition at line 660 of file VectorUtils.h.
References llvm::make_range().
|
inline |
Returns true if we have any interleave groups.
Definition at line 674 of file VectorUtils.h.
Referenced by llvm::AArch64TTIImpl::preferPredicateOverEpilogue().
|
inline |
Invalidate groups, e.g., in case all blocks in loop will be predicated contrary to original assumption.
Although we currently prevent group formation for predicated accesses, we may be able to relax this limitation in the future once we handle more complicated blocks. Returns true if any groups were invalidated.
Definition at line 630 of file VectorUtils.h.
Referenced by llvm::LoopVectorizationPlanner::plan(), and ~InterleavedAccessInfo().
void InterleavedAccessInfo::invalidateGroupsRequiringScalarEpilogue | ( | ) |
Invalidate groups that require a scalar epilogue (due to gaps).
This can happen when optimizing for size forbids a scalar epilogue, and the gap cannot be filtered by masking the load/store.
Definition at line 1506 of file VectorUtils.cpp.
References assert(), llvm::dbgs(), LLVM_DEBUG, and requiresScalarEpilogue().
Referenced by llvm::LoopVectorizationCostModel::computeMaxVF().
|
inline |
Check if Instr
belongs to any interleave group.
Definition at line 647 of file VectorUtils.h.
Referenced by analyzeInterleaving(), and llvm::LoopVectorizationCostModel::isAccessInterleaved().
|
inline |
Returns true if an interleaved group that may access memory out-of-bounds requires a scalar epilogue iteration for correctness.
Definition at line 666 of file VectorUtils.h.
Referenced by invalidateGroupsRequiringScalarEpilogue(), and llvm::LoopVectorizationCostModel::requiresScalarEpilogue().