Drive the analysis of interleaved memory accesses in the loop. More...

#include "llvm/Analysis/VectorUtils.h"

Public Member Functions
LLVM_ABI	InterleavedAccessInfo (PredicatedScalarEvolution &PSE, Loop L, DominatorTree DT, LoopInfo LI, const LoopAccessInfo LAI, bool OptForSize)
	~InterleavedAccessInfo ()
LLVM_ABI void	analyzeInterleaving (bool EnableMaskedInterleavedGroup)
	Analyze the interleaved accesses and collect them in interleave groups.
bool	invalidateGroups ()
	Invalidate groups, e.g., in case all blocks in loop will be predicated contrary to original assumption.
bool	isInterleaved (Instruction *Instr) const
	Check if `Instr` belongs to any interleave group.
InterleaveGroup< Instruction > *	getInterleaveGroup (const Instruction *Instr) const
	Get the interleave group that `Instr` belongs to.
iterator_range< SmallPtrSetIterator< llvm::InterleaveGroup< Instruction > * > >	getInterleaveGroups ()
bool	requiresScalarEpilogue () const
	Returns true if an interleaved group that may access memory out-of-bounds requires a scalar epilogue iteration for correctness.
LLVM_ABI void	invalidateGroupsRequiringScalarEpilogue ()
	Invalidate groups that require a scalar epilogue (due to gaps).
bool	hasGroups () const
	Returns true if we have any interleave groups.

Detailed Description

Drive the analysis of interleaved memory accesses in the loop.

Use this class to analyze interleaved accesses only when we can vectorize a loop. Otherwise it's meaningless to do analysis as the vectorization on interleaved accesses is unsafe.

The analysis collects interleave groups and records the relationships between the member and the group in a map.

Definition at line 690 of file VectorUtils.h.

Constructor & Destructor Documentation

◆ InterleavedAccessInfo()

InterleavedAccessInfo::InterleavedAccessInfo	(	PredicatedScalarEvolution &	PSE,
		Loop *	L,
		DominatorTree *	DT,
		LoopInfo *	LI,
		const LoopAccessInfo *	LAI,
		bool	OptForSize )

Definition at line 1329 of file VectorUtils.cpp.

References llvm::getAnalysisOptions().

◆ ~InterleavedAccessInfo()

llvm::InterleavedAccessInfo::~InterleavedAccessInfo ( )

inline

Definition at line 696 of file VectorUtils.h.

References invalidateGroups().

Member Function Documentation

◆ analyzeInterleaving()

void InterleavedAccessInfo::analyzeInterleaving ( bool EnableMaskedInterleavedGroup )

Analyze the interleaved accesses and collect them in interleave groups.

Substitute symbolic strides using Strides. Consider also predicated loads/stores in the analysis if EnableMaskedInterleavedGroup is true.

Definition at line 1422 of file VectorUtils.cpp.

References A(), assert(), B(), llvm::SetVector< T, Vector, Set, N >::contains(), llvm::SmallPtrSetImpl< PtrType >::contains(), llvm::dbgs(), llvm::dyn_cast(), llvm::MapVector< KeyT, ValueT, MapType, VectorType, N >::empty(), llvm::SetVector< T, Vector, Set, N >::empty(), llvm::MapVector< KeyT, ValueT, MapType, VectorType, N >::find(), llvm::SCEVConstant::getAPInt(), llvm::InterleaveGroup< InstTy >::getFactor(), llvm::InterleaveGroup< InstTy >::getIndex(), getInterleaveGroup(), llvm::getLoadStoreAddressSpace(), llvm::getLoadStorePointerOperand(), llvm::getLoadStoreType(), llvm::InterleaveGroup< InstTy >::getMember(), llvm::getPtrStride(), llvm::APInt::getSExtValue(), llvm::SetVector< T, Vector, Set, N >::insert(), llvm::SmallPtrSetImpl< PtrType >::insert(), llvm::InterleaveGroup< InstTy >::insertMember(), llvm::InterleaveGroup< InstTy >::isFull(), isInterleaved(), llvm::InterleaveGroup< InstTy >::isReverse(), LLVM_DEBUG, llvm::MapVector< KeyT, ValueT, MapType, VectorType, N >::rbegin(), llvm::SetVector< T, Vector, Set, N >::remove(), llvm::MapVector< KeyT, ValueT, MapType, VectorType, N >::rend(), and llvm::InterleaveGroup< InstTy >::setInsertPos().

Referenced by llvm::LoopVectorizePass::processLoop().

◆ getInterleaveGroup()

InterleaveGroup< Instruction > * llvm::InterleavedAccessInfo::getInterleaveGroup ( const Instruction * Instr ) const

inline

Get the interleave group that Instr belongs to.

Returns: nullptr if doesn't have such group.

Definition at line 734 of file VectorUtils.h.

Referenced by analyzeInterleaving().

◆ getInterleaveGroups()

iterator_range< SmallPtrSetIterator< llvm::InterleaveGroup< Instruction > * > > llvm::InterleavedAccessInfo::getInterleaveGroups ( )

inline

Definition at line 739 of file VectorUtils.h.

References llvm::make_range().

◆ hasGroups()

bool llvm::InterleavedAccessInfo::hasGroups ( ) const

inline

Returns true if we have any interleave groups.

Definition at line 753 of file VectorUtils.h.

Referenced by getEpilogueTailLowering(), and llvm::AArch64TTIImpl::preferTailFoldingOverEpilogue().

◆ invalidateGroups()

bool llvm::InterleavedAccessInfo::invalidateGroups ( )

inline

Invalidate groups, e.g., in case all blocks in loop will be predicated contrary to original assumption.

Although we currently prevent group formation for predicated accesses, we may be able to relax this limitation in the future once we handle more complicated blocks. Returns true if any groups were invalidated.

Definition at line 709 of file VectorUtils.h.

References assert().

Referenced by ~InterleavedAccessInfo().

◆ invalidateGroupsRequiringScalarEpilogue()

void InterleavedAccessInfo::invalidateGroupsRequiringScalarEpilogue ( )

Invalidate groups that require a scalar epilogue (due to gaps).

This can happen when optimizing for size forbids a scalar epilogue, and the gap cannot be filtered by masking the load/store.

Definition at line 1732 of file VectorUtils.cpp.

References assert(), llvm::dbgs(), LLVM_DEBUG, llvm::SmallPtrSetImpl< PtrType >::remove_if(), and requiresScalarEpilogue().

◆ isInterleaved()

bool llvm::InterleavedAccessInfo::isInterleaved ( Instruction * Instr ) const

inline

Check if Instr belongs to any interleave group.

Definition at line 726 of file VectorUtils.h.

Referenced by analyzeInterleaving().

◆ requiresScalarEpilogue()

bool llvm::InterleavedAccessInfo::requiresScalarEpilogue ( ) const

inline

Returns true if an interleaved group that may access memory out-of-bounds requires a scalar epilogue iteration for correctness.

Definition at line 745 of file VectorUtils.h.

Referenced by invalidateGroupsRequiringScalarEpilogue(), and llvm::LoopVectorizePass::processLoop().

The documentation for this class was generated from the following files:

include/llvm/Analysis/VectorUtils.h
lib/Analysis/VectorUtils.cpp

Public Member Functions

Detailed Description

Constructor & Destructor Documentation

◆ InterleavedAccessInfo()

◆ ~InterleavedAccessInfo()

Member Function Documentation

◆ analyzeInterleaving()

◆ getInterleaveGroup()

◆ getInterleaveGroups()

◆ hasGroups()

◆ invalidateGroups()

◆ invalidateGroupsRequiringScalarEpilogue()

◆ isInterleaved()

◆ requiresScalarEpilogue()