LLVM 20.0.0git
|
Merges shuffle masks and emits final shuffle instruction, if required. More...
Public Member Functions | |
ShuffleInstructionBuilder (Type *ScalarTy, IRBuilderBase &Builder, BoUpSLP &R) | |
Value * | adjustExtracts (const TreeEntry *E, MutableArrayRef< int > Mask, ArrayRef< std::optional< TTI::ShuffleKind > > ShuffleKinds, unsigned NumParts, bool &UseVecBaseAsInput) |
Adjusts extractelements after reusing them. | |
std::optional< Value * > | needToDelay (const TreeEntry *E, ArrayRef< SmallVector< const TreeEntry * > > Deps) const |
Checks if the specified entry E needs to be delayed because of its dependency nodes. | |
void | add (const TreeEntry &E1, const TreeEntry &E2, ArrayRef< int > Mask) |
Adds 2 input vectors (in form of tree entries) and the mask for their shuffling. | |
void | add (const TreeEntry &E1, ArrayRef< int > Mask) |
Adds single input vector (in form of tree entry) and the mask for its shuffling. | |
void | add (Value *V1, Value *V2, ArrayRef< int > Mask) |
Adds 2 input vectors and the mask for their shuffling. | |
void | add (Value *V1, ArrayRef< int > Mask, bool=false) |
Adds another one input vector and the mask for the shuffling. | |
void | addOrdered (Value *V1, ArrayRef< unsigned > Order) |
Adds another one input vector and the mask for the shuffling. | |
Value * | gather (ArrayRef< Value * > VL, unsigned MaskVF=0, Value *Root=nullptr) |
Value * | createFreeze (Value *V) |
Value * | finalize (ArrayRef< int > ExtMask, unsigned VF=0, function_ref< void(Value *&, SmallVectorImpl< int > &)> Action={}) |
Finalize emission of the shuffles. | |
~ShuffleInstructionBuilder () | |
Merges shuffle masks and emits final shuffle instruction, if required.
It supports shuffling of 2 input vectors. It implements lazy shuffles emission, when the actual shuffle instruction is generated only if this is actually required. Otherwise, the shuffle instruction emission is delayed till the end of the process, to reduce the number of emitted instructions and further analysis/transformations. The class also will look through the previously emitted shuffle instructions and properly mark indices in mask as undef. For example, given the code
and if need to emit shuffle of s1 and s2 with mask <1, 0, 3, 2>, it will look through s1 and s2 and emit
instead. If 2 operands are of different size, the smallest one will be resized and the mask recalculated properly. For example, given the code
and if need to emit shuffle of s1 and s2 with mask <1, 0, 5, 4>, it will look through s1 and s2 and emit
instead.
Definition at line 12049 of file SLPVectorizer.cpp.
|
inline |
Definition at line 12184 of file SLPVectorizer.cpp.
|
inline |
Definition at line 12518 of file SLPVectorizer.cpp.
References assert(), and llvm::SmallVectorBase< Size_T >::empty().
|
inline |
Adds single input vector (in form of tree entry) and the mask for its shuffling.
Definition at line 12353 of file SLPVectorizer.cpp.
References llvm::any_of(), llvm::Value::getType(), and llvm::Type::isIntOrIntVectorTy().
|
inline |
Adds 2 input vectors (in form of tree entries) and the mask for their shuffling.
Definition at line 12336 of file SLPVectorizer.cpp.
References llvm::any_of(), llvm::Value::getType(), and llvm::Type::isIntOrIntVectorTy().
Referenced by llvm::slpvectorizer::BoUpSLP::vectorizeTree().
|
inline |
Adds another one input vector and the mask for the shuffling.
Definition at line 12396 of file SLPVectorizer.cpp.
References assert(), llvm::SmallVectorImpl< T >::assign(), llvm::SmallVectorTemplateCommon< T, typename >::back(), llvm::SmallVectorTemplateCommon< T, typename >::begin(), llvm::SmallVectorBase< Size_T >::empty(), llvm::SmallVectorTemplateCommon< T, typename >::end(), llvm::find(), llvm::SmallVectorTemplateCommon< T, typename >::front(), llvm::Value::getType(), Idx, llvm::PoisonMaskElem, llvm::SmallVectorTemplateBase< T, bool >::push_back(), and llvm::SmallVectorBase< Size_T >::size().
|
inline |
Adds 2 input vectors and the mask for their shuffling.
Definition at line 12363 of file SLPVectorizer.cpp.
References assert(), llvm::SmallVectorImpl< T >::assign(), llvm::SmallVectorTemplateCommon< T, typename >::back(), llvm::SmallVectorBase< Size_T >::empty(), llvm::SmallVectorTemplateCommon< T, typename >::front(), llvm::Value::getType(), Idx, llvm::PoisonMaskElem, llvm::SmallVectorTemplateBase< T, bool >::push_back(), and llvm::SmallVectorBase< Size_T >::size().
|
inline |
Adds another one input vector and the mask for the shuffling.
Definition at line 12448 of file SLPVectorizer.cpp.
References llvm::inversePermutation().
|
inline |
Adjusts extractelements after reusing them.
Definition at line 12188 of file SLPVectorizer.cpp.
References llvm::any_of(), assert(), llvm::count(), I, Idx, llvm::SmallPtrSetImpl< PtrType >::insert(), and llvm::PoisonMaskElem.
Definition at line 12457 of file SLPVectorizer.cpp.
References llvm::IRBuilderBase::CreateFreeze().
|
inline |
Finalize emission of the shuffles.
Action | the action (if any) to be performed before final applying of the ExtMask mask. |
Definition at line 12462 of file SLPVectorizer.cpp.
Referenced by llvm::slpvectorizer::BoUpSLP::vectorizeTree().
|
inline |
Definition at line 12453 of file SLPVectorizer.cpp.
|
inline |
Checks if the specified entry E
needs to be delayed because of its dependency nodes.
Definition at line 12318 of file SLPVectorizer.cpp.
References llvm::all_of(), llvm::IRBuilderBase::CreateAlignedLoad(), llvm::PoisonValue::get(), llvm::Type::getContext(), llvm::PointerType::getUnqual(), and getWidenedType().