Merges shuffle masks and emits final shuffle instruction, if required. More...
Public Member Functions | |
| ShuffleCostEstimator (Type *ScalarTy, TargetTransformInfo &TTI, ArrayRef< Value * > VectorizedVals, BoUpSLP &R, SmallPtrSetImpl< Value * > &CheckedExtracts) | |
| Value * | adjustExtracts (const TreeEntry *E, MutableArrayRef< int > Mask, ArrayRef< std::optional< TTI::ShuffleKind > > ShuffleKinds, unsigned NumParts, bool &UseVecBaseAsInput) |
| std::optional< InstructionCost > | needToDelay (const TreeEntry *, ArrayRef< SmallVector< const TreeEntry * > >) const |
Checks if the specified entry E needs to be delayed because of its dependency nodes. | |
| void | resetForSameNode () |
| Reset the builder to handle perfect diamond match. | |
| void | add (const TreeEntry &E1, const TreeEntry &E2, ArrayRef< int > Mask) |
| void | add (const TreeEntry &E1, ArrayRef< int > Mask) |
| void | add (Value *V1, Value *V2, ArrayRef< int > Mask) |
| Adds 2 input vectors and the mask for their shuffling. | |
| void | add (Value *V1, ArrayRef< int > Mask, bool ForExtracts=false, TTI::VectorInstrContext ContextHint=TTI::VectorInstrContext::None) |
| Adds another one input vector and the mask for the shuffling. | |
| Value * | gather (ArrayRef< Value * > VL, unsigned MaskVF=0, Value *Root=nullptr, TTI::VectorInstrContext ContextHint=TTI::VectorInstrContext::None) |
| InstructionCost | createExtractedFieldsVector (Value *Src, unsigned FieldWidth, ArrayRef< int > Mask, const TreeEntry &E) |
Estimates the cost of the gather of zero-extended sub-fields of width FieldWidth of the same wider integer scalar Src, permuted by Mask, as a bitcast to the field vector plus a permutation and an extension to the lane type. | |
| InstructionCost | createFreeze (InstructionCost Cost) |
| InstructionCost | finalize (ArrayRef< int > ExtMask, ArrayRef< std::pair< const TreeEntry *, unsigned > > SubVectors, ArrayRef< int > SubVectorsMask, unsigned VF=0, function_ref< void(Value *&, SmallVectorImpl< int > &, function_ref< Value *(Value *, Value *, ArrayRef< int >)>)> Action={}) |
| Finalize emission of the shuffles. | |
| ~ShuffleCostEstimator () | |
Additional Inherited Members | |
| Protected Member Functions inherited from llvm::slpvectorizer::BaseShuffleAnalysis | |
| BaseShuffleAnalysis (Type *ScalarTy) | |
| unsigned | getVF (Value *V) const |
| V is expected to be a vectorized value. | |
| Static Protected Member Functions inherited from llvm::slpvectorizer::BaseShuffleAnalysis | |
| static bool | isIdentityMask (ArrayRef< int > Mask, const FixedVectorType *VecTy, bool IsStrict) |
| Checks if the mask is an identity mask. | |
| static void | combineMasks (unsigned LocalVF, SmallVectorImpl< int > &Mask, ArrayRef< int > ExtMask) |
| Tries to combine 2 different masks into single one. | |
| static bool | peekThroughShuffles (Value *&V, SmallVectorImpl< int > &Mask, bool SinglePermute) |
| Looks through shuffles trying to reduce final number of shuffles in the code. | |
| template<typename T, typename ShuffleBuilderTy, typename... Args> | |
| static T | createShuffle (Value *V1, Value *V2, ArrayRef< int > Mask, ShuffleBuilderTy &Builder, Type *ScalarTy, bool ReVec, Args... Arguments) |
| Smart shuffle instruction emission, walks through shuffles trees and tries to find the best matching vector for the actual shuffle instruction. | |
| static void | transformMaskAfterShuffle (MutableArrayRef< int > CommonMask, ArrayRef< int > Mask) |
Transforms mask CommonMask per given Mask to make proper set after shuffle emission. | |
| Protected Attributes inherited from llvm::slpvectorizer::BaseShuffleAnalysis | |
| Type * | ScalarTy = nullptr |
Detailed Description
Merges shuffle masks and emits final shuffle instruction, if required.
It supports shuffling of 2 input vectors. It implements lazy shuffles emission, when the actual shuffle instruction is generated only if this is actually required. Otherwise, the shuffle instruction emission is delayed till the end of the process, to reduce the number of emitted instructions and further analysis/transformations.
Definition at line 14874 of file SLPVectorizer.cpp.
Constructor & Destructor Documentation
◆ ShuffleCostEstimator()
|
inline |
Definition at line 15461 of file SLPVectorizer.cpp.
References llvm::slpvectorizer::BaseShuffleAnalysis::BaseShuffleAnalysis(), slpvectorizer::BoUpSLP::BoUpSLP(), slpvectorizer::BoUpSLP::getCostKind(), and llvm::slpvectorizer::BaseShuffleAnalysis::ScalarTy.
◆ ~ShuffleCostEstimator()
|
inline |
Definition at line 15922 of file SLPVectorizer.cpp.
References assert().
Member Function Documentation
◆ add() [1/4]
|
inline |
Definition at line 15630 of file SLPVectorizer.cpp.
References assert(), llvm::find_if(), llvm::slpvectorizer::getPartNumElems(), llvm::slpvectorizer::getWidenedType(), llvm::not_equal_to(), llvm::PoisonMaskElem, and llvm::slpvectorizer::BaseShuffleAnalysis::ScalarTy.
◆ add() [2/4]
|
inline |
Definition at line 15606 of file SLPVectorizer.cpp.
References add(), llvm::all_of(), assert(), llvm::find_if(), llvm::slpvectorizer::getPartNumElems(), llvm::slpvectorizer::getWidenedType(), llvm::not_equal_to(), llvm::PoisonMaskElem, and llvm::slpvectorizer::BaseShuffleAnalysis::ScalarTy.
Referenced by add().
◆ add() [3/4]
|
inline |
Adds another one input vector and the mask for the shuffling.
Definition at line 15666 of file SLPVectorizer.cpp.
References llvm::all_of(), assert(), llvm::cast(), llvm::enumerate(), llvm::slpvectorizer::BaseShuffleAnalysis::getVF(), llvm::isa(), llvm::TargetTransformInfo::None, P, llvm::PoisonMaskElem, llvm::slpvectorizer::BaseShuffleAnalysis::transformMaskAfterShuffle(), and llvm::V1.
◆ add() [4/4]
|
inline |
Adds 2 input vectors and the mask for their shuffling.
Definition at line 15648 of file SLPVectorizer.cpp.
References llvm::all_of(), assert(), llvm::cast(), llvm::enumerate(), P, llvm::PoisonMaskElem, and llvm::V1.
◆ adjustExtracts()
|
inline |
Definition at line 15467 of file SLPVectorizer.cpp.
References llvm::any_of(), llvm::ArrayRef(), and llvm::slpvectorizer::reorderScalars().
◆ createExtractedFieldsVector()
|
inline |
Estimates the cost of the gather of zero-extended sub-fields of width FieldWidth of the same wider integer scalar Src, permuted by Mask, as a bitcast to the field vector plus a permutation and an extension to the lane type.
Definition at line 15764 of file SLPVectorizer.cpp.
References assert(), llvm::cast(), llvm::IntegerType::get(), llvm::TargetTransformInfo::getCastContextHint(), llvm::TargetTransformInfo::getOperandInfo(), llvm::slpvectorizer::getShuffleCost(), llvm::slpvectorizer::getWidenedType(), I, llvm::isa(), llvm::ShuffleVectorInst::isIdentityMask(), llvm::make_isa_range(), llvm::slpvectorizer::BaseShuffleAnalysis::ScalarTy, and llvm::TargetTransformInfo::SK_PermuteSingleSrc.
◆ createFreeze()
|
inline |
Definition at line 15806 of file SLPVectorizer.cpp.
◆ finalize()
|
inline |
Finalize emission of the shuffles.
Definition at line 15808 of file SLPVectorizer.cpp.
◆ gather()
|
inline |
Definition at line 15719 of file SLPVectorizer.cpp.
References assert(), llvm::ArrayRef< T >::begin(), llvm::cast(), llvm::dyn_cast(), llvm::ArrayRef< T >::end(), llvm::ArrayRef< T >::front(), llvm::ConstantVector::get(), llvm::PoisonValue::get(), llvm::UndefValue::get(), llvm::ElementCount::getFixed(), llvm::Constant::getNullValue(), llvm::Type::getScalarType(), llvm::ConstantVector::getSplat(), llvm::Value::getType(), llvm::isa(), llvm::TargetTransformInfo::None, llvm::SmallVectorTemplateBase< T, bool >::push_back(), llvm::slpvectorizer::replicateMask(), llvm::slpvectorizer::BaseShuffleAnalysis::ScalarTy, llvm::ArrayRef< T >::size(), SLPReVec, and llvm::ArrayRef< T >::take_front().
◆ needToDelay()
|
inline |
Checks if the specified entry E needs to be delayed because of its dependency nodes.
Definition at line 15592 of file SLPVectorizer.cpp.
◆ resetForSameNode()
|
inline |
Reset the builder to handle perfect diamond match.
Definition at line 15598 of file SLPVectorizer.cpp.
The documentation for this class was generated from the following file:
- lib/Transforms/Vectorize/SLPVectorizer.cpp
Generated on for LLVM by
1.14.0