LLVM 19.0.0git
Public Types | Public Member Functions | Protected Types | Protected Member Functions | Protected Attributes | Friends | List of all members
llvm::InnerLoopVectorizer Class Reference

InnerLoopVectorizer vectorizes loops which contain only one basic block to a specified vectorization factor (VF). More...

Inheritance diagram for llvm::InnerLoopVectorizer:
Inheritance graph
[legend]

Public Types

using VectorParts = SmallVector< Value *, 2 >
 A type for vectorized values in the new loop.
 

Public Member Functions

 InnerLoopVectorizer (Loop *OrigLoop, PredicatedScalarEvolution &PSE, LoopInfo *LI, DominatorTree *DT, const TargetLibraryInfo *TLI, const TargetTransformInfo *TTI, AssumptionCache *AC, OptimizationRemarkEmitter *ORE, ElementCount VecWidth, ElementCount MinProfitableTripCount, unsigned UnrollFactor, LoopVectorizationLegality *LVL, LoopVectorizationCostModel *CM, BlockFrequencyInfo *BFI, ProfileSummaryInfo *PSI, GeneratedRTChecks &RTChecks)
 
virtual ~InnerLoopVectorizer ()=default
 
virtual std::pair< BasicBlock *, Value * > createVectorizedLoopSkeleton (const SCEV2ValueTy &ExpandedSCEVs)
 Create a new empty loop that will contain vectorized instructions later on, while the old loop will be used as the scalar remainder.
 
void fixVectorizedLoop (VPTransformState &State, VPlan &Plan)
 Fix the vectorized code, taking care of header phi's, live-outs, and more.
 
bool areSafetyChecksAdded ()
 
void scalarizeInstruction (const Instruction *Instr, VPReplicateRecipe *RepRecipe, const VPIteration &Instance, VPTransformState &State)
 A helper function to scalarize a single Instruction in the innermost loop.
 
void vectorizeInterleaveGroup (const InterleaveGroup< Instruction > *Group, ArrayRef< VPValue * > VPDefs, VPTransformState &State, VPValue *Addr, ArrayRef< VPValue * > StoredValues, VPValue *BlockInMask, bool NeedsMaskForGaps)
 Try to vectorize interleaved access group Group with the base address given in Addr, optionally masking the vector operations if BlockInMask is non-null.
 
void fixNonInductionPHIs (VPlan &Plan, VPTransformState &State)
 Fix the non-induction PHIs in Plan.
 
bool useOrderedReductions (const RecurrenceDescriptor &RdxDesc)
 Returns true if the reordering of FP operations is not allowed, but we are able to vectorize with strict in-order reductions for the given RdxDesc.
 
PHINodecreateInductionResumeValue (PHINode *OrigPhi, const InductionDescriptor &ID, Value *Step, ArrayRef< BasicBlock * > BypassBlocks, std::pair< BasicBlock *, Value * > AdditionalBypass={nullptr, nullptr})
 Create a new phi node for the induction variable OrigPhi to resume iteration count in the scalar epilogue, from where the vectorized loop left off.
 
ValuegetTripCount () const
 Returns the original loop trip count.
 
void setTripCount (Value *TC)
 Used to set the trip count after ILV's construction and after the preheader block has been executed.
 

Protected Types

using PhiVector = SmallVector< PHINode *, 4 >
 A small list of PHINodes.
 
using ScalarParts = SmallVector< SmallVector< Value *, 4 >, 2 >
 A type for scalarized values in the new loop.
 

Protected Member Functions

void fixupIVUsers (PHINode *OrigPhi, const InductionDescriptor &II, Value *VectorTripCount, Value *EndValue, BasicBlock *MiddleBlock, BasicBlock *VectorHeader, VPlan &Plan, VPTransformState &State)
 Set up the values of the IVs correctly when exiting the vector loop.
 
void fixFixedOrderRecurrence (VPFirstOrderRecurrencePHIRecipe *PhiR, VPTransformState &State)
 Create the exit value of first order recurrences in the middle block and update their users.
 
void fixReduction (VPReductionPHIRecipe *Phi, VPTransformState &State)
 Create code for the loop exit value of the reduction.
 
void sinkScalarOperands (Instruction *PredInst)
 Iteratively sink the scalarized operands of a predicated instruction into the block that was created for it.
 
ValuegetOrCreateVectorTripCount (BasicBlock *InsertBlock)
 Returns (and creates if needed) the trip count of the widened loop.
 
ValuecreateBitOrPointerCast (Value *V, VectorType *DstVTy, const DataLayout &DL)
 Returns a bitcasted value to the requested vector type.
 
void emitIterationCountCheck (BasicBlock *Bypass)
 Emit a bypass check to see if the vector trip count is zero, including if it overflows.
 
BasicBlockemitSCEVChecks (BasicBlock *Bypass)
 Emit a bypass check to see if all of the SCEV assumptions we've had to make are correct.
 
BasicBlockemitMemRuntimeChecks (BasicBlock *Bypass)
 Emit bypass checks to check any memory assumptions we may have made.
 
void createVectorLoopSkeleton (StringRef Prefix)
 Emit basic blocks (prefixed with Prefix) for the iteration check, vector loop preheader, middle block and scalar preheader.
 
void createInductionResumeValues (const SCEV2ValueTy &ExpandedSCEVs, std::pair< BasicBlock *, Value * > AdditionalBypass={nullptr, nullptr})
 Create new phi nodes for the induction variables to resume iteration count in the scalar epilogue, from where the vectorized loop left off.
 
BasicBlockcompleteLoopSkeleton ()
 Complete the loop skeleton by adding debug MDs, creating appropriate conditional branches in the middle block, preparing the builder and running the verifier.
 
virtual void printDebugTracesAtStart ()
 Allow subclasses to override and print debug traces before/after vplan execution, when trace information is requested.
 
virtual void printDebugTracesAtEnd ()
 

Protected Attributes

LoopOrigLoop
 The original loop.
 
PredicatedScalarEvolutionPSE
 A wrapper around ScalarEvolution used to add runtime SCEV checks.
 
LoopInfoLI
 Loop Info.
 
DominatorTreeDT
 Dominator Tree.
 
const TargetLibraryInfoTLI
 Target Library Info.
 
const TargetTransformInfoTTI
 Target Transform Info.
 
AssumptionCacheAC
 Assumption Cache.
 
OptimizationRemarkEmitterORE
 Interface to emit optimization remarks.
 
ElementCount VF
 The vectorization SIMD factor to use.
 
ElementCount MinProfitableTripCount
 
unsigned UF
 The vectorization unroll factor to use.
 
IRBuilder Builder
 The builder that we use.
 
BasicBlockLoopVectorPreHeader
 The vector-loop preheader.
 
BasicBlockLoopScalarPreHeader
 The scalar-loop preheader.
 
BasicBlockLoopMiddleBlock
 Middle Block between the vector and the scalar.
 
BasicBlockLoopExitBlock
 The unique ExitBlock of the scalar loop if one exists.
 
BasicBlockLoopScalarBody
 The scalar loop body.
 
SmallVector< BasicBlock *, 4 > LoopBypassBlocks
 A list of all bypass blocks. The first block is the entry of the loop.
 
SmallVector< Instruction *, 4 > PredicatedInstructions
 Store instructions that were predicated.
 
ValueTripCount = nullptr
 Trip count of the original loop.
 
ValueVectorTripCount = nullptr
 Trip count of the widened loop (TripCount - TripCount % (VF*UF))
 
LoopVectorizationLegalityLegal
 The legality analysis.
 
LoopVectorizationCostModelCost
 The profitablity analysis.
 
bool AddedSafetyChecks = false
 
DenseMap< PHINode *, Value * > IVEndValues
 
BlockFrequencyInfoBFI
 BFI and PSI are used to check for profile guided size optimizations.
 
ProfileSummaryInfoPSI
 
bool OptForSizeBasedOnProfile
 
GeneratedRTChecks & RTChecks
 Structure to hold information about generated runtime checks, responsible for cleaning the checks, if vectorization turns out unprofitable.
 
SmallMapVector< const RecurrenceDescriptor *, PHINode *, 4 > ReductionResumeValues
 

Friends

class LoopVectorizationPlanner
 

Detailed Description

InnerLoopVectorizer vectorizes loops which contain only one basic block to a specified vectorization factor (VF).

This class performs the widening of scalars into vectors, or multiple scalars. This class also implements the following features:

Definition at line 499 of file LoopVectorize.cpp.

Member Typedef Documentation

◆ PhiVector

A small list of PHINodes.

Definition at line 601 of file LoopVectorize.cpp.

◆ ScalarParts

A type for scalarized values in the new loop.

Each value from the original loop, when scalarized, is represented by UF x VF scalar values in the new unrolled loop, where UF is the unroll factor and VF is the vectorization factor.

Definition at line 607 of file LoopVectorize.cpp.

◆ VectorParts

A type for vectorized values in the new loop.

Each value from the original loop, when vectorized, is represented by UF vector values in the new unrolled loop, where UF is the unroll factor.

Definition at line 548 of file LoopVectorize.cpp.

Constructor & Destructor Documentation

◆ InnerLoopVectorizer()

llvm::InnerLoopVectorizer::InnerLoopVectorizer ( Loop OrigLoop,
PredicatedScalarEvolution PSE,
LoopInfo LI,
DominatorTree DT,
const TargetLibraryInfo TLI,
const TargetTransformInfo TTI,
AssumptionCache AC,
OptimizationRemarkEmitter ORE,
ElementCount  VecWidth,
ElementCount  MinProfitableTripCount,
unsigned  UnrollFactor,
LoopVectorizationLegality LVL,
LoopVectorizationCostModel CM,
BlockFrequencyInfo BFI,
ProfileSummaryInfo PSI,
GeneratedRTChecks &  RTChecks 
)
inline

◆ ~InnerLoopVectorizer()

virtual llvm::InnerLoopVectorizer::~InnerLoopVectorizer ( )
virtualdefault

Member Function Documentation

◆ areSafetyChecksAdded()

bool llvm::InnerLoopVectorizer::areSafetyChecksAdded ( )
inline

Definition at line 543 of file LoopVectorize.cpp.

References AddedSafetyChecks.

Referenced by llvm::LoopVectorizePass::processLoop().

◆ completeLoopSkeleton()

BasicBlock * InnerLoopVectorizer::completeLoopSkeleton ( )
protected

◆ createBitOrPointerCast()

Value * InnerLoopVectorizer::createBitOrPointerCast ( Value V,
VectorType DstVTy,
const DataLayout DL 
)
protected

Returns a bitcasted value to the requested vector type.

Also handles bitcasts of vector<float> <-> vector<pointer> types.

Definition at line 2761 of file LoopVectorize.cpp.

References assert(), Builder, llvm::IRBuilderBase::CreateBitOrPointerCast(), DL, llvm::VectorType::get(), llvm::Type::getIntNTy(), llvm::CastInst::isBitOrNoopPointerCastable(), llvm::Type::isFloatingPointTy(), llvm::Type::isPointerTy(), and VF.

Referenced by vectorizeInterleaveGroup().

◆ createInductionResumeValue()

PHINode * InnerLoopVectorizer::createInductionResumeValue ( PHINode OrigPhi,
const InductionDescriptor ID,
Value Step,
ArrayRef< BasicBlock * >  BypassBlocks,
std::pair< BasicBlock *, Value * >  AdditionalBypass = {nullptr, nullptr} 
)

Create a new phi node for the induction variable OrigPhi to resume iteration count in the scalar epilogue, from where the vectorized loop left off.

Step is the SCEV-expanded induction step to use. In cases where the loop skeleton is more complicated (i.e., epilogue vectorization) and the resume values can come from an additional bypass block, the AdditionalBypass pair provides information about the bypass block and the end value on the edge from bypass to this loop.

Definition at line 2974 of file LoopVectorize.cpp.

References llvm::PHINode::addIncoming(), assert(), B, llvm::PHINode::Create(), emitTransformedIndex(), llvm::Instruction::getDebugLoc(), llvm::Instruction::getFastMathFlags(), llvm::InductionDescriptor::getInductionBinOp(), llvm::ilist_node_impl< OptionsT >::getIterator(), llvm::InductionDescriptor::getKind(), getOrCreateVectorTripCount(), llvm::LoopVectorizationLegality::getPrimaryInduction(), llvm::InductionDescriptor::getStartValue(), llvm::BasicBlock::getTerminator(), llvm::Value::getType(), IVEndValues, Legal, LoopMiddleBlock, LoopScalarPreHeader, LoopVectorPreHeader, llvm::Instruction::setDebugLoc(), llvm::PHINode::setIncomingValueForBlock(), llvm::Value::setName(), and VectorTripCount.

Referenced by createInductionResumeValues(), and llvm::LoopVectorizePass::processLoop().

◆ createInductionResumeValues()

void InnerLoopVectorizer::createInductionResumeValues ( const SCEV2ValueTy &  ExpandedSCEVs,
std::pair< BasicBlock *, Value * >  AdditionalBypass = {nullptr, nullptr} 
)
protected

Create new phi nodes for the induction variables to resume iteration count in the scalar epilogue, from where the vectorized loop left off.

In cases where the loop skeleton is more complicated (eg. epilogue vectorization) and the resume values can come from an additional bypass block, the AdditionalBypass pair provides information about the bypass block and the end value on the edge from bypass to this loop.

Definition at line 3046 of file LoopVectorize.cpp.

References assert(), createInductionResumeValue(), getExpandedStep(), llvm::LoopVectorizationLegality::getInductionVars(), Legal, LoopBypassBlocks, LoopScalarPreHeader, and llvm::PHINode::setIncomingValueForBlock().

Referenced by llvm::EpilogueVectorizerEpilogueLoop::createEpilogueVectorizedLoopSkeleton(), and createVectorizedLoopSkeleton().

◆ createVectorizedLoopSkeleton()

std::pair< BasicBlock *, Value * > InnerLoopVectorizer::createVectorizedLoopSkeleton ( const SCEV2ValueTy &  ExpandedSCEVs)
virtual

Create a new empty loop that will contain vectorized instructions later on, while the old loop will be used as the scalar remainder.

Control flow is generated around the vectorized (and scalar epilogue) loops consisting of various checks and bypasses. Return the pre-header block of the new loop and the start value for the canonical induction, if it is != 0. The latter is the case when vectorizing the epilogue loop. In the case of epilogue vectorization, this function is overriden to handle the more complex control flow around the loops. ExpandedSCEVs is used to look up SCEV expansions for expressions needed during skeleton creation.

Reimplemented in llvm::InnerLoopAndEpilogueVectorizer.

Definition at line 3115 of file LoopVectorize.cpp.

References completeLoopSkeleton(), createInductionResumeValues(), createVectorLoopSkeleton(), emitIterationCountCheck(), emitMemRuntimeChecks(), emitSCEVChecks(), and LoopScalarPreHeader.

Referenced by llvm::LoopVectorizationPlanner::executePlan().

◆ createVectorLoopSkeleton()

void InnerLoopVectorizer::createVectorLoopSkeleton ( StringRef  Prefix)
protected

◆ emitIterationCountCheck()

void InnerLoopVectorizer::emitIterationCountCheck ( BasicBlock Bypass)
protected

◆ emitMemRuntimeChecks()

BasicBlock * InnerLoopVectorizer::emitMemRuntimeChecks ( BasicBlock Bypass)
protected

◆ emitSCEVChecks()

BasicBlock * InnerLoopVectorizer::emitSCEVChecks ( BasicBlock Bypass)
protected

◆ fixFixedOrderRecurrence()

void InnerLoopVectorizer::fixFixedOrderRecurrence ( VPFirstOrderRecurrencePHIRecipe PhiR,
VPTransformState State 
)
protected

◆ fixNonInductionPHIs()

void InnerLoopVectorizer::fixNonInductionPHIs ( VPlan Plan,
VPTransformState State 
)

◆ fixReduction()

void llvm::InnerLoopVectorizer::fixReduction ( VPReductionPHIRecipe Phi,
VPTransformState State 
)
protected

Create code for the loop exit value of the reduction.

◆ fixupIVUsers()

void InnerLoopVectorizer::fixupIVUsers ( PHINode OrigPhi,
const InductionDescriptor II,
Value VectorTripCount,
Value EndValue,
BasicBlock MiddleBlock,
BasicBlock VectorHeader,
VPlan Plan,
VPTransformState State 
)
protected

◆ fixVectorizedLoop()

void InnerLoopVectorizer::fixVectorizedLoop ( VPTransformState State,
VPlan Plan 
)

◆ getOrCreateVectorTripCount()

Value * InnerLoopVectorizer::getOrCreateVectorTripCount ( BasicBlock InsertBlock)
protected

◆ getTripCount()

Value * llvm::InnerLoopVectorizer::getTripCount ( ) const
inline

◆ printDebugTracesAtEnd()

virtual void llvm::InnerLoopVectorizer::printDebugTracesAtEnd ( )
inlineprotectedvirtual

◆ printDebugTracesAtStart()

virtual void llvm::InnerLoopVectorizer::printDebugTracesAtStart ( )
inlineprotectedvirtual

Allow subclasses to override and print debug traces before/after vplan execution, when trace information is requested.

Reimplemented in llvm::EpilogueVectorizerMainLoop, and llvm::EpilogueVectorizerEpilogueLoop.

Definition at line 670 of file LoopVectorize.cpp.

Referenced by llvm::LoopVectorizationPlanner::executePlan().

◆ scalarizeInstruction()

void InnerLoopVectorizer::scalarizeInstruction ( const Instruction Instr,
VPReplicateRecipe RepRecipe,
const VPIteration Instance,
VPTransformState State 
)

◆ setTripCount()

void llvm::InnerLoopVectorizer::setTripCount ( Value TC)
inline

Used to set the trip count after ILV's construction and after the preheader block has been executed.

Note that this always holds the trip count of the original loop for both main loop and epilogue vectorization.

Definition at line 595 of file LoopVectorize.cpp.

References TripCount.

Referenced by llvm::LoopVectorizationPlanner::executePlan(), and llvm::LoopVectorizePass::processLoop().

◆ sinkScalarOperands()

void InnerLoopVectorizer::sinkScalarOperands ( Instruction PredInst)
protected

◆ useOrderedReductions()

bool InnerLoopVectorizer::useOrderedReductions ( const RecurrenceDescriptor RdxDesc)

Returns true if the reordering of FP operations is not allowed, but we are able to vectorize with strict in-order reductions for the given RdxDesc.

Definition at line 3670 of file LoopVectorize.cpp.

Referenced by llvm::VPReductionRecipe::execute().

◆ vectorizeInterleaveGroup()

void InnerLoopVectorizer::vectorizeInterleaveGroup ( const InterleaveGroup< Instruction > *  Group,
ArrayRef< VPValue * >  VPDefs,
VPTransformState State,
VPValue Addr,
ArrayRef< VPValue * >  StoredValues,
VPValue BlockInMask,
bool  NeedsMaskForGaps 
)

Try to vectorize interleaved access group Group with the base address given in Addr, optionally masking the vector operations if BlockInMask is non-null.

Use State to translate given VPValues to IR values in the vectorized loop.

Definition at line 2411 of file LoopVectorize.cpp.

References llvm::InterleaveGroup< InstTy >::addMetadata(), Addr, assert(), Builder, llvm::IRBuilderBase::CreateAdd(), llvm::IRBuilderBase::CreateAlignedLoad(), llvm::IRBuilderBase::CreateAlignedStore(), llvm::IRBuilderBase::CreateBinOp(), llvm::createBitMaskForGaps(), createBitOrPointerCast(), llvm::IRBuilderBase::CreateExtractValue(), llvm::IRBuilderBase::CreateGEP(), llvm::IRBuilderBase::CreateIntrinsic(), llvm::IRBuilderBase::CreateMaskedLoad(), llvm::IRBuilderBase::CreateMaskedStore(), llvm::IRBuilderBase::CreateMul(), llvm::IRBuilderBase::CreateNeg(), llvm::createReplicatedMask(), llvm::IRBuilderBase::CreateShuffleVector(), llvm::createStrideMask(), llvm::IRBuilderBase::CreateSub(), llvm::IRBuilderBase::CreateVectorReverse(), DL, gep, llvm::VectorType::get(), llvm::PoisonValue::get(), llvm::VPTransformState::get(), llvm::InterleaveGroup< InstTy >::getAlign(), llvm::InterleaveGroup< InstTy >::getFactor(), llvm::InterleaveGroup< InstTy >::getIndex(), llvm::InterleaveGroup< InstTy >::getInsertPos(), llvm::IRBuilderBase::getInt1Ty(), llvm::IRBuilderBase::getInt32(), llvm::IRBuilderBase::getInt32Ty(), llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::getKnownMinValue(), llvm::getLoadStoreType(), llvm::InterleaveGroup< InstTy >::getMember(), llvm::getRuntimeVF(), llvm::Value::getType(), I, Idx, interleaveVectors(), llvm::InterleaveGroup< InstTy >::isReverse(), llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::isScalable(), llvm::SmallVectorTemplateBase< T, bool >::push_back(), llvm::VPTransformState::set(), llvm::VPTransformState::setDebugLocFrom(), llvm::Value::stripPointerCasts(), UF, useMaskedInterleavedAccesses(), and VF.

Referenced by llvm::VPInterleaveRecipe::execute().

Friends And Related Function Documentation

◆ LoopVectorizationPlanner

friend class LoopVectorizationPlanner
friend

Definition at line 598 of file LoopVectorize.cpp.

Member Data Documentation

◆ AC

AssumptionCache* llvm::InnerLoopVectorizer::AC
protected

Assumption Cache.

Definition at line 694 of file LoopVectorize.cpp.

Referenced by scalarizeInstruction().

◆ AddedSafetyChecks

bool llvm::InnerLoopVectorizer::AddedSafetyChecks = false
protected

Definition at line 749 of file LoopVectorize.cpp.

Referenced by areSafetyChecksAdded(), emitMemRuntimeChecks(), and emitSCEVChecks().

◆ BFI

BlockFrequencyInfo* llvm::InnerLoopVectorizer::BFI
protected

BFI and PSI are used to check for profile guided size optimizations.

Definition at line 756 of file LoopVectorize.cpp.

Referenced by InnerLoopVectorizer().

◆ Builder

IRBuilder llvm::InnerLoopVectorizer::Builder
protected

◆ Cost

LoopVectorizationCostModel* llvm::InnerLoopVectorizer::Cost
protected

The profitablity analysis.

Definition at line 746 of file LoopVectorize.cpp.

◆ DT

DominatorTree* llvm::InnerLoopVectorizer::DT
protected

◆ IVEndValues

DenseMap<PHINode *, Value *> llvm::InnerLoopVectorizer::IVEndValues
protected

Definition at line 753 of file LoopVectorize.cpp.

Referenced by createInductionResumeValue(), and fixVectorizedLoop().

◆ Legal

LoopVectorizationLegality* llvm::InnerLoopVectorizer::Legal
protected

◆ LI

LoopInfo* llvm::InnerLoopVectorizer::LI
protected

◆ LoopBypassBlocks

SmallVector<BasicBlock *, 4> llvm::InnerLoopVectorizer::LoopBypassBlocks
protected

◆ LoopExitBlock

BasicBlock* llvm::InnerLoopVectorizer::LoopExitBlock
protected

The unique ExitBlock of the scalar loop if one exists.

Note that there can be multiple exiting edges reaching this block.

Definition at line 725 of file LoopVectorize.cpp.

Referenced by llvm::EpilogueVectorizerEpilogueLoop::createEpilogueVectorizedLoopSkeleton(), createVectorLoopSkeleton(), emitIterationCountCheck(), llvm::EpilogueVectorizerMainLoop::emitIterationCountCheck(), and emitSCEVChecks().

◆ LoopMiddleBlock

BasicBlock* llvm::InnerLoopVectorizer::LoopMiddleBlock
protected

Middle Block between the vector and the scalar.

Definition at line 721 of file LoopVectorize.cpp.

Referenced by completeLoopSkeleton(), createInductionResumeValue(), createVectorLoopSkeleton(), fixFixedOrderRecurrence(), and fixVectorizedLoop().

◆ LoopScalarBody

BasicBlock* llvm::InnerLoopVectorizer::LoopScalarBody
protected

The scalar loop body.

Definition at line 728 of file LoopVectorize.cpp.

Referenced by createVectorLoopSkeleton(), and fixVectorizedLoop().

◆ LoopScalarPreHeader

BasicBlock* llvm::InnerLoopVectorizer::LoopScalarPreHeader
protected

◆ LoopVectorPreHeader

BasicBlock* llvm::InnerLoopVectorizer::LoopVectorPreHeader
protected

◆ MinProfitableTripCount

ElementCount llvm::InnerLoopVectorizer::MinProfitableTripCount
protected

Definition at line 703 of file LoopVectorize.cpp.

Referenced by emitIterationCountCheck(), and InnerLoopVectorizer().

◆ OptForSizeBasedOnProfile

bool llvm::InnerLoopVectorizer::OptForSizeBasedOnProfile
protected

Definition at line 761 of file LoopVectorize.cpp.

Referenced by emitMemRuntimeChecks(), emitSCEVChecks(), and InnerLoopVectorizer().

◆ ORE

OptimizationRemarkEmitter* llvm::InnerLoopVectorizer::ORE
protected

Interface to emit optimization remarks.

Definition at line 697 of file LoopVectorize.cpp.

Referenced by emitMemRuntimeChecks().

◆ OrigLoop

Loop* llvm::InnerLoopVectorizer::OrigLoop
protected

◆ PredicatedInstructions

SmallVector<Instruction *, 4> llvm::InnerLoopVectorizer::PredicatedInstructions
protected

Store instructions that were predicated.

Definition at line 734 of file LoopVectorize.cpp.

Referenced by fixVectorizedLoop(), and scalarizeInstruction().

◆ PSE

PredicatedScalarEvolution& llvm::InnerLoopVectorizer::PSE
protected

A wrapper around ScalarEvolution used to add runtime SCEV checks.

Applies dynamic knowledge to simplify SCEV expressions and converts them to a more usable form.

Definition at line 679 of file LoopVectorize.cpp.

Referenced by fixVectorizedLoop().

◆ PSI

ProfileSummaryInfo* llvm::InnerLoopVectorizer::PSI
protected

Definition at line 757 of file LoopVectorize.cpp.

Referenced by InnerLoopVectorizer().

◆ ReductionResumeValues

SmallMapVector<const RecurrenceDescriptor *, PHINode *, 4> llvm::InnerLoopVectorizer::ReductionResumeValues
protected

Definition at line 770 of file LoopVectorize.cpp.

◆ RTChecks

GeneratedRTChecks& llvm::InnerLoopVectorizer::RTChecks
protected

Structure to hold information about generated runtime checks, responsible for cleaning the checks, if vectorization turns out unprofitable.

Definition at line 765 of file LoopVectorize.cpp.

Referenced by emitMemRuntimeChecks(), and emitSCEVChecks().

◆ TLI

const TargetLibraryInfo* llvm::InnerLoopVectorizer::TLI
protected

Target Library Info.

Definition at line 688 of file LoopVectorize.cpp.

◆ TripCount

Value* llvm::InnerLoopVectorizer::TripCount = nullptr
protected

◆ TTI

const TargetTransformInfo* llvm::InnerLoopVectorizer::TTI
protected

Target Transform Info.

Definition at line 691 of file LoopVectorize.cpp.

◆ UF

unsigned llvm::InnerLoopVectorizer::UF
protected

◆ VectorTripCount

Value* llvm::InnerLoopVectorizer::VectorTripCount = nullptr
protected

Trip count of the widened loop (TripCount - TripCount % (VF*UF))

Definition at line 740 of file LoopVectorize.cpp.

Referenced by completeLoopSkeleton(), createInductionResumeValue(), fixupIVUsers(), and getOrCreateVectorTripCount().

◆ VF

ElementCount llvm::InnerLoopVectorizer::VF
protected

The documentation for this class was generated from the following file: