LLVM 20.0.0git
Classes | Public Member Functions | Public Attributes | List of all members
llvm::VPTransformState Struct Reference

VPTransformState holds information passed down when "executing" a VPlan, needed for generating the output IR. More...

#include "Transforms/Vectorize/VPlan.h"

Classes

struct  CFGState
 Hold state information used when constructing the CFG of the output IR, traversing the VPBasicBlocks and generating corresponding IR BasicBlocks. More...
 
struct  DataState
 

Public Member Functions

 VPTransformState (const TargetTransformInfo *TTI, ElementCount VF, unsigned UF, LoopInfo *LI, DominatorTree *DT, IRBuilderBase &Builder, InnerLoopVectorizer *ILV, VPlan *Plan, Type *CanonicalIVTy)
 
Valueget (VPValue *Def, bool IsScalar=false)
 Get the generated vector Value for a given VPValue Def if IsScalar is false, otherwise return the generated scalar.
 
Valueget (VPValue *Def, const VPLane &Lane)
 Get the generated Value for a given VPValue and given Part and Lane.
 
bool hasVectorValue (VPValue *Def)
 
bool hasScalarValue (VPValue *Def, VPLane Lane)
 
void set (VPValue *Def, Value *V, bool IsScalar=false)
 Set the generated vector Value for a given VPValue, if IsScalar is false.
 
void reset (VPValue *Def, Value *V)
 Reset an existing vector value for Def and a given Part.
 
void set (VPValue *Def, Value *V, const VPLane &Lane)
 Set the generated scalar V for Def and the given Lane.
 
void reset (VPValue *Def, Value *V, const VPLane &Lane)
 Reset an existing scalar value for Def and a given Lane.
 
void addNewMetadata (Instruction *To, const Instruction *Orig)
 Add additional metadata to To that was not present on Orig.
 
void addMetadata (Value *To, Instruction *From)
 Add metadata from one instruction to another.
 
void setDebugLocFrom (DebugLoc DL)
 Set the debug location in the builder using the debug location DL.
 
void packScalarIntoVectorValue (VPValue *Def, const VPLane &Lane)
 Construct the vector value of a scalarized value V one lane at a time.
 

Public Attributes

const TargetTransformInfoTTI
 Target Transform Info.
 
ElementCount VF
 The chosen Vectorization Factor of the loop being vectorized.
 
std::optional< VPLaneLane
 Hold the index to generate specific scalar instructions.
 
struct llvm::VPTransformState::DataState Data
 
struct llvm::VPTransformState::CFGState CFG
 
LoopInfoLI
 Hold a pointer to LoopInfo to register new basic blocks in the loop.
 
IRBuilderBaseBuilder
 Hold a reference to the IRBuilder used to generate output IR code.
 
InnerLoopVectorizerILV
 Hold a pointer to InnerLoopVectorizer to reuse its IR generation methods.
 
VPlanPlan
 Pointer to the VPlan code is generated for.
 
LoopCurrentVectorLoop = nullptr
 The loop object for the current parent region, or nullptr.
 
LoopVersioningLVer = nullptr
 LoopVersioning.
 
DenseMap< const SCEV *, Value * > ExpandedSCEVs
 Map SCEVs to their expanded values.
 
VPTypeAnalysis TypeAnalysis
 VPlan-based type analysis.
 

Detailed Description

VPTransformState holds information passed down when "executing" a VPlan, needed for generating the output IR.

Definition at line 236 of file VPlan.h.

Constructor & Destructor Documentation

◆ VPTransformState()

VPTransformState::VPTransformState ( const TargetTransformInfo TTI,
ElementCount  VF,
unsigned  UF,
LoopInfo LI,
DominatorTree DT,
IRBuilderBase Builder,
InnerLoopVectorizer ILV,
VPlan Plan,
Type CanonicalIVTy 
)

Definition at line 220 of file VPlan.cpp.

Member Function Documentation

◆ addMetadata()

void VPTransformState::addMetadata ( Value To,
Instruction From 
)

◆ addNewMetadata()

void VPTransformState::addNewMetadata ( Instruction To,
const Instruction Orig 
)

Add additional metadata to To that was not present on Orig.

Currently this is used to add the noalias annotations based on the inserted memchecks. Use this for instructions that are cloned into the vector loop.

Definition at line 357 of file VPlan.cpp.

References llvm::LoopVersioning::annotateInstWithNoAlias(), and LVer.

Referenced by addMetadata(), and llvm::InnerLoopVectorizer::scalarizeInstruction().

◆ get() [1/2]

Value * VPTransformState::get ( VPValue Def,
bool  IsScalar = false 
)

Get the generated vector Value for a given VPValue Def if IsScalar is false, otherwise return the generated scalar.

\See set.

Definition at line 253 of file VPlan.cpp.

References assert(), B, Builder, CFG, llvm::IRBuilderBase::CreateVectorSplat(), Data, llvm::VectorType::get(), llvm::PoisonValue::get(), get(), llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::getKnownMinValue(), llvm::BasicBlock::getTerminator(), llvm::VPlan::getVectorPreheader(), hasScalarValue(), hasVectorValue(), llvm::isa(), llvm::details::FixedOrScalableQuantity< LeafTy, ValueTy >::isScalable(), llvm::ElementCount::isScalar(), llvm::vputils::isUniformAfterVectorization(), Lane, llvm::vputils::onlyFirstLaneUsed(), packScalarIntoVectorValue(), Plan, llvm::IRBuilderBase::restoreIP(), llvm::IRBuilderBase::saveIP(), set(), llvm::IRBuilderBase::SetInsertPoint(), VF, llvm::VPTransformState::CFGState::VPBB2IRBB, llvm::VPTransformState::DataState::VPV2Scalars, and llvm::VPTransformState::DataState::VPV2Vector.

Referenced by llvm::VPIRInstruction::execute(), llvm::VPWidenRecipe::execute(), llvm::VPWidenCastRecipe::execute(), llvm::VPWidenIntrinsicRecipe::execute(), llvm::VPWidenCallRecipe::execute(), llvm::VPHistogramRecipe::execute(), llvm::VPWidenGEPRecipe::execute(), llvm::VPReverseVectorPointerRecipe::execute(), llvm::VPVectorPointerRecipe::execute(), llvm::VPWidenIntOrFpInductionRecipe::execute(), llvm::VPWidenPointerInductionRecipe::execute(), llvm::VPScalarPHIRecipe::execute(), llvm::VPWidenPHIRecipe::execute(), llvm::VPReductionPHIRecipe::execute(), llvm::VPBlendRecipe::execute(), llvm::VPInterleaveRecipe::execute(), llvm::VPReductionRecipe::execute(), llvm::VPReductionEVLRecipe::execute(), llvm::VPBranchOnMaskRecipe::execute(), llvm::VPPredInstPHIRecipe::execute(), llvm::VPWidenLoadRecipe::execute(), llvm::VPWidenLoadEVLRecipe::execute(), llvm::VPWidenStoreRecipe::execute(), llvm::VPWidenStoreEVLRecipe::execute(), llvm::VPExpandSCEVRecipe::execute(), llvm::VPActiveLaneMaskPHIRecipe::execute(), llvm::VPWidenCanonicalIVRecipe::execute(), llvm::VPDerivedIVRecipe::execute(), llvm::VPScalarIVStepsRecipe::execute(), llvm::VPWidenEVLRecipe::execute(), llvm::VPlan::execute(), llvm::LoopVectorizationPlanner::executePlan(), llvm::InnerLoopVectorizer::fixNonInductionPHIs(), fixReductionScalarResumeWhenVectorizingEpilog(), llvm::InnerLoopVectorizer::fixupIVUsers(), get(), packScalarIntoVectorValue(), and llvm::InnerLoopVectorizer::scalarizeInstruction().

◆ get() [2/2]

Value * VPTransformState::get ( VPValue Def,
const VPLane Lane 
)

◆ hasScalarValue()

bool llvm::VPTransformState::hasScalarValue ( VPValue Def,
VPLane  Lane 
)
inline

Definition at line 268 of file VPlan.h.

References Data, I, Lane, VF, and llvm::VPTransformState::DataState::VPV2Scalars.

Referenced by llvm::VPPredInstPHIRecipe::execute(), and get().

◆ hasVectorValue()

bool llvm::VPTransformState::hasVectorValue ( VPValue Def)
inline

Definition at line 266 of file VPlan.h.

References Data, and llvm::VPTransformState::DataState::VPV2Vector.

Referenced by llvm::VPPredInstPHIRecipe::execute(), and get().

◆ packScalarIntoVectorValue()

void VPTransformState::packScalarIntoVectorValue ( VPValue Def,
const VPLane Lane 
)

Construct the vector value of a scalarized value V one lane at a time.

Definition at line 398 of file VPlan.cpp.

References Builder, llvm::IRBuilderBase::CreateInsertElement(), get(), Lane, set(), and VF.

Referenced by llvm::VPReplicateRecipe::execute(), and get().

◆ reset() [1/2]

void llvm::VPTransformState::reset ( VPValue Def,
Value V 
)
inline

Reset an existing vector value for Def and a given Part.

Definition at line 289 of file VPlan.h.

References assert(), Data, and llvm::VPTransformState::DataState::VPV2Vector.

Referenced by llvm::VPPredInstPHIRecipe::execute().

◆ reset() [2/2]

void llvm::VPTransformState::reset ( VPValue Def,
Value V,
const VPLane Lane 
)
inline

Reset an existing scalar value for Def and a given Lane.

Definition at line 305 of file VPlan.h.

References assert(), Data, Lane, VF, and llvm::VPTransformState::DataState::VPV2Scalars.

◆ set() [1/2]

void llvm::VPTransformState::set ( VPValue Def,
Value V,
bool  IsScalar = false 
)
inline

Set the generated vector Value for a given VPValue, if IsScalar is false.

If IsScalar is true, set the scalar in lane 0.

Definition at line 278 of file VPlan.h.

References assert(), Data, llvm::ElementCount::isScalar(), set(), VF, and llvm::VPTransformState::DataState::VPV2Vector.

Referenced by llvm::VPInstruction::execute(), llvm::VPWidenRecipe::execute(), llvm::VPWidenCastRecipe::execute(), llvm::VPScalarCastRecipe::execute(), llvm::VPWidenIntrinsicRecipe::execute(), llvm::VPWidenCallRecipe::execute(), llvm::VPWidenGEPRecipe::execute(), llvm::VPReverseVectorPointerRecipe::execute(), llvm::VPVectorPointerRecipe::execute(), llvm::VPWidenIntOrFpInductionRecipe::execute(), llvm::VPWidenPointerInductionRecipe::execute(), llvm::VPScalarPHIRecipe::execute(), llvm::VPWidenPHIRecipe::execute(), llvm::VPFirstOrderRecurrencePHIRecipe::execute(), llvm::VPReductionPHIRecipe::execute(), llvm::VPBlendRecipe::execute(), llvm::VPInterleaveRecipe::execute(), llvm::VPReductionRecipe::execute(), llvm::VPReductionEVLRecipe::execute(), llvm::VPReplicateRecipe::execute(), llvm::VPPredInstPHIRecipe::execute(), llvm::VPWidenLoadRecipe::execute(), llvm::VPWidenLoadEVLRecipe::execute(), llvm::VPExpandSCEVRecipe::execute(), llvm::VPActiveLaneMaskPHIRecipe::execute(), llvm::VPWidenCanonicalIVRecipe::execute(), llvm::VPDerivedIVRecipe::execute(), llvm::VPScalarIVStepsRecipe::execute(), llvm::VPWidenEVLRecipe::execute(), get(), packScalarIntoVectorValue(), llvm::InnerLoopVectorizer::scalarizeInstruction(), and set().

◆ set() [2/2]

void llvm::VPTransformState::set ( VPValue Def,
Value V,
const VPLane Lane 
)
inline

Set the generated scalar V for Def and the given Lane.

Definition at line 295 of file VPlan.h.

References assert(), Data, Lane, VF, and llvm::VPTransformState::DataState::VPV2Scalars.

◆ setDebugLocFrom()

void VPTransformState::setDebugLocFrom ( DebugLoc  DL)

Member Data Documentation

◆ Builder

IRBuilderBase& llvm::VPTransformState::Builder

Hold a reference to the IRBuilder used to generate output IR code.

Definition at line 368 of file VPlan.h.

Referenced by llvm::VPlan::createInitialVPlan(), llvm::VPInstruction::execute(), llvm::VPIRInstruction::execute(), llvm::VPWidenRecipe::execute(), llvm::VPWidenCastRecipe::execute(), llvm::VPWidenIntrinsicRecipe::execute(), llvm::VPWidenCallRecipe::execute(), llvm::VPHistogramRecipe::execute(), llvm::VPWidenGEPRecipe::execute(), llvm::VPReverseVectorPointerRecipe::execute(), llvm::VPVectorPointerRecipe::execute(), llvm::VPWidenIntOrFpInductionRecipe::execute(), llvm::VPWidenPointerInductionRecipe::execute(), llvm::VPScalarPHIRecipe::execute(), llvm::VPWidenPHIRecipe::execute(), llvm::VPFirstOrderRecurrencePHIRecipe::execute(), llvm::VPReductionPHIRecipe::execute(), llvm::VPBlendRecipe::execute(), llvm::VPInterleaveRecipe::execute(), llvm::VPReductionRecipe::execute(), llvm::VPReductionEVLRecipe::execute(), llvm::VPBranchOnMaskRecipe::execute(), llvm::VPPredInstPHIRecipe::execute(), llvm::VPWidenLoadRecipe::execute(), llvm::VPWidenLoadEVLRecipe::execute(), llvm::VPWidenStoreRecipe::execute(), llvm::VPWidenStoreEVLRecipe::execute(), llvm::VPExpandSCEVRecipe::execute(), llvm::VPActiveLaneMaskPHIRecipe::execute(), llvm::VPDerivedIVRecipe::execute(), llvm::VPScalarIVStepsRecipe::execute(), llvm::VPWidenEVLRecipe::execute(), llvm::VPlan::execute(), llvm::VPBasicBlock::execute(), llvm::VPIRBasicBlock::execute(), llvm::LoopVectorizationPlanner::executePlan(), get(), packScalarIntoVectorValue(), llvm::VPlan::prepareToExecute(), llvm::InnerLoopVectorizer::scalarizeInstruction(), and setDebugLocFrom().

◆ CFG

struct llvm::VPTransformState::CFGState llvm::VPTransformState::CFG

◆ CurrentVectorLoop

Loop* llvm::VPTransformState::CurrentVectorLoop = nullptr

The loop object for the current parent region, or nullptr.

Definition at line 377 of file VPlan.h.

Referenced by llvm::VPReductionPHIRecipe::execute(), llvm::VPBasicBlock::execute(), and llvm::VPRegionBlock::execute().

◆ Data

struct llvm::VPTransformState::DataState llvm::VPTransformState::Data

◆ ExpandedSCEVs

DenseMap<const SCEV *, Value *> llvm::VPTransformState::ExpandedSCEVs

Map SCEVs to their expanded values.

Populated when executing VPExpandSCEVRecipes.

Definition at line 388 of file VPlan.h.

Referenced by llvm::VPExpandSCEVRecipe::execute(), and llvm::LoopVectorizationPlanner::executePlan().

◆ ILV

InnerLoopVectorizer* llvm::VPTransformState::ILV

Hold a pointer to InnerLoopVectorizer to reuse its IR generation methods.

Definition at line 371 of file VPlan.h.

Referenced by llvm::VPReplicateRecipe::execute().

◆ Lane

std::optional<VPLane> llvm::VPTransformState::Lane

◆ LI

LoopInfo* llvm::VPTransformState::LI

Hold a pointer to LoopInfo to register new basic blocks in the loop.

Definition at line 365 of file VPlan.h.

Referenced by llvm::VPBasicBlock::execute(), and llvm::VPRegionBlock::execute().

◆ LVer

LoopVersioning* llvm::VPTransformState::LVer = nullptr

LoopVersioning.

It's only set up (non-null) if memchecks were used.

This is currently only used to add no-alias metadata based on the memchecks. The actually versioning is performed manually.

Definition at line 384 of file VPlan.h.

Referenced by addNewMetadata(), and llvm::LoopVectorizationPlanner::executePlan().

◆ Plan

VPlan* llvm::VPTransformState::Plan

◆ TTI

const TargetTransformInfo* llvm::VPTransformState::TTI

Target Transform Info.

Definition at line 241 of file VPlan.h.

Referenced by llvm::VPWidenIntrinsicRecipe::execute().

◆ TypeAnalysis

VPTypeAnalysis llvm::VPTransformState::TypeAnalysis

◆ VF

ElementCount llvm::VPTransformState::VF

The documentation for this struct was generated from the following files: