LLVM 19.0.0git
Public Member Functions | Protected Member Functions | Protected Attributes | List of all members
llvm::PeelingModuloScheduleExpander Class Reference

A reimplementation of ModuloScheduleExpander. More...

#include "llvm/CodeGen/ModuloSchedule.h"

Public Member Functions

 PeelingModuloScheduleExpander (MachineFunction &MF, ModuloSchedule &S, LiveIntervals *LIS)
 
void expand ()
 
void validateAgainstModuloScheduleExpander ()
 Runs ModuloScheduleExpander and treats it as a golden input to validate aspects of the code generated by PeelingModuloScheduleExpander.
 

Protected Member Functions

void rewriteKernel ()
 Converts BB from the original loop body to the rewritten, pipelined steady-state.
 
MachineBasicBlockpeelKernel (LoopPeelDirection LPD)
 Peels one iteration of the rewritten kernel (BB) in the specified direction.
 
void filterInstructions (MachineBasicBlock *MB, int MinStage)
 
void moveStageBetweenBlocks (MachineBasicBlock *DestBB, MachineBasicBlock *SourceBB, unsigned Stage)
 
void peelPrologAndEpilogs ()
 Peel the kernel forwards and backwards to produce prologs and epilogs, and stitch them together.
 
Register getEquivalentRegisterIn (Register Reg, MachineBasicBlock *BB)
 All prolog and epilog blocks are clones of the kernel, so any produced register in one block has an corollary in all other blocks.
 
void rewriteUsesOf (MachineInstr *MI)
 Change all users of MI, if MI is predicated out (LiveStages[MI->getParent()] == false).
 
void fixupBranches ()
 Insert branches between prologs, kernel and epilogs.
 
MachineBasicBlockCreateLCSSAExitingBlock ()
 Create a poor-man's LCSSA by cloning only the PHIs from the kernel block to a block dominated by all prologs and epilogs.
 
unsigned getStage (MachineInstr *MI)
 Helper to get the stage of an instruction in the schedule.
 
Register getPhiCanonicalReg (MachineInstr *CanonicalPhi, MachineInstr *Phi)
 Helper function to find the right canonical register for a phi instruction coming from a peeled out prologue.
 

Protected Attributes

ModuloScheduleSchedule
 
MachineFunctionMF
 
const TargetSubtargetInfoST
 
MachineRegisterInfoMRI
 
const TargetInstrInfoTII = nullptr
 
LiveIntervalsLIS = nullptr
 
MachineBasicBlockBB = nullptr
 The original loop block that gets rewritten in-place.
 
MachineBasicBlockPreheader = nullptr
 The original loop preheader.
 
SmallVector< MachineBasicBlock *, 4 > Prologs
 All prolog and epilog blocks.
 
SmallVector< MachineBasicBlock *, 4 > Epilogs
 
DenseMap< MachineBasicBlock *, BitVectorLiveStages
 For every block, the stages that are produced.
 
DenseMap< MachineBasicBlock *, BitVectorAvailableStages
 For every block, the stages that are available.
 
DenseMap< MachineInstr *, unsignedPhiNodeLoopIteration
 When peeling the epilogue keep track of the distance between the phi nodes and the kernel.
 
DenseMap< MachineInstr *, MachineInstr * > CanonicalMIs
 CanonicalMIs and BlockMIs form a bidirectional map between any of the loop kernel clones.
 
DenseMap< std::pair< MachineBasicBlock *, MachineInstr * >, MachineInstr * > BlockMIs
 
std::deque< MachineBasicBlock * > PeeledFront
 State passed from peelKernel to peelPrologAndEpilogs().
 
std::deque< MachineBasicBlock * > PeeledBack
 
SmallVector< MachineInstr *, 4 > IllegalPhisToDelete
 Illegal phis that need to be deleted once we re-link stages.
 
std::unique_ptr< TargetInstrInfo::PipelinerLoopInfoLoopInfo
 Target loop info before kernel peeling.
 

Detailed Description

A reimplementation of ModuloScheduleExpander.

It works by generating a standalone kernel loop and peeling out the prologs and epilogs.

Definition at line 283 of file ModuloSchedule.h.

Constructor & Destructor Documentation

◆ PeelingModuloScheduleExpander()

llvm::PeelingModuloScheduleExpander::PeelingModuloScheduleExpander ( MachineFunction MF,
ModuloSchedule S,
LiveIntervals LIS 
)
inline

Definition at line 285 of file ModuloSchedule.h.

Member Function Documentation

◆ CreateLCSSAExitingBlock()

MachineBasicBlock * PeelingModuloScheduleExpander::CreateLCSSAExitingBlock ( )
protected

◆ expand()

void PeelingModuloScheduleExpander::expand ( )

◆ filterInstructions()

void PeelingModuloScheduleExpander::filterInstructions ( MachineBasicBlock MB,
int  MinStage 
)
protected

◆ fixupBranches()

void PeelingModuloScheduleExpander::fixupBranches ( )
protected

◆ getEquivalentRegisterIn()

Register PeelingModuloScheduleExpander::getEquivalentRegisterIn ( Register  Reg,
MachineBasicBlock BB 
)
protected

All prolog and epilog blocks are clones of the kernel, so any produced register in one block has an corollary in all other blocks.

Definition at line 1899 of file ModuloSchedule.cpp.

References BB, BlockMIs, CanonicalMIs, llvm::MachineRegisterInfo::getUniqueVRegDef(), MI, and MRI.

Referenced by filterInstructions(), peelPrologAndEpilogs(), and rewriteUsesOf().

◆ getPhiCanonicalReg()

Register PeelingModuloScheduleExpander::getPhiCanonicalReg ( MachineInstr CanonicalPhi,
MachineInstr Phi 
)
protected

◆ getStage()

unsigned llvm::PeelingModuloScheduleExpander::getStage ( MachineInstr MI)
inlineprotected

Helper to get the stage of an instruction in the schedule.

Definition at line 361 of file ModuloSchedule.h.

References CanonicalMIs, llvm::ModuloSchedule::getStage(), MI, and Schedule.

Referenced by filterInstructions(), moveStageBetweenBlocks(), and rewriteUsesOf().

◆ moveStageBetweenBlocks()

void PeelingModuloScheduleExpander::moveStageBetweenBlocks ( MachineBasicBlock DestBB,
MachineBasicBlock SourceBB,
unsigned  Stage 
)
protected

◆ peelKernel()

MachineBasicBlock * PeelingModuloScheduleExpander::peelKernel ( LoopPeelDirection  LPD)
protected

Peels one iteration of the rewritten kernel (BB) in the specified direction.

Definition at line 1591 of file ModuloSchedule.cpp.

References BB, llvm::MachineBasicBlock::begin(), BlockMIs, CanonicalMIs, I, llvm::LPD_Front, MRI, PeeledBack, PeeledFront, llvm::PeelSingleBlockLoop(), and TII.

Referenced by peelPrologAndEpilogs().

◆ peelPrologAndEpilogs()

void PeelingModuloScheduleExpander::peelPrologAndEpilogs ( )
protected

◆ rewriteKernel()

void PeelingModuloScheduleExpander::rewriteKernel ( )
protected

Converts BB from the original loop body to the rewritten, pipelined steady-state.

Definition at line 1996 of file ModuloSchedule.cpp.

References BB, llvm::ModuloSchedule::getLoop(), and Schedule.

Referenced by expand().

◆ rewriteUsesOf()

void PeelingModuloScheduleExpander::rewriteUsesOf ( MachineInstr MI)
protected

◆ validateAgainstModuloScheduleExpander()

void PeelingModuloScheduleExpander::validateAgainstModuloScheduleExpander ( )

Member Data Documentation

◆ AvailableStages

DenseMap<MachineBasicBlock *, BitVector> llvm::PeelingModuloScheduleExpander::AvailableStages
protected

For every block, the stages that are available.

A stage can be available but not produced (in the epilog) or produced but not available (in the prolog).

Definition at line 315 of file ModuloSchedule.h.

Referenced by peelPrologAndEpilogs(), and rewriteUsesOf().

◆ BB

MachineBasicBlock* llvm::PeelingModuloScheduleExpander::BB = nullptr
protected

The original loop block that gets rewritten in-place.

Definition at line 305 of file ModuloSchedule.h.

Referenced by CreateLCSSAExitingBlock(), expand(), getEquivalentRegisterIn(), peelKernel(), peelPrologAndEpilogs(), rewriteKernel(), and validateAgainstModuloScheduleExpander().

◆ BlockMIs

DenseMap<std::pair<MachineBasicBlock *, MachineInstr *>, MachineInstr *> llvm::PeelingModuloScheduleExpander::BlockMIs
protected

◆ CanonicalMIs

DenseMap<MachineInstr *, MachineInstr *> llvm::PeelingModuloScheduleExpander::CanonicalMIs
protected

CanonicalMIs and BlockMIs form a bidirectional map between any of the loop kernel clones.

Definition at line 322 of file ModuloSchedule.h.

Referenced by CreateLCSSAExitingBlock(), getEquivalentRegisterIn(), getStage(), moveStageBetweenBlocks(), peelKernel(), and peelPrologAndEpilogs().

◆ Epilogs

SmallVector<MachineBasicBlock *, 4> llvm::PeelingModuloScheduleExpander::Epilogs
protected

Definition at line 309 of file ModuloSchedule.h.

Referenced by fixupBranches(), and peelPrologAndEpilogs().

◆ IllegalPhisToDelete

SmallVector<MachineInstr *, 4> llvm::PeelingModuloScheduleExpander::IllegalPhisToDelete
protected

Illegal phis that need to be deleted once we re-link stages.

Definition at line 329 of file ModuloSchedule.h.

Referenced by peelPrologAndEpilogs(), and rewriteUsesOf().

◆ LIS

LiveIntervals* llvm::PeelingModuloScheduleExpander::LIS = nullptr
protected

◆ LiveStages

DenseMap<MachineBasicBlock *, BitVector> llvm::PeelingModuloScheduleExpander::LiveStages
protected

For every block, the stages that are produced.

Definition at line 311 of file ModuloSchedule.h.

Referenced by peelPrologAndEpilogs(), and rewriteUsesOf().

◆ LoopInfo

std::unique_ptr<TargetInstrInfo::PipelinerLoopInfo> llvm::PeelingModuloScheduleExpander::LoopInfo
protected

Target loop info before kernel peeling.

Definition at line 370 of file ModuloSchedule.h.

◆ MF

MachineFunction& llvm::PeelingModuloScheduleExpander::MF
protected

◆ MRI

MachineRegisterInfo& llvm::PeelingModuloScheduleExpander::MRI
protected

◆ PeeledBack

std::deque<MachineBasicBlock *> llvm::PeelingModuloScheduleExpander::PeeledBack
protected

Definition at line 327 of file ModuloSchedule.h.

Referenced by peelKernel(), and peelPrologAndEpilogs().

◆ PeeledFront

std::deque<MachineBasicBlock *> llvm::PeelingModuloScheduleExpander::PeeledFront
protected

State passed from peelKernel to peelPrologAndEpilogs().

Definition at line 327 of file ModuloSchedule.h.

Referenced by peelKernel(), and peelPrologAndEpilogs().

◆ PhiNodeLoopIteration

DenseMap<MachineInstr *, unsigned> llvm::PeelingModuloScheduleExpander::PhiNodeLoopIteration
protected

When peeling the epilogue keep track of the distance between the phi nodes and the kernel.

Definition at line 318 of file ModuloSchedule.h.

Referenced by getPhiCanonicalReg(), moveStageBetweenBlocks(), and peelPrologAndEpilogs().

◆ Preheader

MachineBasicBlock* llvm::PeelingModuloScheduleExpander::Preheader = nullptr
protected

The original loop preheader.

Definition at line 307 of file ModuloSchedule.h.

Referenced by expand(), and validateAgainstModuloScheduleExpander().

◆ Prologs

SmallVector<MachineBasicBlock *, 4> llvm::PeelingModuloScheduleExpander::Prologs
protected

All prolog and epilog blocks.

Definition at line 309 of file ModuloSchedule.h.

Referenced by fixupBranches(), and peelPrologAndEpilogs().

◆ Schedule

ModuloSchedule& llvm::PeelingModuloScheduleExpander::Schedule
protected

◆ ST

const TargetSubtargetInfo& llvm::PeelingModuloScheduleExpander::ST
protected

Definition at line 299 of file ModuloSchedule.h.

◆ TII

const TargetInstrInfo* llvm::PeelingModuloScheduleExpander::TII = nullptr
protected

The documentation for this class was generated from the following files: