|
LLVM 24.0.0git
|
The main class in the implementation of the target independent window scheduler. More...
#include "llvm/CodeGen/WindowScheduler.h"
Public Member Functions | |
| WindowScheduler (MachineSchedContext *C, MachineLoop &ML) | |
| virtual | ~WindowScheduler ()=default |
| bool | run () |
Protected Member Functions | |
| virtual ScheduleDAGInstrs * | createMachineScheduler (bool OnlyBuildGraph=false) |
| Two types of ScheduleDAGs are needed, one for creating dependency graphs only, and the other for list scheduling as determined by the target. | |
| virtual bool | initialize () |
| Initializes the algorithm and determines if it can be executed. | |
| virtual void | preProcess () |
| Add some related processing before running window scheduling. | |
| void | collectLoopCarriedMemDeps () |
| Detect loop-carried memory dependences in the original MBB that must be preserved across loop iterations. | |
| virtual void | postProcess () |
| Add some related processing after running window scheduling. | |
| void | backupMBB () |
| Back up the MIs in the original MBB and remove them from MBB. | |
| void | restoreMBB () |
| Erase the MIs in current MBB and restore the original MIs. | |
| virtual void | generateTripleMBB () |
| Make three copies of the original MBB to generate a new TripleMBB. | |
| virtual void | restoreTripleMBB () |
| Restore the order of MIs in TripleMBB after each list scheduling. | |
| virtual SmallVector< unsigned > | getSearchIndexes (unsigned SearchNum, unsigned SearchRatio) |
| Give the folding position in the window algorithm, where different heuristics can be used. | |
| virtual int | calculateMaxCycle (ScheduleDAGInstrs &DAG, unsigned Offset) |
| Calculate MIs execution cycle after list scheduling. | |
| virtual int | calculateStallCycle (unsigned Offset, int MaxCycle) |
| Calculate the stall cycle between two trips after list scheduling. | |
| virtual unsigned | analyseII (ScheduleDAGInstrs &DAG, unsigned Offset) |
| Analyzes the II value after each list scheduling. | |
| virtual void | schedulePhi (int Offset, unsigned &II) |
| Phis are scheduled separately after each list scheduling. | |
| DenseMap< MachineInstr *, int > | getIssueOrder (unsigned Offset, unsigned II) |
| Get the final issue order of all scheduled MIs including phis. | |
| virtual void | updateScheduleResult (unsigned Offset, unsigned II) |
| Update the scheduling result after each list scheduling. | |
| virtual bool | isScheduleValid () |
| Check whether the final result of window scheduling is valid. | |
| virtual void | expand () |
| Using the scheduling infrastructure to expand the results of window scheduling. | |
| virtual void | updateLiveIntervals () |
| Update the live intervals for all registers used within MBB. | |
| int | getEstimatedII (ScheduleDAGInstrs &DAG) |
| Estimate a II value at which all MIs will be scheduled successfully. | |
| iterator_range< MachineBasicBlock::iterator > | getScheduleRange (unsigned Offset, unsigned Num) |
| Gets the iterator range of MIs in the scheduling window. | |
| int | getOriCycle (MachineInstr *NewMI) |
| Get the issue cycle of the new MI based on the cycle of the original MI. | |
| MachineInstr * | getOriMI (MachineInstr *NewMI) |
| Get the original MI from which the new MI is cloned. | |
| unsigned | getOriStage (MachineInstr *OriMI, unsigned Offset) |
| Get the scheduling stage, where the stage of the new MI is identical to the original MI. | |
| Register | getAntiRegister (MachineInstr *Phi) |
| Gets the register in phi which is generated from the current MBB. | |
Protected Attributes | |
| MachineSchedContext * | Context = nullptr |
| MachineFunction * | MF = nullptr |
| MachineBasicBlock * | MBB = nullptr |
| MachineLoop & | Loop |
| const TargetSubtargetInfo * | Subtarget = nullptr |
| const TargetInstrInfo * | TII = nullptr |
| const TargetRegisterInfo * | TRI = nullptr |
| MachineRegisterInfo * | MRI = nullptr |
| std::unique_ptr< ScheduleDAGInstrs > | TripleDAG |
| To innovatively identify the dependencies between MIs across two trips, we construct a DAG for a new MBB, which is created by copying the original MBB three times. | |
| SmallVector< MachineInstr * > | OriMIs |
| OriMIs keeps the MIs removed from the original MBB. | |
| SmallVector< MachineInstr * > | TriMIs |
| TriMIs keeps the MIs of TripleMBB, which is used to restore TripleMBB. | |
| DenseMap< MachineInstr *, TriCloneInfo > | TriToOri |
| TriToOri keeps the mappings between the MI clones in TripleMBB and the information about their original MI and copy index. | |
| DenseMap< MachineInstr *, int > | OriToCycle |
| OriToCycle keeps the mappings between the original MI and its issue cycle. | |
| DenseSet< std::pair< MachineInstr *, MachineInstr * > > | LoopCarriedMemDeps |
| LoopCarriedMemDeps keeps ordered pairs (Pred, Succ) of original memory MIs such that the iteration-N access Pred must be ordered before the iteration-(N+1) access Succ. | |
| bool | MemDepEdgeFailed = false |
| MemDepEdgeFailed is set when enforcing a loop-carried memory dependence as an artificial DAG edge would have created a cycle, so the edge could not be added. | |
| SmallVector< std::tuple< MachineInstr *, int, int, int >, 256 > | SchedResult |
| SchedResult keeps the result of each list scheduling, and the format of the tuple is <MI pointer, Cycle, Stage, Order ID>. | |
| unsigned | SchedPhiNum = 0 |
| SchedPhiNum records the number of phi in the original MBB, and the scheduling starts with MI after phis. | |
| unsigned | SchedInstrNum = 0 |
| SchedInstrNum records the MIs involved in scheduling in the original MBB, excluding debug instructions. | |
| unsigned | BestII = UINT_MAX |
| BestII and BestOffset record the characteristics of the best scheduling result and are used together with SchedResult as the final window scheduling result. | |
| unsigned | BestOffset = 0 |
| unsigned | BaseII = 0 |
| BaseII is the II obtained when the window offset is SchedPhiNum. | |
The main class in the implementation of the target independent window scheduler.
Definition at line 69 of file WindowScheduler.h.
| WindowScheduler::WindowScheduler | ( | MachineSchedContext * | C, |
| MachineLoop & | ML ) |
|
virtualdefault |
References analyseII(), backupMBB(), calculateMaxCycle(), calculateStallCycle(), collectLoopCarriedMemDeps(), createMachineScheduler(), generateTripleMBB(), getIssueOrder(), getSearchIndexes(), II, llvm::Offset, postProcess(), preProcess(), restoreMBB(), restoreTripleMBB(), run(), schedulePhi(), and updateScheduleResult().
|
protectedvirtual |
Analyzes the II value after each list scheduling.
Definition at line 711 of file WindowScheduler.cpp.
References calculateMaxCycle(), calculateStallCycle(), llvm::dbgs(), LLVM_DEBUG, llvm::Offset, and WindowIILimit.
Referenced by run(), and ~WindowScheduler().
|
protected |
Back up the MIs in the original MBB and remove them from MBB.
Definition at line 459 of file WindowScheduler.cpp.
References Context, llvm::make_early_inc_range(), MBB, MI, and OriMIs.
Referenced by preProcess(), and ~WindowScheduler().
|
protectedvirtual |
Calculate MIs execution cycle after list scheduling.
Definition at line 612 of file WindowScheduler.cpp.
References llvm::dbgs(), getEstimatedII(), getOriCycle(), getOriMI(), getOriStage(), getScheduleRange(), llvm::ScheduleDAGInstrs::getSUnit(), LLVM_DEBUG, MI, llvm::Offset, OriToCycle, Range, SchedInstrNum, Subtarget, TII, and WindowIILimit.
Referenced by analyseII(), and ~WindowScheduler().
|
protectedvirtual |
Calculate the stall cycle between two trips after list scheduling.
Definition at line 682 of file WindowScheduler.cpp.
References llvm::dbgs(), getOriCycle(), getScheduleRange(), LLVM_DEBUG, MI, llvm::Offset, Range, SchedInstrNum, TripleDAG, and WindowIILimit.
Referenced by analyseII(), and ~WindowScheduler().
|
protected |
Detect loop-carried memory dependences in the original MBB that must be preserved across loop iterations.
This must run on the live MBB before backupMBB(), as it analyzes the induction recurrence of memory accesses. The hazardous pairs of original MIs are recorded in LoopCarriedMemDeps.
Definition at line 419 of file WindowScheduler.cpp.
References Context, llvm::BatchAAResults::enableCrossIterationMode(), hasLoopCarriedMemDep(), LoopCarriedMemDeps, MBB, MI, MRI, llvm::none_of(), llvm::SmallVectorTemplateBase< T, bool >::push_back(), and TII.
Referenced by preProcess(), and ~WindowScheduler().
|
protectedvirtual |
Two types of ScheduleDAGs are needed, one for creating dependency graphs only, and the other for list scheduling as determined by the target.
Definition at line 321 of file WindowScheduler.cpp.
References Context.
Referenced by run(), WindowScheduler(), and ~WindowScheduler().
|
protectedvirtual |
Using the scheduling infrastructure to expand the results of window scheduling.
It is usually necessary to add prologue and epilogue MBBs.
Definition at line 810 of file WindowScheduler.cpp.
References A(), B(), llvm::ModuloScheduleExpander::cleanup(), Context, llvm::dbgs(), llvm::ModuloScheduleExpander::expand(), LLVM_DEBUG, Loop, MF, MI, SchedResult, and llvm::stable_sort().
Referenced by run().
|
protectedvirtual |
Make three copies of the original MBB to generate a new TripleMBB.
Definition at line 480 of file WindowScheduler.cpp.
References assert(), getAntiRegister(), MBB, MF, MI, MRI, OriMIs, TRI, TriMIs, TriToOri, and updateLiveIntervals().
Referenced by preProcess(), and ~WindowScheduler().
|
protected |
Gets the register in phi which is generated from the current MBB.
Definition at line 889 of file WindowScheduler.cpp.
Referenced by generateTripleMBB(), and schedulePhi().
|
protected |
Estimate a II value at which all MIs will be scheduled successfully.
Definition at line 604 of file WindowScheduler.cpp.
References llvm::ScheduleDAG::SUnits.
Referenced by calculateMaxCycle().
|
protected |
Get the final issue order of all scheduled MIs including phis.
Definition at line 759 of file WindowScheduler.cpp.
References llvm::DenseMapBase< StorageT, KeyT, ValueT, KeyInfoT, BucketT >::end(), llvm::DenseMapBase< StorageT, KeyT, ValueT, KeyInfoT, BucketT >::find(), getOriCycle(), getOriMI(), getScheduleRange(), II, MBB, MI, llvm::Offset, Range, and SchedInstrNum.
Referenced by updateScheduleResult(), and ~WindowScheduler().
|
protected |
Get the issue cycle of the new MI based on the cycle of the original MI.
Definition at line 859 of file WindowScheduler.cpp.
References assert(), OriToCycle, and TriToOri.
Referenced by calculateMaxCycle(), calculateStallCycle(), getIssueOrder(), and schedulePhi().
|
protected |
Get the original MI from which the new MI is cloned.
Definition at line 866 of file WindowScheduler.cpp.
References assert(), and TriToOri.
Referenced by calculateMaxCycle(), getIssueOrder(), and schedulePhi().
|
protected |
Get the scheduling stage, where the stage of the new MI is identical to the original MI.
Definition at line 871 of file WindowScheduler.cpp.
References assert(), llvm::is_contained(), MI, llvm::Offset, OriMIs, and SchedPhiNum.
Referenced by calculateMaxCycle(), schedulePhi(), and updateScheduleResult().
|
protected |
Gets the iterator range of MIs in the scheduling window.
Definition at line 851 of file WindowScheduler.cpp.
References llvm::make_range(), MBB, and llvm::Offset.
Referenced by calculateMaxCycle(), calculateStallCycle(), getIssueOrder(), and run().
|
protectedvirtual |
Give the folding position in the window algorithm, where different heuristics can be used.
It determines the performance and compilation time of the algorithm.
Definition at line 589 of file WindowScheduler.cpp.
References assert(), llvm::SmallVectorTemplateBase< T, bool >::push_back(), and SchedInstrNum.
Referenced by run(), and ~WindowScheduler().
|
protectedvirtual |
Initializes the algorithm and determines if it can be executed.
Definition at line 329 of file WindowScheduler.cpp.
References BaseII, BestII, BestOffset, Context, llvm::SmallSet< T, N, C >::count(), llvm::dbgs(), I, llvm::SmallSet< T, N, C >::insert(), LLVM_DEBUG, MBB, MF, MI, OriMIs, OriToCycle, SchedInstrNum, SchedPhiNum, SchedResult, Subtarget, TII, TriMIs, and TriToOri.
Referenced by run().
|
inlineprotectedvirtual |
Check whether the final result of window scheduling is valid.
Definition at line 171 of file WindowScheduler.h.
References BestOffset, and SchedPhiNum.
Referenced by run().
|
protectedvirtual |
Add some related processing after running window scheduling.
Definition at line 451 of file WindowScheduler.cpp.
References restoreMBB(), and TripleDAG.
Referenced by run(), and ~WindowScheduler().
|
protectedvirtual |
Add some related processing before running window scheduling.
Definition at line 404 of file WindowScheduler.cpp.
References backupMBB(), collectLoopCarriedMemDeps(), Context, generateTripleMBB(), MBB, and TripleDAG.
Referenced by run(), and ~WindowScheduler().
|
protected |
Erase the MIs in current MBB and restore the original MIs.
Definition at line 469 of file WindowScheduler.cpp.
References llvm::append_range(), Context, llvm::make_early_inc_range(), MBB, MI, OriMIs, and updateLiveIntervals().
Referenced by postProcess(), and ~WindowScheduler().
|
protectedvirtual |
Restore the order of MIs in TripleMBB after each list scheduling.
Definition at line 575 of file WindowScheduler.cpp.
References Context, I, MBB, MI, and TriMIs.
Referenced by run(), and ~WindowScheduler().
| bool WindowScheduler::run | ( | ) |
Definition at line 246 of file WindowScheduler.cpp.
References llvm::ScheduleDAGMI::addMutation(), analyseII(), assert(), BestII, BestOffset, Context, createMachineScheduler(), llvm::createSchedLive(), llvm::dbgs(), expand(), getScheduleRange(), getSearchIndexes(), II, initialize(), isScheduleValid(), LLVM_DEBUG, LoopCarriedMemDeps, MBB, MemDepEdgeFailed, llvm::Offset, OriToCycle, postProcess(), preProcess(), Range, restoreTripleMBB(), SchedInstrNum, SchedPhiNum, schedulePhi(), TriToOri, updateScheduleResult(), and WindowIILimit.
Referenced by ~WindowScheduler().
|
protectedvirtual |
Phis are scheduled separately after each list scheduling.
Definition at line 723 of file WindowScheduler.cpp.
References llvm::SDep::Data, llvm::dbgs(), getAntiRegister(), getOriCycle(), getOriMI(), getOriStage(), II, LLVM_DEBUG, MBB, MRI, llvm::Offset, OriToCycle, and TripleDAG.
Referenced by run(), and ~WindowScheduler().
|
protectedvirtual |
Update the live intervals for all registers used within MBB.
Definition at line 837 of file WindowScheduler.cpp.
References Context, llvm::is_contained(), MBB, MI, and llvm::SmallVectorTemplateBase< T, bool >::push_back().
Referenced by generateTripleMBB(), and restoreMBB().
Update the scheduling result after each list scheduling.
Definition at line 783 of file WindowScheduler.cpp.
References assert(), BaseII, BestII, BestOffset, getIssueOrder(), getOriStage(), II, llvm::Offset, OriToCycle, SchedPhiNum, and SchedResult.
Referenced by run(), and ~WindowScheduler().
|
protected |
BaseII is the II obtained when the window offset is SchedPhiNum.
This offset is the initial position of the sliding window.
Definition at line 121 of file WindowScheduler.h.
Referenced by initialize(), and updateScheduleResult().
|
protected |
BestII and BestOffset record the characteristics of the best scheduling result and are used together with SchedResult as the final window scheduling result.
Definition at line 117 of file WindowScheduler.h.
Referenced by initialize(), run(), and updateScheduleResult().
|
protected |
Definition at line 118 of file WindowScheduler.h.
Referenced by initialize(), isScheduleValid(), run(), and updateScheduleResult().
|
protected |
Definition at line 71 of file WindowScheduler.h.
Referenced by backupMBB(), collectLoopCarriedMemDeps(), createMachineScheduler(), expand(), initialize(), preProcess(), restoreMBB(), restoreTripleMBB(), run(), updateLiveIntervals(), and WindowScheduler().
|
protected |
Definition at line 74 of file WindowScheduler.h.
Referenced by expand(), and WindowScheduler().
|
protected |
LoopCarriedMemDeps keeps ordered pairs (Pred, Succ) of original memory MIs such that the iteration-N access Pred must be ordered before the iteration-(N+1) access Succ.
Definition at line 99 of file WindowScheduler.h.
Referenced by collectLoopCarriedMemDeps(), and run().
|
protected |
Definition at line 73 of file WindowScheduler.h.
Referenced by backupMBB(), collectLoopCarriedMemDeps(), generateTripleMBB(), getAntiRegister(), getIssueOrder(), getScheduleRange(), initialize(), preProcess(), restoreMBB(), restoreTripleMBB(), run(), schedulePhi(), updateLiveIntervals(), and WindowScheduler().
|
protected |
MemDepEdgeFailed is set when enforcing a loop-carried memory dependence as an artificial DAG edge would have created a cycle, so the edge could not be added.
In that case the current window must be abandoned, as its schedule would not honor the dependence.
Definition at line 104 of file WindowScheduler.h.
Referenced by run().
|
protected |
Definition at line 72 of file WindowScheduler.h.
Referenced by expand(), generateTripleMBB(), initialize(), and WindowScheduler().
|
protected |
Definition at line 78 of file WindowScheduler.h.
Referenced by collectLoopCarriedMemDeps(), generateTripleMBB(), schedulePhi(), and WindowScheduler().
|
protected |
OriMIs keeps the MIs removed from the original MBB.
Definition at line 88 of file WindowScheduler.h.
Referenced by backupMBB(), generateTripleMBB(), getOriStage(), initialize(), and restoreMBB().
|
protected |
OriToCycle keeps the mappings between the original MI and its issue cycle.
Definition at line 95 of file WindowScheduler.h.
Referenced by calculateMaxCycle(), getOriCycle(), initialize(), run(), schedulePhi(), and updateScheduleResult().
|
protected |
SchedInstrNum records the MIs involved in scheduling in the original MBB, excluding debug instructions.
Definition at line 113 of file WindowScheduler.h.
Referenced by calculateMaxCycle(), calculateStallCycle(), getIssueOrder(), getSearchIndexes(), initialize(), and run().
|
protected |
SchedPhiNum records the number of phi in the original MBB, and the scheduling starts with MI after phis.
Definition at line 110 of file WindowScheduler.h.
Referenced by getOriStage(), initialize(), isScheduleValid(), run(), and updateScheduleResult().
|
protected |
SchedResult keeps the result of each list scheduling, and the format of the tuple is <MI pointer, Cycle, Stage, Order ID>.
Definition at line 107 of file WindowScheduler.h.
Referenced by expand(), initialize(), and updateScheduleResult().
|
protected |
Definition at line 75 of file WindowScheduler.h.
Referenced by calculateMaxCycle(), initialize(), and WindowScheduler().
|
protected |
Definition at line 76 of file WindowScheduler.h.
Referenced by calculateMaxCycle(), collectLoopCarriedMemDeps(), initialize(), and WindowScheduler().
|
protected |
Definition at line 77 of file WindowScheduler.h.
Referenced by generateTripleMBB(), and WindowScheduler().
|
protected |
TriMIs keeps the MIs of TripleMBB, which is used to restore TripleMBB.
Definition at line 90 of file WindowScheduler.h.
Referenced by generateTripleMBB(), initialize(), and restoreTripleMBB().
|
protected |
To innovatively identify the dependencies between MIs across two trips, we construct a DAG for a new MBB, which is created by copying the original MBB three times.
We refer to this new MBB as 'TripleMBB' and the corresponding DAG as 'TripleDAG'. If the dependencies are more than two trips, we avoid applying window algorithm by identifying successive phis in the old MBB.
Definition at line 86 of file WindowScheduler.h.
Referenced by calculateStallCycle(), postProcess(), preProcess(), schedulePhi(), and WindowScheduler().
|
protected |
TriToOri keeps the mappings between the MI clones in TripleMBB and the information about their original MI and copy index.
Definition at line 93 of file WindowScheduler.h.
Referenced by generateTripleMBB(), getOriCycle(), getOriMI(), initialize(), and run().