|
LLVM
4.0.0
|
Each Scheduling boundary is associated with ready queues. More...
#include <MachineScheduler.h>
Public Types | |
| enum | { TopQID = 1, BotQID = 2, LogMaxQID = 2 } |
| SUnit::NodeQueueId: 0 (none), 1 (top), 2 (bot), 3 (both) More... | |
Public Member Functions | |
| SchedBoundary (unsigned ID, const Twine &Name) | |
| Pending queues extend the ready queues with the same ID and the PendingFlag set. More... | |
| ~SchedBoundary () | |
| void | reset () |
| void | init (ScheduleDAGMI *dag, const TargetSchedModel *smodel, SchedRemainder *rem) |
| bool | isTop () const |
| unsigned | getCurrCycle () const |
| Number of cycles to issue the instructions scheduled in this zone. More... | |
| unsigned | getCurrMOps () const |
| Micro-ops issued in the current cycle. More... | |
| unsigned | getDependentLatency () const |
| unsigned | getScheduledLatency () const |
| Get the number of latency cycles "covered" by the scheduled instructions. More... | |
| unsigned | getUnscheduledLatency (SUnit *SU) const |
| unsigned | getResourceCount (unsigned ResIdx) const |
| unsigned | getCriticalCount () const |
| Get the scaled count of scheduled micro-ops and resources, including executed resources. More... | |
| unsigned | getExecutedCount () const |
| Get a scaled count for the minimum execution time of the scheduled micro-ops that are ready to execute by getExecutedCount. More... | |
| unsigned | getZoneCritResIdx () const |
| bool | isResourceLimited () const |
| unsigned | getLatencyStallCycles (SUnit *SU) |
| Get the difference between the given SUnit's ready time and the current cycle. More... | |
| unsigned | getNextResourceCycle (unsigned PIdx, unsigned Cycles) |
| Compute the next cycle at which the given processor resource can be scheduled. More... | |
| bool | checkHazard (SUnit *SU) |
| Does this SU have a hazard within the current instruction group. More... | |
| unsigned | findMaxLatency (ArrayRef< SUnit * > ReadySUs) |
| unsigned | getOtherResourceCount (unsigned &OtherCritIdx) |
| void | releaseNode (SUnit *SU, unsigned ReadyCycle) |
| void | bumpCycle (unsigned NextCycle) |
| Move the boundary of scheduled code by one cycle. More... | |
| void | incExecutedResources (unsigned PIdx, unsigned Count) |
| unsigned | countResource (unsigned PIdx, unsigned Cycles, unsigned ReadyCycle) |
| Add the given processor resource to this scheduled zone. More... | |
| void | bumpNode (SUnit *SU) |
| Move the boundary of scheduled code by one SUnit. More... | |
| void | releasePending () |
| Release pending ready nodes in to the available queue. More... | |
| void | removeReady (SUnit *SU) |
| Remove SU from the ready set for this boundary. More... | |
| SUnit * | pickOnlyChoice () |
| Call this before applying any other heuristics to the Available queue. More... | |
| void | dumpScheduledState () |
Public Attributes | |
| ScheduleDAGMI * | DAG |
| const TargetSchedModel * | SchedModel |
| SchedRemainder * | Rem |
| ReadyQueue | Available |
| ReadyQueue | Pending |
| ScheduleHazardRecognizer * | HazardRec |
Each Scheduling boundary is associated with ready queues.
It tracks the current cycle in the direction of movement, and maintains the state of "hazards" and other interlocks at the current cycle.
Definition at line 592 of file MachineScheduler.h.
| anonymous enum |
SUnit::NodeQueueId: 0 (none), 1 (top), 2 (bot), 3 (both)
| Enumerator | |
|---|---|
| TopQID | |
| BotQID | |
| LogMaxQID | |
Definition at line 595 of file MachineScheduler.h.
Pending queues extend the ready queues with the same ID and the PendingFlag set.
Definition at line 667 of file MachineScheduler.h.
References reset().
| SchedBoundary::~SchedBoundary | ( | ) |
Definition at line 1824 of file MachineScheduler.cpp.
References HazardRec.
| void SchedBoundary::bumpCycle | ( | unsigned | NextCycle | ) |
Move the boundary of scheduled code by one cycle.
Definition at line 2040 of file MachineScheduler.cpp.
References llvm::ScheduleHazardRecognizer::AdvanceCycle(), assert(), Available, llvm::dbgs(), DEBUG, getCriticalCount(), llvm::TargetSchedModel::getIssueWidth(), llvm::TargetSchedModel::getLatencyFactor(), llvm::TargetSchedModel::getMicroOpBufferSize(), llvm::ReadyQueue::getName(), getScheduledLatency(), HazardRec, llvm::ScheduleHazardRecognizer::isEnabled(), isTop(), llvm::ScheduleHazardRecognizer::RecedeCycle(), and SchedModel.
Referenced by bumpNode(), and pickOnlyChoice().
| void SchedBoundary::bumpNode | ( | SUnit * | SU | ) |
Move the boundary of scheduled code by one SUnit.
Definition at line 2121 of file MachineScheduler.cpp.
References assert(), Available, llvm::SUnit::BotReadyCycle, llvm::MCProcResourceDesc::BufferSize, bumpCycle(), countResource(), llvm::dbgs(), DEBUG, dumpScheduledState(), llvm::ScheduleHazardRecognizer::EmitInstruction(), getCriticalCount(), llvm::SUnit::getDepth(), llvm::SUnit::getHeight(), llvm::SUnit::getInstr(), llvm::TargetSchedModel::getIssueWidth(), llvm::TargetSchedModel::getLatencyFactor(), llvm::TargetSchedModel::getMicroOpBufferSize(), llvm::TargetSchedModel::getMicroOpFactor(), llvm::ReadyQueue::getName(), getNextResourceCycle(), llvm::TargetSchedModel::getNumMicroOps(), llvm::TargetSchedModel::getProcResource(), getResourceCount(), llvm::ScheduleDAGInstrs::getSchedClass(), getScheduledLatency(), llvm::TargetSchedModel::getWriteProcResBegin(), llvm::TargetSchedModel::getWriteProcResEnd(), llvm::TargetSchedModel::hasInstrSchedModel(), llvm::SUnit::hasReservedResource, HazardRec, llvm::SUnit::isCall, llvm::ScheduleHazardRecognizer::isEnabled(), isTop(), llvm::SUnit::isUnbuffered, llvm::SUnit::NodeNum, Rem, llvm::SchedRemainder::RemIssueCount, llvm::ScheduleHazardRecognizer::Reset(), llvm::PPCISD::SC, SchedModel, and llvm::SUnit::TopReadyCycle.
Does this SU have a hazard within the current instruction group.
The scheduler supports two modes of hazard recognition. The first is the ScheduleHazardRecognizer API. It is a fully general hazard recognizer that supports highly complicated in-order reservation tables (ScoreboardHazardRecognizer) and arbitraty target-specific logic.
The second is a streamlined mechanism that checks for hazards based on simple counters that the scheduler itself maintains. It explicitly checks for instruction dispatch limitations, including the number of micro-ops that can dispatch per cycle.
TODO: Also check whether the SU must start a new group.
Definition at line 1935 of file MachineScheduler.cpp.
References llvm::dbgs(), DEBUG, llvm::ScheduleHazardRecognizer::getHazardType(), llvm::SUnit::getInstr(), llvm::TargetSchedModel::getIssueWidth(), getNextResourceCycle(), llvm::TargetSchedModel::getNumMicroOps(), llvm::TargetSchedModel::getResourceName(), llvm::ScheduleDAGInstrs::getSchedClass(), llvm::TargetSchedModel::getWriteProcResBegin(), llvm::TargetSchedModel::getWriteProcResEnd(), llvm::TargetSchedModel::hasInstrSchedModel(), llvm::SUnit::hasReservedResource, HazardRec, llvm::ScheduleHazardRecognizer::isEnabled(), llvm::SUnit::NodeNum, llvm::ScheduleHazardRecognizer::NoHazard, llvm::PPCISD::SC, and SchedModel.
Referenced by pickOnlyChoice(), releaseNode(), and releasePending().
Add the given processor resource to this scheduled zone.
| Cycles | indicates the number of consecutive (non-pipelined) cycles during which this resource is consumed. |
Definition at line 2091 of file MachineScheduler.cpp.
References assert(), llvm::dbgs(), DEBUG, getCriticalCount(), llvm::TargetSchedModel::getLatencyFactor(), getNextResourceCycle(), llvm::TargetSchedModel::getProcResource(), getResourceCount(), llvm::TargetSchedModel::getResourceFactor(), llvm::TargetSchedModel::getResourceName(), incExecutedResources(), llvm::MCProcResourceDesc::Name, Rem, llvm::SchedRemainder::RemainingCounts, and SchedModel.
Referenced by bumpNode().
| void SchedBoundary::dumpScheduledState | ( | ) |
Definition at line 2329 of file MachineScheduler.cpp.
References Available, llvm::dbgs(), getExecutedCount(), llvm::TargetSchedModel::getLatencyFactor(), llvm::TargetSchedModel::getMicroOpFactor(), llvm::ReadyQueue::getName(), getResourceCount(), llvm::TargetSchedModel::getResourceFactor(), llvm::TargetSchedModel::getResourceName(), and SchedModel.
Referenced by bumpNode().
Definition at line 1968 of file MachineScheduler.cpp.
References Available, llvm::ArrayRef< T >::begin(), llvm::dbgs(), DEBUG, E, llvm::ArrayRef< T >::end(), llvm::ReadyQueue::getName(), getUnscheduledLatency(), I, L, and llvm::SUnit::NodeNum.
Referenced by llvm::GenericSchedulerBase::setPolicy().
|
inline |
Get the scaled count of scheduled micro-ops and resources, including executed resources.
Definition at line 711 of file MachineScheduler.h.
References llvm::TargetSchedModel::getMicroOpFactor(), getResourceCount(), and SchedModel.
Referenced by bumpCycle(), bumpNode(), and countResource().
|
inline |
Number of cycles to issue the instructions scheduled in this zone.
Definition at line 686 of file MachineScheduler.h.
Referenced by llvm::GenericSchedulerBase::setPolicy().
|
inline |
Micro-ops issued in the current cycle.
Definition at line 689 of file MachineScheduler.h.
Referenced by llvm::GenericScheduler::tryCandidate().
|
inline |
Definition at line 692 of file MachineScheduler.h.
Referenced by llvm::GenericSchedulerBase::setPolicy().
|
inline |
Get a scaled count for the minimum execution time of the scheduled micro-ops that are ready to execute by getExecutedCount.
Notice the feedback loop.
Definition at line 720 of file MachineScheduler.h.
References llvm::TargetSchedModel::getLatencyFactor(), and SchedModel.
Referenced by dumpScheduledState().
Get the difference between the given SUnit's ready time and the current cycle.
Compute the stall cycles based on this SUnit's ready time.
Heuristics treat these "soft stalls" differently than the hard stall cycles based on CPU resources and computed by checkHazard(). A fully in-order model (MicroOpBufferSize==0) will not make use of this since instructions are not available for scheduling until they are ready. However, a weaker in-order model may use this for heuristics. For example, if a processor has in-order behavior when reading certain resources, this may come into play.
Definition at line 1898 of file MachineScheduler.cpp.
References llvm::SUnit::BotReadyCycle, isTop(), llvm::SUnit::isUnbuffered, and llvm::SUnit::TopReadyCycle.
Referenced by llvm::GenericScheduler::tryCandidate().
Compute the next cycle at which the given processor resource can be scheduled.
Definition at line 1911 of file MachineScheduler.cpp.
References InvalidCycle, and isTop().
Referenced by bumpNode(), checkHazard(), and countResource().
Definition at line 1990 of file MachineScheduler.cpp.
References Available, llvm::dbgs(), DEBUG, llvm::TargetSchedModel::getMicroOpFactor(), llvm::ReadyQueue::getName(), llvm::TargetSchedModel::getNumProcResourceKinds(), getResourceCount(), llvm::TargetSchedModel::getResourceFactor(), llvm::TargetSchedModel::getResourceName(), llvm::TargetSchedModel::hasInstrSchedModel(), Rem, llvm::SchedRemainder::RemainingCounts, llvm::SchedRemainder::RemIssueCount, and SchedModel.
Referenced by llvm::GenericSchedulerBase::setPolicy().
Definition at line 705 of file MachineScheduler.h.
Referenced by bumpNode(), countResource(), dumpScheduledState(), getCriticalCount(), and getOtherResourceCount().
|
inline |
Get the number of latency cycles "covered" by the scheduled instructions.
This is the larger of the critical path within the zone and the number of cycles required to issue the instructions.
Definition at line 697 of file MachineScheduler.h.
Referenced by bumpCycle(), bumpNode(), and tryLatency().
Definition at line 701 of file MachineScheduler.h.
References llvm::SUnit::getDepth(), llvm::SUnit::getHeight(), and isTop().
Referenced by findMaxLatency().
|
inline |
Definition at line 725 of file MachineScheduler.h.
Referenced by llvm::GenericSchedulerBase::setPolicy().
Definition at line 2077 of file MachineScheduler.cpp.
Referenced by countResource().
| void SchedBoundary::init | ( | ScheduleDAGMI * | dag, |
| const TargetSchedModel * | smodel, | ||
| SchedRemainder * | rem | ||
| ) |
Definition at line 1880 of file MachineScheduler.cpp.
References llvm::TargetSchedModel::getNumProcResourceKinds(), llvm::TargetSchedModel::hasInstrSchedModel(), InvalidCycle, Rem, reset(), llvm::SmallVectorImpl< T >::resize(), and SchedModel.
|
inline |
Definition at line 728 of file MachineScheduler.h.
Referenced by llvm::GenericSchedulerBase::setPolicy().
|
inline |
Definition at line 681 of file MachineScheduler.h.
References Available, llvm::ReadyQueue::getID(), and TopQID.
Referenced by bumpCycle(), bumpNode(), getLatencyStallCycles(), getNextResourceCycle(), getUnscheduledLatency(), llvm::GenericScheduler::pickNodeFromQueue(), releasePending(), llvm::GenericScheduler::tryCandidate(), and tryLatency().
| SUnit * SchedBoundary::pickOnlyChoice | ( | ) |
Call this before applying any other heuristics to the Available queue.
If this queue only has one ready candidate, return it.
Updates the Available/Pending Q's if necessary and returns the single available instruction, or NULL if there are multiple candidates.
As a side effect, defer any nodes that now hit a hazard, and advance the cycle until at least one node is ready. If multiple instructions are ready, return NULL.
Definition at line 2294 of file MachineScheduler.cpp.
References Available, llvm::ReadyQueue::begin(), bumpCycle(), checkHazard(), DEBUG, llvm::ReadyQueue::dump(), llvm::ReadyQueue::empty(), llvm::ReadyQueue::end(), I, i, Pending, llvm::ReadyQueue::push(), releasePending(), llvm::ReadyQueue::remove(), and llvm::ReadyQueue::size().
Referenced by llvm::GCNMaxOccupancySchedStrategy::pickNode().
Definition at line 2015 of file MachineScheduler.cpp.
References assert(), Available, checkHazard(), llvm::SUnit::getInstr(), llvm::TargetSchedModel::getMicroOpBufferSize(), Pending, llvm::ReadyQueue::push(), ReadyListLimit, SchedModel, and llvm::ReadyQueue::size().
Referenced by llvm::GenericScheduler::releaseBottomNode(), llvm::GenericScheduler::releaseTopNode(), and llvm::PostGenericScheduler::releaseTopNode().
| void SchedBoundary::releasePending | ( | ) |
Release pending ready nodes in to the available queue.
This makes them visible to heuristics.
Definition at line 2250 of file MachineScheduler.cpp.
References Available, llvm::ReadyQueue::begin(), llvm::SUnit::BotReadyCycle, checkHazard(), llvm::ReadyQueue::empty(), llvm::TargetSchedModel::getMicroOpBufferSize(), i, isTop(), Pending, llvm::ReadyQueue::push(), ReadyListLimit, llvm::ReadyQueue::remove(), SchedModel, llvm::ReadyQueue::size(), and llvm::SUnit::TopReadyCycle.
Referenced by pickOnlyChoice().
| void SchedBoundary::removeReady | ( | SUnit * | SU | ) |
Remove SU from the ready set for this boundary.
Definition at line 2282 of file MachineScheduler.cpp.
References assert(), Available, llvm::ReadyQueue::find(), llvm::ReadyQueue::isInQueue(), Pending, and llvm::ReadyQueue::remove().
Referenced by llvm::GCNMaxOccupancySchedStrategy::pickNode().
| void SchedBoundary::reset | ( | ) |
Definition at line 1826 of file MachineScheduler.cpp.
References assert(), Available, llvm::SmallVectorImpl< T >::clear(), llvm::ReadyQueue::clear(), HazardRec, llvm::ScheduleHazardRecognizer::isEnabled(), Pending, and llvm::SmallVectorImpl< T >::resize().
Referenced by init(), and SchedBoundary().
| ReadyQueue llvm::SchedBoundary::Available |
Definition at line 605 of file MachineScheduler.h.
Referenced by bumpCycle(), bumpNode(), dumpScheduledState(), findMaxLatency(), getOtherResourceCount(), isTop(), llvm::GCNMaxOccupancySchedStrategy::pickNode(), llvm::GenericScheduler::pickNodeFromQueue(), pickOnlyChoice(), releaseNode(), releasePending(), removeReady(), reset(), and llvm::GenericSchedulerBase::setPolicy().
| ScheduleDAGMI* llvm::SchedBoundary::DAG |
Definition at line 601 of file MachineScheduler.h.
| ScheduleHazardRecognizer* llvm::SchedBoundary::HazardRec |
Definition at line 608 of file MachineScheduler.h.
Referenced by bumpCycle(), bumpNode(), checkHazard(), reset(), and ~SchedBoundary().
| ReadyQueue llvm::SchedBoundary::Pending |
Definition at line 606 of file MachineScheduler.h.
Referenced by llvm::GCNMaxOccupancySchedStrategy::pickNode(), pickOnlyChoice(), releaseNode(), releasePending(), removeReady(), reset(), and llvm::GenericSchedulerBase::setPolicy().
| SchedRemainder* llvm::SchedBoundary::Rem |
Definition at line 603 of file MachineScheduler.h.
Referenced by bumpNode(), countResource(), getOtherResourceCount(), and init().
| const TargetSchedModel* llvm::SchedBoundary::SchedModel |
Definition at line 602 of file MachineScheduler.h.
Referenced by bumpCycle(), bumpNode(), checkHazard(), countResource(), dumpScheduledState(), getCriticalCount(), getExecutedCount(), getOtherResourceCount(), init(), releaseNode(), and releasePending().
1.8.6