#include <RegisterPressure.h>

Public Member Functions
	RegPressureTracker (IntervalPressure &rp)
	RegPressureTracker (RegionPressure &rp)
void	init (const MachineFunction mf, const RegisterClassInfo rci, const LiveIntervals lis, const MachineBasicBlock mbb, MachineBasicBlock::const_iterator pos)
void	addLiveRegs (ArrayRef< unsigned > Regs)
	Force liveness of registers.
MachineBasicBlock::const_iterator	getPos () const
	Get the MI position corresponding to this register pressure.
void	setPos (MachineBasicBlock::const_iterator Pos)
SlotIndex	getCurrSlot () const
	Get the SlotIndex for the first nondebug instruction including or after the current position.
bool	recede ()
	Recede across the previous instruction.
bool	advance ()
	Advance across the current instruction.
void	closeRegion ()
	Finalize the region boundaries and recored live ins and live outs.
RegisterPressure &	getPressure ()
const RegisterPressure &	getPressure () const
std::vector< unsigned > &	getRegSetPressureAtPos ()
void	discoverLiveOut (unsigned Reg)
	Add Reg to the live out set and increase max pressure.
void	discoverLiveIn (unsigned Reg)
	Add Reg to the live in set and increase max pressure.
bool	isTopClosed () const
	Does this pressure result have a valid top position and live ins.
bool	isBottomClosed () const
	Does this pressure result have a valid bottom position and live outs.
void	closeTop ()
	Set the boundary for the top of the region and summarize live ins.
void	closeBottom ()
	Set the boundary for the bottom of the region and summarize live outs.
void	getMaxUpwardPressureDelta (const MachineInstr *MI, RegPressureDelta &Delta, ArrayRef< PressureElement > CriticalPSets, ArrayRef< unsigned > MaxPressureLimit)
void	getMaxDownwardPressureDelta (const MachineInstr *MI, RegPressureDelta &Delta, ArrayRef< PressureElement > CriticalPSets, ArrayRef< unsigned > MaxPressureLimit)
void	getMaxPressureDelta (const MachineInstr *MI, RegPressureDelta &Delta, ArrayRef< PressureElement > CriticalPSets, ArrayRef< unsigned > MaxPressureLimit)
void	getUpwardPressure (const MachineInstr *MI, std::vector< unsigned > &PressureResult, std::vector< unsigned > &MaxPressureResult)
	Get the pressure of each PSet after traversing this instruction bottom-up.
void	getDownwardPressure (const MachineInstr *MI, std::vector< unsigned > &PressureResult, std::vector< unsigned > &MaxPressureResult)
	Get the pressure of each PSet after traversing this instruction top-down.
void	getPressureAfterInst (const MachineInstr *MI, std::vector< unsigned > &PressureResult, std::vector< unsigned > &MaxPressureResult)
void	dump () const
Protected Member Functions
const LiveInterval *	getInterval (unsigned Reg) const
void	increaseRegPressure (ArrayRef< unsigned > Regs)
void	decreaseRegPressure (ArrayRef< unsigned > Regs)
	Simply decrease the current pressure as impacted by these registers.
void	bumpUpwardPressure (const MachineInstr *MI)
void	bumpDownwardPressure (const MachineInstr *MI)

Detailed Description

Track the current register pressure at some position in the instruction stream, and remember the high water mark within the region traversed. This does not automatically consider live-through ranges. The client may independently adjust for global liveness.

Each RegPressureTracker only works within a MachineBasicBlock. Pressure can be tracked across a larger region by storing a RegisterPressure result at each block boundary and explicitly adjusting pressure to account for block live-in and live-out register sets.

RegPressureTracker holds a reference to a RegisterPressure result that it computes incrementally. During downward tracking, P.BottomIdx or P.BottomPos is invalid until it reaches the end of the block or closeRegion() is explicitly called. Similarly, P.TopIdx is invalid during upward tracking. Changing direction has the side effect of closing region, and traversing past TopIdx or BottomIdx reopens it.

Definition at line 169 of file RegisterPressure.h.

Constructor & Destructor Documentation

llvm::RegPressureTracker::RegPressureTracker ( IntervalPressure & rp ) [inline]

Definition at line 198 of file RegisterPressure.h.

llvm::RegPressureTracker::RegPressureTracker ( RegionPressure & rp ) [inline]

Definition at line 201 of file RegisterPressure.h.

Member Function Documentation

void RegPressureTracker::addLiveRegs ( ArrayRef< unsigned > Regs )

Force liveness of registers.

Force liveness of virtual registers or physical register units. Particularly useful to initialize the livein/out state of the tracker before the first call to advance/recede.

Definition at line 371 of file RegisterPressure.cpp.

References increaseRegPressure(), llvm::LiveRegSet::insert(), and llvm::ArrayRef< T >::size().

Referenced by llvm::ScheduleDAGMI::initRegPressure().

bool RegPressureTracker::advance ( )

Advance across the current instruction.

Definition at line 466 of file RegisterPressure.cpp.

References closeRegion(), closeTop(), collectOperands(), llvm::LiveRegSet::contains(), RegisterOperands::DeadDefs, decreaseRegPressure(), RegisterOperands::Defs, discoverLiveIn(), llvm::MachineBasicBlock::end(), llvm::LiveRegSet::erase(), getCurrSlot(), getInterval(), increaseRegPressure(), llvm::LiveRegSet::insert(), isBottomClosed(), isTopClosed(), llvm::TargetRegisterInfo::isVirtualRegister(), llvm::LiveInterval::killedAt(), LI, llvm::SmallVectorTemplateCommon< T, typename >::size(), and RegisterOperands::Uses.

Referenced by llvm::ScheduleDAGMI::scheduleMI().

void RegPressureTracker::bumpDownwardPressure ( const MachineInstr * MI ) [protected]

Record the downward impact of a single instruction on current register pressure. Unlike the advance/recede pressure tracking interface, this does not discover live in/outs.

This is intended for speculative queries. It leaves pressure inconsistent with the current position, so must be restored by the caller.

Definition at line 697 of file RegisterPressure.cpp.

References collectOperands(), decreaseRegPressure(), findUseBetween(), getCurrSlot(), llvm::LiveIntervals::getInstructionIndex(), getInterval(), llvm::SlotIndex::getRegSlot(), increaseRegPressure(), llvm::MachineInstr::isDebugValue(), llvm::TargetRegisterInfo::isVirtualRegister(), llvm::LiveInterval::killedAt(), and LI.

Referenced by getDownwardPressure(), and getMaxDownwardPressureDelta().

void RegPressureTracker::bumpUpwardPressure ( const MachineInstr * MI ) [protected]

Record the upward impact of a single instruction on current register pressure. Unlike the advance/recede pressure tracking interface, this does not discover live in/outs.

This is intended for speculative queries. It leaves pressure inconsistent with the current position, so must be restored by the caller.

Definition at line 613 of file RegisterPressure.cpp.

References collectOperands(), llvm::LiveRegSet::contains(), containsReg(), decreaseRegPressure(), increaseRegPressure(), and llvm::MachineInstr::isDebugValue().

Referenced by getMaxUpwardPressureDelta(), and getUpwardPressure().

void RegPressureTracker::closeBottom ( )

Set the boundary for the bottom of the region and summarize live outs.

Definition at line 276 of file RegisterPressure.cpp.

References llvm::SmallVectorImpl< T >::append(), llvm::SmallVectorTemplateCommon< T, typename >::begin(), llvm::SparseSet< ValueT, KeyFunctorT, SparseT >::begin(), llvm::SmallVectorBase::empty(), llvm::SmallVectorTemplateCommon< T, typename >::end(), llvm::SparseSet< ValueT, KeyFunctorT, SparseT >::end(), llvm::SmallVectorImpl< T >::erase(), getCurrSlot(), I, llvm::RegisterPressure::LiveOutRegs, llvm::LiveRegSet::PhysRegs, llvm::SmallVectorTemplateBase< T, isPodLike >::push_back(), llvm::SmallVectorImpl< T >::reserve(), llvm::SparseSet< ValueT, KeyFunctorT, SparseT >::size(), and llvm::LiveRegSet::VirtRegs.

Referenced by closeRegion(), llvm::ScheduleDAGMI::initRegPressure(), and recede().

void RegPressureTracker::closeRegion ( )

Finalize the region boundaries and recored live ins and live outs.

Finalize the region boundaries and record live ins and live outs.

Definition at line 294 of file RegisterPressure.cpp.

References closeBottom(), closeTop(), llvm::SparseSet< ValueT, KeyFunctorT, SparseT >::empty(), isBottomClosed(), isTopClosed(), llvm::LiveRegSet::PhysRegs, and llvm::LiveRegSet::VirtRegs.

Referenced by advance(), llvm::ScheduleDAGMI::initRegPressure(), and recede().

void RegPressureTracker::closeTop ( )

Set the boundary for the top of the region and summarize live ins.

Definition at line 258 of file RegisterPressure.cpp.

References llvm::SmallVectorImpl< T >::append(), llvm::SmallVectorTemplateCommon< T, typename >::begin(), llvm::SparseSet< ValueT, KeyFunctorT, SparseT >::begin(), llvm::SmallVectorBase::empty(), llvm::SmallVectorTemplateCommon< T, typename >::end(), llvm::SparseSet< ValueT, KeyFunctorT, SparseT >::end(), llvm::SmallVectorImpl< T >::erase(), getCurrSlot(), I, llvm::RegisterPressure::LiveInRegs, llvm::LiveRegSet::PhysRegs, llvm::SmallVectorTemplateBase< T, isPodLike >::push_back(), llvm::SmallVectorImpl< T >::reserve(), llvm::SparseSet< ValueT, KeyFunctorT, SparseT >::size(), and llvm::LiveRegSet::VirtRegs.

Referenced by advance(), closeRegion(), and llvm::ScheduleDAGMI::initRegPressure().

void RegPressureTracker::decreaseRegPressure ( ArrayRef< unsigned > Regs ) [protected]

Simply decrease the current pressure as impacted by these registers.

Definition at line 126 of file RegisterPressure.cpp.

References decreaseSetPressure(), llvm::MachineRegisterInfo::getRegClass(), llvm::TargetRegisterInfo::getRegClassPressureSets(), llvm::TargetRegisterInfo::getRegClassWeight(), llvm::TargetRegisterInfo::getRegUnitPressureSets(), llvm::TargetRegisterInfo::getRegUnitWeight(), I, llvm::TargetRegisterInfo::isVirtualRegister(), llvm::RegClassWeight::RegWeight, and llvm::ArrayRef< T >::size().

Referenced by advance(), bumpDownwardPressure(), bumpUpwardPressure(), and recede().

void RegPressureTracker::discoverLiveIn ( unsigned Reg )

Add Reg to the live in set and increase max pressure.

Definition at line 379 of file RegisterPressure.cpp.

References llvm::LiveRegSet::contains(), containsReg(), llvm::RegisterPressure::increase(), llvm::RegisterPressure::LiveInRegs, and llvm::SmallVectorTemplateBase< T, isPodLike >::push_back().

Referenced by advance().

void RegPressureTracker::discoverLiveOut ( unsigned Reg )

Add Reg to the live out set and increase max pressure.

Definition at line 390 of file RegisterPressure.cpp.

References llvm::LiveRegSet::contains(), containsReg(), llvm::RegisterPressure::increase(), llvm::RegisterPressure::LiveOutRegs, and llvm::SmallVectorTemplateBase< T, isPodLike >::push_back().

Referenced by recede().

void RegPressureTracker::dump ( ) const

Definition at line 100 of file RegisterPressure.cpp.

References llvm::dbgs(), llvm::RegisterPressure::dump(), and dumpSetPressure().

SlotIndex RegPressureTracker::getCurrSlot ( ) const

Get the SlotIndex for the first nondebug instruction including or after the current position.

Definition at line 248 of file RegisterPressure.cpp.

References llvm::MachineBasicBlock::end(), llvm::LiveIntervals::getInstructionIndex(), llvm::LiveIntervals::getMBBEndIdx(), and llvm::SlotIndex::getRegSlot().

Referenced by advance(), bumpDownwardPressure(), closeBottom(), and closeTop().

void RegPressureTracker::getDownwardPressure	(	const MachineInstr *	MI,
		std::vector< unsigned > &	PressureResult,
		std::vector< unsigned > &	MaxPressureResult
	)

Get the pressure of each PSet after traversing this instruction top-down.

Definition at line 781 of file RegisterPressure.cpp.

References bumpDownwardPressure(), and llvm::RegisterPressure::MaxSetPressure.

Referenced by getPressureAfterInst().

const LiveInterval * RegPressureTracker::getInterval ( unsigned Reg ) const [protected]

Definition at line 190 of file RegisterPressure.cpp.

References llvm::LiveIntervals::getCachedRegUnit(), llvm::LiveIntervals::getInterval(), and llvm::TargetRegisterInfo::isVirtualRegister().

Referenced by advance(), bumpDownwardPressure(), and recede().

void RegPressureTracker::getMaxDownwardPressureDelta	(	const MachineInstr *	MI,
		RegPressureDelta &	Delta,
		ArrayRef< PressureElement >	CriticalPSets,
		ArrayRef< unsigned >	MaxPressureLimit
	)

Consider the pressure increase caused by traversing this instruction top-down. Find the pressure set with the most change beyond its pressure limit based on the tracker's current pressure, and record the number of excess register units of that pressure set introduced by this instruction.

Consider the pressure increase caused by traversing this instruction top-down. Find the register class with the most change in its pressure limit based on the tracker's current pressure, and return the number of excess register units of that pressure set introduced by this instruction.

This assumes that the current LiveIn set is sufficient.

Definition at line 743 of file RegisterPressure.cpp.

References bumpDownwardPressure(), computeExcessPressureDelta(), computeMaxPressureDelta(), llvm::RegPressureDelta::CriticalMax, llvm::RegPressureDelta::CurrentMax, llvm::RegisterPressure::MaxSetPressure, and llvm::PressureElement::UnitIncrease.

Referenced by getMaxPressureDelta().

void llvm::RegPressureTracker::getMaxPressureDelta	(	const MachineInstr *	MI,
		RegPressureDelta &	Delta,
		ArrayRef< PressureElement >	CriticalPSets,
		ArrayRef< unsigned >	MaxPressureLimit
	)		`[inline]`

Find the pressure set with the most change beyond its pressure limit after traversing this instruction either upward or downward depending on the closed end of the current region.

Definition at line 275 of file RegisterPressure.h.

References getMaxDownwardPressureDelta(), getMaxUpwardPressureDelta(), isBottomClosed(), and isTopClosed().

void RegPressureTracker::getMaxUpwardPressureDelta	(	const MachineInstr *	MI,
		RegPressureDelta &	Delta,
		ArrayRef< PressureElement >	CriticalPSets,
		ArrayRef< unsigned >	MaxPressureLimit
	)

Consider the pressure increase caused by traversing this instruction bottom-up. Find the pressure set with the most change beyond its pressure limit based on the tracker's current pressure, and record the number of excess register units of that pressure set introduced by this instruction.

Consider the pressure increase caused by traversing this instruction bottom-up. Find the pressure set with the most change beyond its pressure limit based on the tracker's current pressure, and return the change in number of register units of that pressure set introduced by this instruction.

This assumes that the current LiveOut set is sufficient.

FIXME: This is expensive for an on-the-fly query. We need to cache the result per-SUnit with enough information to adjust for the current scheduling position. But this works as a proof of concept.

Definition at line 651 of file RegisterPressure.cpp.

References bumpUpwardPressure(), computeExcessPressureDelta(), computeMaxPressureDelta(), llvm::RegPressureDelta::CriticalMax, llvm::RegPressureDelta::CurrentMax, llvm::RegisterPressure::MaxSetPressure, and llvm::PressureElement::UnitIncrease.

Referenced by getMaxPressureDelta().

MachineBasicBlock::const_iterator llvm::RegPressureTracker::getPos ( ) const [inline]

Get the MI position corresponding to this register pressure.

Definition at line 214 of file RegisterPressure.h.

Referenced by llvm::ScheduleDAGInstrs::buildSchedGraph(), llvm::ScheduleDAGMI::initQueues(), llvm::ScheduleDAGMI::initRegPressure(), and llvm::ScheduleDAGMI::scheduleMI().

RegisterPressure& llvm::RegPressureTracker::getPressure ( ) [inline]

Get the resulting register pressure over the traversed region. This result is complete if either advance() or recede() has returned true, or if closeRegion() was explicitly invoked.

Definition at line 238 of file RegisterPressure.h.

Referenced by llvm::ScheduleDAGMI::initRegPressure(), and llvm::ScheduleDAGMI::scheduleMI().

const RegisterPressure& llvm::RegPressureTracker::getPressure ( ) const [inline]

Definition at line 239 of file RegisterPressure.h.

void llvm::RegPressureTracker::getPressureAfterInst	(	const MachineInstr *	MI,
		std::vector< unsigned > &	PressureResult,
		std::vector< unsigned > &	MaxPressureResult
	)		`[inline]`

Definition at line 297 of file RegisterPressure.h.

References getDownwardPressure(), getUpwardPressure(), isBottomClosed(), and isTopClosed().

std::vector<unsigned>& llvm::RegPressureTracker::getRegSetPressureAtPos ( ) [inline]

Get the register set pressure at the current position, which may be less than the pressure across the traversed region.

Definition at line 243 of file RegisterPressure.h.

void RegPressureTracker::getUpwardPressure	(	const MachineInstr *	MI,
		std::vector< unsigned > &	PressureResult,
		std::vector< unsigned > &	MaxPressureResult
	)

Get the pressure of each PSet after traversing this instruction bottom-up.

Definition at line 765 of file RegisterPressure.cpp.

References bumpUpwardPressure(), and llvm::RegisterPressure::MaxSetPressure.

Referenced by getPressureAfterInst().

void RegPressureTracker::increaseRegPressure ( ArrayRef< unsigned > Regs ) [protected]

Increase the current pressure as impacted by these registers and bump the high water mark if needed.

Definition at line 109 of file RegisterPressure.cpp.

References llvm::MachineRegisterInfo::getRegClass(), llvm::TargetRegisterInfo::getRegClassPressureSets(), llvm::TargetRegisterInfo::getRegClassWeight(), llvm::TargetRegisterInfo::getRegUnitPressureSets(), llvm::TargetRegisterInfo::getRegUnitWeight(), I, increaseSetPressure(), llvm::TargetRegisterInfo::isVirtualRegister(), llvm::RegisterPressure::MaxSetPressure, llvm::RegClassWeight::RegWeight, and llvm::ArrayRef< T >::size().

Referenced by addLiveRegs(), advance(), bumpDownwardPressure(), bumpUpwardPressure(), and recede().

void RegPressureTracker::init	(	const MachineFunction *	mf,
		const RegisterClassInfo *	rci,
		const LiveIntervals *	lis,
		const MachineBasicBlock *	mbb,
		MachineBasicBlock::const_iterator	pos
	)