Profiling information for a single function. More...

#include "llvm/ProfileData/InstrProf.h"

Inheritance diagram for llvm::InstrProfRecord:

Public Types
enum	CountPseudoKind { NotPseudo = 0 , PseudoHot , PseudoWarm }
enum	PseudoCountVal { HotFunctionVal = -1 , WarmFunctionVal = -2 }

Public Member Functions
	InstrProfRecord ()=default
	InstrProfRecord (std::vector< uint64_t > Counts)
	InstrProfRecord (std::vector< uint64_t > Counts, std::vector< uint8_t > BitmapBytes)
	InstrProfRecord (InstrProfRecord &&)=default
	InstrProfRecord (const InstrProfRecord &RHS)
InstrProfRecord &	operator= (InstrProfRecord &&)=default
InstrProfRecord &	operator= (const InstrProfRecord &RHS)
bool	isBlockUniform (unsigned BlockIdx) const
	Check if a basic block is entered via a wave-uniform branch.
LLVM_ABI void	computeBlockUniformity ()
	Recompute uniformity metadata from raw uniform counters, when present.
uint32_t	getNumValueKinds () const
	Return the number of value profile kinds with non-zero number of profile sites.
uint32_t	getNumValueSites (uint32_t ValueKind) const
	Return the number of instrumented sites for ValueKind.
uint32_t	getNumValueData (uint32_t ValueKind) const
	Return the total number of ValueData for ValueKind.
ArrayRef< InstrProfValueData >	getValueArrayForSite (uint32_t ValueKind, uint32_t Site) const
	Return the array of profiled values at `Site`.
void	reserveSites (uint32_t ValueKind, uint32_t NumValueSites)
	Reserve space for NumValueSites sites.
LLVM_ABI void	addValueData (uint32_t ValueKind, uint32_t Site, ArrayRef< InstrProfValueData > VData, InstrProfSymtab *SymTab)
	Add ValueData for ValueKind at value Site.
LLVM_ABI void	merge (InstrProfRecord &Other, uint64_t Weight, function_ref< void(instrprof_error)> Warn)
	Merge the counts in `Other` into this one.
LLVM_ABI void	scale (uint64_t N, uint64_t D, function_ref< void(instrprof_error)> Warn)
	Scale up profile counts (including value profile data) by a factor of (N / D).
void	sortValueData ()
	Sort value profile data (per site) by count.
void	Clear ()
	Clear value data entries, edge counters, and uniformity data.
void	clearValueData ()
	Clear value data entries.
LLVM_ABI void	accumulateCounts (CountSumOrPercent &Sum) const
	Compute the sums of all counts and store in Sum.
LLVM_ABI void	overlap (InstrProfRecord &Other, OverlapStats &Overlap, OverlapStats &FuncLevelOverlap, uint64_t ValueCutoff)
	Compute the overlap b/w this IntrprofRecord and Other.
LLVM_ABI void	overlapValueProfData (uint32_t ValueKind, InstrProfRecord &Src, OverlapStats &Overlap, OverlapStats &FuncLevelOverlap)
	Compute the overlap of value profile counts.
CountPseudoKind	getCountPseudoKind () const
void	setPseudoCount (CountPseudoKind Kind)

Public Attributes
std::vector< uint64_t >	Counts
std::vector< uint8_t >	BitmapBytes
std::vector< uint64_t >	UniformCounts
	For AMDGPU offload profiling: raw or merged uniform counters.
std::vector< uint8_t >	UniformityBits
	For AMDGPU offload profiling: 1 bit per basic block indicating whether the block is usually entered with all lanes active.
uint16_t	OffloadDeviceWaveSize = 0

Detailed Description

Profiling information for a single function.

Definition at line 906 of file InstrProf.h.

Member Enumeration Documentation

◆ CountPseudoKind

enum llvm::InstrProfRecord::CountPseudoKind

Enumerator
NotPseudo
PseudoHot
PseudoWarm

Definition at line 1026 of file InstrProf.h.

◆ PseudoCountVal

enum llvm::InstrProfRecord::PseudoCountVal

Enumerator
HotFunctionVal
WarmFunctionVal

Definition at line 1031 of file InstrProf.h.

Constructor & Destructor Documentation

◆ InstrProfRecord() [1/5]

llvm::InstrProfRecord::InstrProfRecord ( )

default

Referenced by InstrProfRecord(), InstrProfRecord(), merge(), llvm::NamedInstrProfRecord::NamedInstrProfRecord(), llvm::NamedInstrProfRecord::NamedInstrProfRecord(), llvm::NamedInstrProfRecord::NamedInstrProfRecord(), operator=(), operator=(), overlap(), and overlapValueProfData().

◆ InstrProfRecord() [2/5]

llvm::InstrProfRecord::InstrProfRecord ( std::vector< uint64_t > Counts )

inline

Definition at line 919 of file InstrProf.h.

References Counts, and llvm::move().

◆ InstrProfRecord() [3/5]

llvm::InstrProfRecord::InstrProfRecord	(	std::vector< uint64_t >	Counts,
		std::vector< uint8_t >	BitmapBytes )

inline

Definition at line 920 of file InstrProf.h.

References BitmapBytes, Counts, and llvm::move().

◆ InstrProfRecord() [4/5]

llvm::InstrProfRecord::InstrProfRecord ( InstrProfRecord && )

default

References InstrProfRecord().

◆ InstrProfRecord() [5/5]

llvm::InstrProfRecord::InstrProfRecord ( const InstrProfRecord & RHS )

inline

Definition at line 924 of file InstrProf.h.

References BitmapBytes, Counts, InstrProfRecord(), OffloadDeviceWaveSize, RHS, UniformCounts, and UniformityBits.

Member Function Documentation

◆ accumulateCounts()

void llvm::InstrProfRecord::accumulateCounts ( CountSumOrPercent & Sum ) const

Compute the sums of all counts and store in Sum.

Definition at line 774 of file InstrProf.cpp.

References llvm::Count, Counts, llvm::CountSumOrPercent::CountSum, getNumValueSites(), getValueArrayForSite(), I, llvm::CountSumOrPercent::NumEntries, llvm::NumValueSites, and llvm::CountSumOrPercent::ValueCounts.

Referenced by overlap().

◆ addValueData()

void llvm::InstrProfRecord::addValueData	(	uint32_t	ValueKind,
		uint32_t	Site,
		ArrayRef< InstrProfValueData >	VData,
		InstrProfSymtab *	SymTab )

Add ValueData for ValueKind at value Site.

We do not support adding sites out of order. Site must go up from 0 one by one.

Definition at line 1124 of file InstrProf.cpp.

References assert(), and llvm::ArrayRef< T >::size().

◆ Clear()

void llvm::InstrProfRecord::Clear ( )

inline

Clear value data entries, edge counters, and uniformity data.

Definition at line 1003 of file InstrProf.h.

References clearValueData(), Counts, OffloadDeviceWaveSize, UniformCounts, and UniformityBits.

◆ clearValueData()

void llvm::InstrProfRecord::clearValueData ( )

inline

Clear value data entries.

Definition at line 1012 of file InstrProf.h.

Referenced by Clear().

◆ computeBlockUniformity()

void llvm::InstrProfRecord::computeBlockUniformity ( )

Recompute uniformity metadata from raw uniform counters, when present.

Definition at line 952 of file InstrProf.cpp.

References Counts, I, UniformCounts, and UniformityBits.

Referenced by merge(), and scale().

◆ getCountPseudoKind()

CountPseudoKind llvm::InstrProfRecord::getCountPseudoKind ( ) const

inline

Definition at line 1035 of file InstrProf.h.

References Counts, HotFunctionVal, NotPseudo, PseudoHot, PseudoWarm, and WarmFunctionVal.

Referenced by merge().

◆ getNumValueData()

uint32_t llvm::InstrProfRecord::getNumValueData ( uint32_t ValueKind ) const

inline

Return the total number of ValueData for ValueKind.

Definition at line 1140 of file InstrProf.h.

References N.

◆ getNumValueKinds()

uint32_t llvm::InstrProfRecord::getNumValueKinds ( ) const

inline

Return the number of value profile kinds with non-zero number of profile sites.

Definition at line 1133 of file InstrProf.h.

◆ getNumValueSites()

uint32_t llvm::InstrProfRecord::getNumValueSites ( uint32_t ValueKind ) const

inline

Return the number of instrumented sites for ValueKind.

Definition at line 1147 of file InstrProf.h.

Referenced by accumulateCounts(), overlap(), and overlapValueProfData().

◆ getValueArrayForSite()

ArrayRef< InstrProfValueData > llvm::InstrProfRecord::getValueArrayForSite	(	uint32_t	ValueKind,
		uint32_t	Site ) const

inline

Return the array of profiled values at Site.

Definition at line 1152 of file InstrProf.h.

Referenced by accumulateCounts(), llvm::annotateValueSite(), and llvm::getNumValueDataForSiteInstrProf().

◆ isBlockUniform()

bool llvm::InstrProfRecord::isBlockUniform ( unsigned BlockIdx ) const

inline

Check if a basic block is entered via a wave-uniform branch.

Returns true if uniform (safe for PGO spill optimization) or if no uniformity data is available (conservative default).

Definition at line 952 of file InstrProf.h.

References UniformityBits.

◆ merge()

void llvm::InstrProfRecord::merge	(	InstrProfRecord &	Other,
		uint64_t	Weight,
		function_ref< void(instrprof_error)>	Warn )

Merge the counts in Other into this one.

Optionally scale merged counts by Weight.

Definition at line 990 of file InstrProf.cpp.

References llvm::bitmap_mismatch, BitmapBytes, computeBlockUniformity(), llvm::count_mismatch, llvm::counter_overflow, Counts, getCountPseudoKind(), llvm::getInstrMaxCountValue(), I, InstrProfRecord(), llvm::mergeUniformityBits(), NotPseudo, OffloadDeviceWaveSize, llvm::Other, PseudoHot, PseudoWarm, llvm::SaturatingMultiplyAdd(), setPseudoCount(), UniformCounts, and UniformityBits.

◆ operator=() [1/2]

InstrProfRecord & llvm::InstrProfRecord::operator= ( const InstrProfRecord & RHS )

inline

Definition at line 932 of file InstrProf.h.

References BitmapBytes, Counts, InstrProfRecord(), OffloadDeviceWaveSize, RHS, UniformCounts, and UniformityBits.

◆ operator=() [2/2]

InstrProfRecord & llvm::InstrProfRecord::operator= ( InstrProfRecord && )

default

References InstrProfRecord().

◆ overlap()

void llvm::InstrProfRecord::overlap	(	InstrProfRecord &	Other,
		OverlapStats &	Overlap,
		OverlapStats &	FuncLevelOverlap,
		uint64_t	ValueCutoff )

Compute the overlap b/w this IntrprofRecord and Other.

Definition at line 841 of file InstrProf.cpp.

References accumulateCounts(), llvm::OverlapStats::addOneMismatch(), assert(), llvm::OverlapStats::Base, Counts, llvm::CountSumOrPercent::CountSum, getNumValueSites(), I, InstrProfRecord(), llvm::CountSumOrPercent::NumEntries, llvm::Other, llvm::OverlapStats::Overlap, overlapValueProfData(), llvm::OverlapStats::score(), llvm::OverlapStats::Test, and llvm::OverlapStats::Valid.

Referenced by llvm::InstrProfWriter::overlapRecord(), and overlapValueProfData().

◆ overlapValueProfData()

void llvm::InstrProfRecord::overlapValueProfData	(	uint32_t	ValueKind,
		InstrProfRecord &	Src,
		OverlapStats &	Overlap,
		OverlapStats &	FuncLevelOverlap )

Compute the overlap of value profile counts.

Definition at line 823 of file InstrProf.cpp.

References assert(), getNumValueSites(), I, InstrProfRecord(), llvm::Other, and overlap().

Referenced by overlap().

◆ reserveSites()

void llvm::InstrProfRecord::reserveSites	(	uint32_t	ValueKind,
		uint32_t	NumValueSites )

inline

Reserve space for NumValueSites sites.

Definition at line 1156 of file InstrProf.h.

References llvm::NumValueSites.

◆ scale()

void llvm::InstrProfRecord::scale	(	uint64_t	N,
		uint64_t	D,
		function_ref< void(instrprof_error)>	Warn )

Scale up profile counts (including value profile data) by a factor of (N / D).

Definition at line 1081 of file InstrProf.cpp.

References assert(), computeBlockUniformity(), llvm::Count, llvm::counter_overflow, Counts, D(), llvm::getInstrMaxCountValue(), N, llvm::SaturatingMultiply(), and UniformCounts.

◆ setPseudoCount()

void llvm::InstrProfRecord::setPseudoCount ( CountPseudoKind Kind )

inline

Definition at line 1043 of file InstrProf.h.

References Counts, HotFunctionVal, PseudoHot, PseudoWarm, and WarmFunctionVal.

Referenced by merge().

◆ sortValueData()

void llvm::InstrProfRecord::sortValueData ( )

inline

Sort value profile data (per site) by count.

Definition at line 996 of file InstrProf.h.

Member Data Documentation

◆ BitmapBytes

std::vector<uint8_t> llvm::InstrProfRecord::BitmapBytes

Definition at line 908 of file InstrProf.h.

Referenced by llvm::InstrProfRecordWriterTrait::EmitData(), llvm::InstrProfRecordWriterTrait::EmitKeyDataLength(), InstrProfRecord(), InstrProfRecord(), merge(), llvm::NamedInstrProfRecord::NamedInstrProfRecord(), llvm::NamedInstrProfRecord::NamedInstrProfRecord(), and operator=().

◆ Counts

std::vector<uint64_t> llvm::InstrProfRecord::Counts

Definition at line 907 of file InstrProf.h.

Referenced by accumulateCounts(), Clear(), computeBlockUniformity(), llvm::InstrProfRecordWriterTrait::EmitData(), llvm::InstrProfRecordWriterTrait::EmitKeyDataLength(), getCountPseudoKind(), InstrProfRecord(), InstrProfRecord(), InstrProfRecord(), merge(), llvm::NamedInstrProfRecord::NamedInstrProfRecord(), llvm::NamedInstrProfRecord::NamedInstrProfRecord(), llvm::NamedInstrProfRecord::NamedInstrProfRecord(), operator=(), overlap(), scale(), and setPseudoCount().

◆ OffloadDeviceWaveSize

uint16_t llvm::InstrProfRecord::OffloadDeviceWaveSize = 0

Definition at line 916 of file InstrProf.h.

Referenced by Clear(), InstrProfRecord(), merge(), and operator=().

◆ UniformCounts

std::vector<uint64_t> llvm::InstrProfRecord::UniformCounts

For AMDGPU offload profiling: raw or merged uniform counters.

One uint64_t per instrumented block, tracking entries where all lanes were active.

Definition at line 911 of file InstrProf.h.

Referenced by Clear(), computeBlockUniformity(), InstrProfRecord(), merge(), operator=(), and scale().

◆ UniformityBits

std::vector<uint8_t> llvm::InstrProfRecord::UniformityBits

For AMDGPU offload profiling: 1 bit per basic block indicating whether the block is usually entered with all lanes active.

Raw uniform counters are reduced to these bits when profiles are written or merged.

Definition at line 915 of file InstrProf.h.

Referenced by Clear(), computeBlockUniformity(), llvm::InstrProfRecordWriterTrait::EmitData(), llvm::InstrProfRecordWriterTrait::EmitKeyDataLength(), InstrProfRecord(), isBlockUniform(), merge(), llvm::NamedInstrProfRecord::NamedInstrProfRecord(), and operator=().

The documentation for this struct was generated from the following files:

include/llvm/ProfileData/InstrProf.h
lib/ProfileData/InstrProf.cpp

Public Types

Public Member Functions

Public Attributes

Detailed Description

Member Enumeration Documentation

◆ CountPseudoKind

◆ PseudoCountVal

Constructor & Destructor Documentation

◆ InstrProfRecord() [1/5]

◆ InstrProfRecord() [2/5]

◆ InstrProfRecord() [3/5]

◆ InstrProfRecord() [4/5]

◆ InstrProfRecord() [5/5]

Member Function Documentation

◆ accumulateCounts()

◆ addValueData()

◆ Clear()

◆ clearValueData()

◆ computeBlockUniformity()

◆ getCountPseudoKind()

◆ getNumValueData()

◆ getNumValueKinds()

◆ getNumValueSites()

◆ getValueArrayForSite()

◆ isBlockUniform()

◆ merge()

◆ operator=() [1/2]

◆ operator=() [2/2]

◆ overlap()

◆ overlapValueProfData()

◆ reserveSites()

◆ scale()

◆ setPseudoCount()

◆ sortValueData()

Member Data Documentation

◆ BitmapBytes

◆ Counts

◆ OffloadDeviceWaveSize

◆ UniformCounts

◆ UniformityBits