Mila::Dnn::MemoryStats Struct Reference

Memory allocation breakdown for a single component. More...

Public Member Functions
MemoryStats &	operator+= (const MemoryStats &rhs) noexcept
	Accumulate another MemoryStats into this one.
std::string	toString () const
	Produce a human-readable summary of this stats instance.
std::size_t	totalBytes () const noexcept
	Total memory allocated across all categories and locations.
std::size_t	totalDeviceBytes () const noexcept
	Total device memory allocated by this component.
std::size_t	totalHostBytes () const noexcept
	Total host memory allocated by this component.

Public Attributes
std::size_t	device_gradient_bytes { 0 }
	Input and parameter gradient buffers.
std::size_t	device_parameter_bytes { 0 }
	Learnable parameter buffers (weights, biases).
std::size_t	device_state_bytes { 0 }
	Forward and decode output buffers, KV cache.
std::size_t	host_gradient_bytes { 0 }
	Gradient buffers on host (if any).
std::size_t	host_parameter_bytes { 0 }
	Learnable parameter buffers pinned on host (if any).
std::size_t	host_state_bytes { 0 }
	Forward and decode output buffers on host (if any).

Detailed Description

Memory allocation breakdown for a single component.

Reflects the current allocation state at the moment of the call. Categories map directly onto the component build lifecycle:

After construction parameters only After build() parameters + state After setTraining(true) parameters + state + gradients

All sizes are in bytes. Device and host allocations are tracked separately as they represent distinct, independently constrained resources.

Member Function Documentation

operator+=()

MemoryStats & Mila::Dnn::MemoryStats::operator+= ( const MemoryStats & rhs )

inlinenoexcept

Accumulate another MemoryStats into this one.

Used by CompositeComponent and Network to aggregate child stats.

toString()

std::string Mila::Dnn::MemoryStats::toString ( ) const

inlinenodiscard

Produce a human-readable summary of this stats instance.

Returns

Formatted multi-line string.

Here is the call graph for this function:

totalBytes()

std::size_t Mila::Dnn::MemoryStats::totalBytes ( ) const

inlinenodiscardnoexcept

Total memory allocated across all categories and locations.

Here is the call graph for this function:

Here is the caller graph for this function:

totalDeviceBytes()

std::size_t Mila::Dnn::MemoryStats::totalDeviceBytes ( ) const

inlinenodiscardnoexcept

Total device memory allocated by this component.

Here is the caller graph for this function:

totalHostBytes()

std::size_t Mila::Dnn::MemoryStats::totalHostBytes ( ) const

inlinenodiscardnoexcept

Total host memory allocated by this component.

Here is the caller graph for this function:

Member Data Documentation

device_gradient_bytes

std::size_t Mila::Dnn::MemoryStats::device_gradient_bytes { 0 }

Input and parameter gradient buffers.

Allocated lazily on first setTraining(true). Retained thereafter.

device_parameter_bytes

std::size_t Mila::Dnn::MemoryStats::device_parameter_bytes { 0 }

Learnable parameter buffers (weights, biases).

Allocated at construction. Static for the component lifetime.

device_state_bytes

std::size_t Mila::Dnn::MemoryStats::device_state_bytes { 0 }

Forward and decode output buffers, KV cache.

Allocated at build(). Static after build().

host_gradient_bytes

std::size_t Mila::Dnn::MemoryStats::host_gradient_bytes { 0 }

Gradient buffers on host (if any).

host_parameter_bytes

std::size_t Mila::Dnn::MemoryStats::host_parameter_bytes { 0 }

Learnable parameter buffers pinned on host (if any).

host_state_bytes

std::size_t Mila::Dnn::MemoryStats::host_state_bytes { 0 }

Forward and decode output buffers on host (if any).

---

The documentation for this struct was generated from the following file:

• /__w/Mila/Mila/Mila/Src/Dnn/Core/Component.MemoryStats.ixx