Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory > Class Template Reference

Device-agnostic buffer for storing tensor data with abstract type system. More...

Public Types
using	DataType = TensorDataType
	Abstract data type enumeration.
using	DataTypeTraits = TensorDataTypeTraits<TDataType>
	Compile-time data type characteristics.

Public Member Functions
	TensorBuffer (const TensorBuffer &)=delete
	Copy operations explicitly deleted for performance safety.
	TensorBuffer (int device_id, size_t logical_size)
	Constructs buffer with owned memory.
	TensorBuffer (TensorBuffer &&other) noexcept
	Move constructor for efficient ownership transfer.
	~TensorBuffer ()
	Destructor with automatic memory cleanup via RAII.
size_t	alignedSize () const noexcept
	Returns the aligned memory allocation size in bytes.
const void *	data () const noexcept
	Returns const raw pointer to buffer data.
void *	data () noexcept
	Returns raw pointer to buffer data.
bool	empty () const noexcept
	Checks if the buffer is empty.
Compute::MemoryResource *	getMemoryResource () const noexcept
	Returns pointer to the memory resource managing this buffer's storage.
bool	isAligned () const noexcept
	Checks if buffer memory is properly aligned.
TensorBuffer &	operator= (const TensorBuffer &)=delete
TensorBuffer &	operator= (TensorBuffer &&other) noexcept
	Move assignment operator for efficient ownership transfer.
void	resize (size_t new_logical_size)
	Resizes buffer WITHOUT preserving existing data.
size_t	size () const noexcept
	Returns the number of logical elements in the buffer.
size_t	storageBytes () const noexcept
	Returns the storage size in bytes.

Static Public Attributes
static constexpr size_t	alignment = Detail::get_alignment<TDataType, TMemoryResource>()
	Optimal memory alignment.
static constexpr TensorDataType	data_type = TDataType
	Compile-time data type constant.
static constexpr size_t	element_size = DataTypeTraits::size_in_bytes
	Storage size per element.
static constexpr bool	is_device_only = DataTypeTraits::is_device_only
	Device-only type restriction.
static constexpr bool	is_float_type = DataTypeTraits::is_float_type
	Floating-point type classification.
static constexpr bool	is_integer_type = DataTypeTraits::is_integer_type
	Integer type classification.

Private Member Functions
size_t	calculateAlignedSize (size_t storage_bytes) const noexcept
	Calculates aligned size for given storage requirements.
std::unique_ptr< Compute::MemoryResource >	createMemoryResource ()
	Creates appropriate memory resource with device context.
void	logAllocation () const
	Logs memory allocation for tracking and profiling.
void	logDeallocation () const
	Logs memory deallocation for tracking and profiling.

Private Attributes
size_t	aligned_size_ { 0 }
	Total allocated bytes including alignment.
std::byte *	data_ { nullptr }
	Pointer to allocated memory buffer.
int	device_id_ { -1 }
	Device Id for memory resource operations.
size_t	logical_size_ { 0 }
	Number of logical elements in buffer.
std::unique_ptr< Compute::MemoryResource >	mr_ { nullptr }
	Memory resource for allocation (null for external).
size_t	storage_bytes_ { 0 }
	Actual storage bytes.

Detailed Description

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>
requires isValidTensor<TDataType, TMemoryResource>
class Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >

Device-agnostic buffer for storing tensor data with abstract type system.

Advanced memory management container providing efficient storage for tensor data across heterogeneous compute environments using abstract TensorDataType enumeration. Supports device-specific alignment optimization and automatic compatibility validation.

Core architectural principles:

• Abstract data types prevent device-specific compilation issues
• Automatic memory alignment optimization for target hardware
• Support for precision formats including FP32, FP16, BF16, FP8, and integer types
• Device-agnostic memory operations with compile-time dispatch optimization
• Optional allocation tracking for memory profiling and debugging
• Exception-safe design with strong safety guarantees

The buffer supports both owned memory management and external memory wrapping, enabling integration with existing memory pools and external libraries while maintaining optimal performance characteristics.

Template Parameters

TDataType	Abstract tensor data type from TensorDataType enumeration
TMemoryResource	Memory resource type determining allocation strategy and device targeting
TrackMemory	When true, enables detailed memory allocation tracking and profiling

Note

Thread Safety: Buffer operations are not thread-safe; external synchronization required

Exception Safety: Strong guarantee for most operations; basic guarantee for constructors

Memory Layout: Automatic optimization for device-specific alignment

Memory Transfers: Transfer operations belong at the Tensor level where device contexts are meaningful

See also

TensorDataType for supported abstract data type enumeration

TensorDataTypeTraits for compile-time data type characteristics

MemoryResource for device memory abstraction layer

Example usage:

// CPU buffer with FP32 data type
TensorBuffer<TensorDataType::FP32, Compute::CpuMemoryResource> cpuBuffer(device_context, 100);
 
// GPU buffer with FP16 data type
TensorBuffer<TensorDataType::FP16, Compute::CudaDeviceMemoryResource> gpuBuffer(device_context, 100);
 
// Buffer with memory tracking enabled
TensorBuffer<TensorDataType::BF16, Compute::CudaDeviceMemoryResource, true> trackedBuffer(device_context, 100);

Member Typedef Documentation

DataType

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

using Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::DataType = TensorDataType

Abstract data type enumeration.

DataTypeTraits

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

using Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::DataTypeTraits = TensorDataTypeTraits<TDataType>

Compile-time data type characteristics.

Constructor & Destructor Documentation

TensorBuffer() [1/3]

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::TensorBuffer ( int device_id, size_t logical_size )

inlineexplicit

Constructs buffer with owned memory.

Allocates optimally aligned memory using the specified memory resource and initializes all memory to zero for deterministic behavior. Memory alignment is optimized based on data type and target hardware.

Parameters

device_context	Device context for memory resource initialization
logical_size	Number of logical elements to store in the buffer

Exceptions

std::invalid_argument	If device_context is null
std::overflow_error	If size causes overflow in storage calculations
std::bad_alloc	If memory allocation fails
std::runtime_error	If memory resource operations fail

Note

Memory is aligned according to hardware optimization requirements

Zero-sized buffers are handled efficiently without allocation

Here is the call graph for this function:

Here is the caller graph for this function:

~TensorBuffer()

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::~TensorBuffer ( )

inline

Destructor with automatic memory cleanup via RAII.

Automatically deallocates owned memory through the memory resource. External memory is not deallocated, maintaining safe resource management. Provides optional deallocation tracking for memory profiling.

Here is the call graph for this function:

TensorBuffer() [2/3]

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::TensorBuffer ( const TensorBuffer< TDataType, TMemoryResource, TrackMemory > & )

delete

Copy operations explicitly deleted for performance safety.

Prevents accidental expensive copy operations involving large memory transfers across different memory spaces and devices.

Here is the call graph for this function:

TensorBuffer() [3/3]

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::TensorBuffer ( TensorBuffer< TDataType, TMemoryResource, TrackMemory > && other )

inlinenoexcept

Move constructor for efficient ownership transfer.

Transfers all resources from source buffer without memory copying, leaving source in valid but empty state.

Here is the call graph for this function:

Member Function Documentation

alignedSize()

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

size_t Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::alignedSize ( ) const

inlinenoexcept

Returns the aligned memory allocation size in bytes.

This represents the actual memory allocated, which includes alignment padding for optimal hardware performance.

Returns

Total allocated memory size including alignment padding

calculateAlignedSize()

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

size_t Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::calculateAlignedSize ( size_t storage_bytes ) const

inlineprivatenoexcept

Calculates aligned size for given storage requirements.

Ensures memory allocations meet hardware alignment requirements for optimal performance on target devices.

Here is the caller graph for this function:

createMemoryResource()

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

std::unique_ptr< Compute::MemoryResource > Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::createMemoryResource ( )

inlineprivate

Creates appropriate memory resource with device context.

Instantiates memory resource with device context and optional allocation tracking wrapper for debugging and profiling purposes.

Here is the caller graph for this function:

data() [1/2]

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

const void * Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::data ( ) const

inlinenoexcept

Returns const raw pointer to buffer data.

Provides read-only access to the underlying memory buffer.

Returns

Const raw pointer to buffer memory

Warning

No type safety or bounds checking

data() [2/2]

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

void * Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::data ( )

inlinenoexcept

Returns raw pointer to buffer data.

Provides direct access to the underlying memory buffer for performance-critical operations and device kernel interfacing.

Returns

Raw pointer to buffer memory

Warning

No type safety or bounds checking

Note

Use with appropriate casting based on data type requirements

Here is the caller graph for this function:

empty()

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

bool Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::empty ( ) const

inlinenoexcept

Checks if the buffer is empty.

Returns

true if buffer contains no logical elements, false otherwise

getMemoryResource()

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

Compute::MemoryResource * Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::getMemoryResource ( ) const

inlinenoexcept

Returns pointer to the memory resource managing this buffer's storage.

Provides access to the memory resource for efficient dispatch optimization, zero-copy operations when memory resources are compatible, and type-safe downcasting to specific memory resource types.

Returns

Pointer to memory resource (nullptr if using external memory)

Note

Returns nullptr for buffers constructed with external memory

For owned memory, returns pointer to the underlying memory resource

Can be safely cast to TMemoryResource* based on template parameter

Enables efficient memory resource compatibility checks

Example:

TensorBuffer<TensorDataType::FP32, CudaMemoryResource> buffer(ctx, 100);
auto* mr = buffer.getMemoryResource();
 
// Type-safe downcast check
if (auto* cuda_mr = dynamic_cast<CudaMemoryResource*>(mr)) {
    // Use CUDA-specific operations
}

isAligned()

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

bool Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::isAligned ( ) const

inlinenoexcept

Checks if buffer memory is properly aligned.

Verifies that the buffer's data pointer meets the alignment requirements for optimal performance on the target hardware.

Returns

true if properly aligned, false otherwise

logAllocation()

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

void Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::logAllocation ( ) const

inlineprivate

Logs memory allocation for tracking and profiling.

Here is the call graph for this function:

Here is the caller graph for this function:

logDeallocation()

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

void Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::logDeallocation ( ) const

inlineprivate

Logs memory deallocation for tracking and profiling.

Here is the call graph for this function:

Here is the caller graph for this function:

operator=() [1/2]

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

TensorBuffer & Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::operator= ( const TensorBuffer< TDataType, TMemoryResource, TrackMemory > & )

delete

Here is the call graph for this function:

operator=() [2/2]

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

TensorBuffer & Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::operator= ( TensorBuffer< TDataType, TMemoryResource, TrackMemory > && other )

inlinenoexcept

Move assignment operator for efficient ownership transfer.

Safely transfers ownership while properly cleaning up existing resources.

Here is the call graph for this function:

resize()

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

void Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::resize ( size_t new_logical_size )

inline

Resizes buffer WITHOUT preserving existing data.

Allocates new optimally aligned memory. Existing data is DISCARDED. Use for buffer reuse scenarios where data preservation is not required.

Parameters

new_logical_size

New number of logical elements

Exceptions

std::runtime_error	If buffer uses external memory
std::overflow_error	If new size causes storage overflow
std::bad_alloc	If memory allocation fails

Note

Does NOT preserve existing data (caller must reinitialize)

New memory is uninitialized for performance

Here is the call graph for this function:

size()

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

size_t Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::size ( ) const

inlinenoexcept

Returns the number of logical elements in the buffer.

Returns

Number of logical elements stored in the buffer

storageBytes()

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

size_t Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::storageBytes ( ) const

inlinenoexcept

Returns the storage size in bytes.

Provides the actual memory storage used.

Returns

Storage size in bytes

Here is the caller graph for this function:

Member Data Documentation

aligned_size_

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

size_t Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::aligned_size_ { 0 }

private

Total allocated bytes including alignment.

alignment

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

size_t Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::alignment = Detail::get_alignment<TDataType, TMemoryResource>()

staticconstexpr

Optimal memory alignment.

data_

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

std::byte* Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::data_ { nullptr }

private

Pointer to allocated memory buffer.

data_type

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

TensorDataType Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::data_type = TDataType

staticconstexpr

Compile-time data type constant.

device_id_

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

int Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::device_id_ { -1 }

private

Device Id for memory resource operations.

element_size

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

size_t Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::element_size = DataTypeTraits::size_in_bytes

staticconstexpr

Storage size per element.

is_device_only

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

bool Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::is_device_only = DataTypeTraits::is_device_only

staticconstexpr

Device-only type restriction.

is_float_type

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

bool Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::is_float_type = DataTypeTraits::is_float_type

staticconstexpr

Floating-point type classification.

is_integer_type

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

bool Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::is_integer_type = DataTypeTraits::is_integer_type

staticconstexpr

Integer type classification.

logical_size_

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

size_t Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::logical_size_ { 0 }

private

Number of logical elements in buffer.

mr_

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

std::unique_ptr<Compute::MemoryResource> Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::mr_ { nullptr }

private

Memory resource for allocation (null for external).

storage_bytes_

template<TensorDataType TDataType, typename TMemoryResource, bool TrackMemory = false>

size_t Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >::storage_bytes_ { 0 }

private

Actual storage bytes.

---

The documentation for this class was generated from the following file:

• /__w/Mila/Mila/Mila/Src/Dnn/Tensors/TensorBuffer.ixx