TensorBuffer.ixx File Reference

Device-agnostic memory management layer for tensor data using abstract data types. More...

#include <memory>
#include <limits>
#include <iostream>
#include <stdexcept>
#include <type_traits>
#include <cstring>
#include <cstddef>
#include <cassert>
#include <string>
import Compute.CpuMemoryResource;
import Compute.MemoryResourceTracker;
import Compute.MemoryResource;
import Dnn.TensorDataTypeTraits;
import Dnn.TensorDataTypeMap;
import Dnn.TensorDataType;

Classes
class	Mila::Dnn::TensorBuffer< TDataType, TMemoryResource, TrackMemory >
	Device-agnostic buffer for storing tensor data with abstract type system. More...

Namespaces
namespace	Mila
	Mila main API namespace.
namespace	Mila::Dnn
namespace	Mila::Dnn::Detail

Functions
std::string	Mila::Dnn::Detail::formatBytes (size_t bytes)
template<TensorDataType TDataType, typename MR>
constexpr size_t	Mila::Dnn::Detail::get_alignment ()
	Determines optimal memory alignment based on memory resource and data type.
template<TensorDataType TDataType>
constexpr size_t	Mila::Dnn::Detail::getStorageSize (size_t logical_size)
	Calculates storage size in bytes for given logical element count.

Variables
constexpr size_t	Mila::Dnn::Detail::CPU_SIMD_ALIGN = 64
	AVX-512 alignment boundary for optimal CPU SIMD operations.
constexpr size_t	Mila::Dnn::Detail::CUDA_WARP_SIZE = 32
	CUDA warp size alignment for optimal GPU memory access patterns.

Detailed Description

Device-agnostic memory management layer for tensor data using abstract data types.

This module provides a sophisticated memory management system for tensor data that operates across heterogeneous compute environments (CPU, CUDA, Metal, Rocm) using abstract TensorDataType enumeration. The system handles device-specific alignment optimization and automatic memory resource selection based on data type compatibility constraints.

Key architectural features:

• Abstract data type system eliminates device-specific compilation dependencies
• Automatic memory alignment optimization for target hardware (SIMD, CUDA warps)
• Device-agnostic memory operations with compile-time dispatch
• Optional memory allocation tracking and profiling capabilities
• Exception-safe memory management with strong guarantees