Mila::Dnn::TensorBuffer< T, MR, TrackMemory > Class Template Reference

A buffer for storing tensor data with configurable memory management. More...

Public Member Functions
	TensorBuffer (const TensorBuffer &)=delete
	Copy operations are explicitly deleted.
	TensorBuffer (size_t size, T *data_ptr)
	Constructs a new TensorBuffer object with externally managed memory.
	TensorBuffer (size_t size, T value=T{})
	Constructs a new TensorBuffer object with owned memory and optional value initialization.
	~TensorBuffer ()
	Destroys the TensorBuffer object and deallocates memory if owned.
size_t	aligned_size () const
	Gets the aligned size of the buffer in bytes.
T *	data ()
	Gets a pointer to the buffer's data.
bool	is_aligned () const noexcept
	Checks if the buffer's data pointer is properly aligned.
TensorBuffer &	operator= (const TensorBuffer &)=delete
void	resize (size_t new_size)
	Resizes the buffer to a new size, preserving existing data when possible.
size_t	size () const
	Gets the size of the buffer.

Static Public Attributes
static constexpr size_t	alignment = detail::get_alignment<MR>()
	The alignment boundary for memory allocation based on the memory resource type.

Private Member Functions
size_t	calculateAlignedSize (size_t num_elements) const
	Calculates the aligned size in bytes for a given number of elements.
std::unique_ptr< Compute::MemoryResource >	createMemoryResource ()
	Creates a memory resource based on the template parameter.
void	fillBuffer (T value=T{})
	Initializes the buffer with a specified value.

Private Attributes
size_t	aligned_size_ { 0 }
	The actual size in bytes of the allocated memory, including alignment padding.
T *	data_ { nullptr }
	A pointer to the allocated memory containing the buffer's data.
std::unique_ptr< Compute::MemoryResource >	mr_ { nullptr }
	Memory resource used for allocation/deallocation (null for externally managed buffers).
size_t	size_ { 0 }
	The number of elements in the buffer.

Detailed Description

template<typename T, typename MR, bool TrackMemory = false>
requires std::is_base_of_v<Compute::MemoryResource, MR>
class Mila::Dnn::TensorBuffer< T, MR, TrackMemory >

A buffer for storing tensor data with configurable memory management.

TensorBuffer provides a generic container for tensor data that can be stored either in CPU or GPU memory based on the memory resource type. It supports both owned and externally managed memory.

Template Parameters

TElementType	The type of the elements stored in the buffer.
MR	The memory resource type that determines where and how memory is allocated. Must derive from Compute::MemoryResource.

Note

Thread Safety: The buffer is not thread-safe. Multiple threads accessing the same buffer must be synchronized externally.

Exception Safety: Basic guarantee - if an exception occurs during operations, no memory is leaked but the buffer may be left in an undefined state.

Example usage:

// Create a CPU buffer with 100 floats initialized to 0.0f
TensorBuffer<float, Compute::HostMemoryResource> cpuBuffer(100, 0.0f);
 
// Create a GPU buffer with 100 floats
TensorBuffer<float, Compute::CudaMemoryResource> gpuBuffer(100);

Constructor & Destructor Documentation

TensorBuffer() [1/3]

template<typename T , typename MR , bool TrackMemory = false>

Mila::Dnn::TensorBuffer< T, MR, TrackMemory >::TensorBuffer ( size_t  size, T  value = T{}  )

inlineexplicit

Constructs a new TensorBuffer object with owned memory and optional value initialization.

This constructor allocates memory using the specified memory resource and optionally initializes all elements to the provided value.

Parameters

size	The number of elements in the buffer.
value	Value to initialize the buffer with (default: default-constructed TElementType).

Exceptions

std::overflow_error	If size is too large, causing overflow in aligned size calculation.
std::bad_alloc	If memory allocation fails.

TensorBuffer() [2/3]

template<typename T , typename MR , bool TrackMemory = false>

Mila::Dnn::TensorBuffer< T, MR, TrackMemory >::TensorBuffer ( size_t  size, T *  data_ptr  )

inline

Constructs a new TensorBuffer object with externally managed memory.

This constructor does not allocate memory but instead wraps the provided pre-allocated memory. The buffer does not take ownership of this memory.

Parameters

size	The number of elements in the buffer.
data_ptr	Pointer to the pre-allocated memory.

Exceptions

std::invalid_argument

If data_ptr is null.

~TensorBuffer()

template<typename T , typename MR , bool TrackMemory = false>

Mila::Dnn::TensorBuffer< T, MR, TrackMemory >::~TensorBuffer ( )

inline

Destroys the TensorBuffer object and deallocates memory if owned.

If the buffer owns its memory (mr_ is not null), it will deallocate the memory using the memory resource. If the buffer uses externally managed memory, it will not deallocate anything.

TensorBuffer() [3/3]

template<typename T , typename MR , bool TrackMemory = false>

Mila::Dnn::TensorBuffer< T, MR, TrackMemory >::TensorBuffer ( const TensorBuffer< T, MR, TrackMemory > &  )

delete

Copy operations are explicitly deleted.

TensorBuffer does not support copying since it may involve deep copying of large memory blocks across different memory spaces.

Member Function Documentation

aligned_size()

template<typename T , typename MR , bool TrackMemory = false>

size_t Mila::Dnn::TensorBuffer< T, MR, TrackMemory >::aligned_size ( ) const

inline

Gets the aligned size of the buffer in bytes.

This is the actual memory size allocated, which may be larger than size() * sizeof(TPrecision) due to alignment padding.

Returns

size_t The aligned size in bytes.

calculateAlignedSize()

template<typename T , typename MR , bool TrackMemory = false>

size_t Mila::Dnn::TensorBuffer< T, MR, TrackMemory >::calculateAlignedSize ( size_t  num_elements ) const

inlineprivate

Calculates the aligned size in bytes for a given number of elements.

Parameters

num_elements

The number of elements.

Returns

size_t The aligned size in bytes.

Here is the caller graph for this function:

createMemoryResource()

template<typename T , typename MR , bool TrackMemory = false>

std::unique_ptr< Compute::MemoryResource > Mila::Dnn::TensorBuffer< T, MR, TrackMemory >::createMemoryResource ( )

inlineprivate

Creates a memory resource based on the template parameter.

Returns

std::unique_ptr<Compute::MemoryResource> The created memory resource.

data()

template<typename T , typename MR , bool TrackMemory = false>

T * Mila::Dnn::TensorBuffer< T, MR, TrackMemory >::data ( )

inline

Gets a pointer to the buffer's data.

Returns

TElementType* A pointer to the data stored in the buffer.

fillBuffer()

template<typename T , typename MR , bool TrackMemory = false>

void Mila::Dnn::TensorBuffer< T, MR, TrackMemory >::fillBuffer ( T  value = T{} )

inlineprivate

Initializes the buffer with a specified value.

This method fills the buffer with the specified value, using different approaches depending on whether the buffer is in CPU or GPU memory.

Parameters

value

Value to initialize the buffer with (default: default-constructed TElementType).

Note

For CPU memory, SIMD optimizations may be used for larger trivially copyable types.

For GPU memory, a temporary host buffer is created and copied to the device.

Here is the caller graph for this function:

is_aligned()

template<typename T , typename MR , bool TrackMemory = false>

bool Mila::Dnn::TensorBuffer< T, MR, TrackMemory >::is_aligned ( ) const

inlinenoexcept

Checks if the buffer's data pointer is properly aligned.

This method verifies that the buffer's data pointer is aligned according to the alignment requirements of the memory resource.

Returns

bool True if the data pointer is properly aligned, false otherwise.

operator=()

template<typename T , typename MR , bool TrackMemory = false>

TensorBuffer & Mila::Dnn::TensorBuffer< T, MR, TrackMemory >::operator= ( const TensorBuffer< T, MR, TrackMemory > &  )

delete

resize()

template<typename T , typename MR , bool TrackMemory = false>

void Mila::Dnn::TensorBuffer< T, MR, TrackMemory >::resize ( size_t  new_size )

inline

Resizes the buffer to a new size, preserving existing data when possible.

This method allocates a new buffer of the specified size, copies data from the old buffer to the new one (up to the smaller of the old and new sizes), and then deallocates the old buffer. If the new size is larger than the old size, the additional elements are default-initialized.

Parameters

new_size

The new size of the buffer. Can be zero.

Exceptions

std::runtime_error	If the buffer is externally managed.
std::overflow_error	If new_size is too large, causing overflow in aligned size calculation.
std::bad_alloc	If memory allocation fails.

Here is the call graph for this function:

size()

template<typename T , typename MR , bool TrackMemory = false>

size_t Mila::Dnn::TensorBuffer< T, MR, TrackMemory >::size ( ) const

inline

Gets the size of the buffer.

Returns

size_t The number of elements in the buffer.

Member Data Documentation

aligned_size_

template<typename T , typename MR , bool TrackMemory = false>

size_t Mila::Dnn::TensorBuffer< T, MR, TrackMemory >::aligned_size_ { 0 }

private

The actual size in bytes of the allocated memory, including alignment padding.

alignment

template<typename T , typename MR , bool TrackMemory = false>

constexpr size_t Mila::Dnn::TensorBuffer< T, MR, TrackMemory >::alignment = detail::get_alignment<MR>()

staticconstexpr

The alignment boundary for memory allocation based on the memory resource type.

This value is determined at compile time to ensure optimal memory access patterns for the specific memory resource type (CPU or GPU).

data_

template<typename T , typename MR , bool TrackMemory = false>

T* Mila::Dnn::TensorBuffer< T, MR, TrackMemory >::data_ { nullptr }

private

A pointer to the allocated memory containing the buffer's data.

mr_

template<typename T , typename MR , bool TrackMemory = false>

std::unique_ptr<Compute::MemoryResource> Mila::Dnn::TensorBuffer< T, MR, TrackMemory >::mr_ { nullptr }

private

Memory resource used for allocation/deallocation (null for externally managed buffers).

size_

template<typename T , typename MR , bool TrackMemory = false>

size_t Mila::Dnn::TensorBuffer< T, MR, TrackMemory >::size_ { 0 }

private

The number of elements in the buffer.

---

The documentation for this class was generated from the following file:

• /home/runner/work/Mila/Mila/Mila/Src/Dnn/Tensor/TensorBuffer.ixx