Abstract base class for unary operations in the compute framework. More...

Inheritance diagram for Mila::Dnn::Compute::UnaryOperation< TDeviceType, TInput, TOutput >:

[legend]

Collaboration diagram for Mila::Dnn::Compute::UnaryOperation< TDeviceType, TInput, TOutput >:

Public Types
using	MR = std::conditional_t< TDeviceType==DeviceType::Cuda, CudaMemoryResource, HostMemoryResource >
	Memory resource type based on device type.

Public Member Functions
	UnaryOperation (OperationType operation_type)
	Constructs a UnaryOperation with the specified operation type.

	UnaryOperation (OperationType operation_type, std::shared_ptr< DeviceContext > context)
	Constructs a UnaryOperation with the specified operation type and device context.

virtual	~UnaryOperation ()=default
	Virtual destructor for proper cleanup of derived classes.

virtual void	backward (const Tensor< TInput, MR > &grad, const std::vector< std::shared_ptr< Tensor< TOutput, MR > > > &parameters, std::vector< std::shared_ptr< Tensor< TOutput, MR > > > &output_grads) const
	Executes the backward pass of a unary operation.

virtual void	backward (const Tensor< TInput, MR > &input, const Tensor< TOutput, MR > &output_grad, const std::vector< std::shared_ptr< Tensor< TOutput, MR > > > &parameters, std::vector< std::shared_ptr< Tensor< TOutput, MR > > > &parameter_grads, Tensor< TInput, MR > &input_grad, const OperationAttributes &properties, const std::vector< std::shared_ptr< Tensor< TOutput, MR > > > &output_state) const
	Executes the comprehensive backward pass of a unary operation.

virtual void	forward (const Tensor< TInput, MR > &input, const std::vector< std::shared_ptr< Tensor< TOutput, MR > > > &parameters, const OperationAttributes &properties, Tensor< TOutput, MR > &output, std::vector< std::shared_ptr< Tensor< TOutput, MR > > > &output_state) const =0
	Executes the forward pass of a unary operation.

Public Member Functions inherited from Mila::Dnn::Compute::OperationBase< TDeviceType, TInput1, TInput2, TOutput >
	OperationBase (OperationType operation_type, std::shared_ptr< DeviceContext > context)
	Constructs an OperationBase object with a specific device context and compute precision.

virtual	~OperationBase ()=default
	Virtual destructor for the OperationBase class.

std::shared_ptr< DeviceContext >	getDeviceContext () const
	Gets the device context associated with this operation.

DeviceType	getDeviceType () const
	Gets the device type for this operation.

virtual std::string	getName () const =0
	Gets the name of the operation.

OperationType	getOperationType () const
	Gets the operation type enumeration value.

Detailed Description

template<DeviceType TDeviceType = DeviceType::Cuda, typename TInput = float, typename TOutput = TInput>
requires ValidTensorType<TInput>&& ValidFloatTensorType<TOutput>
class Mila::Dnn::Compute::UnaryOperation< TDeviceType, TInput, TOutput >

Abstract base class for unary operations in the compute framework.

The UnaryOperation class defines the interface for operations that take a single input tensor and produce a single output tensor. Derived classes must implement the forward() method for the forward pass and may optionally override the backward() method for the backward pass.

Additional methods for shape validation and parameter initialization are provided to ensure correctness and flexibility in derived classes.

Template Parameters

TInput	The data type of the input tensor elements.
TOutput	The data type of the output tensor elements. Defaults to TInput.
TDeviceType	The device type (e.g., CPU, CUDA) on which the operation is executed.

Member Typedef Documentation

◆ MR

template<DeviceType TDeviceType = DeviceType::Cuda, typename TInput = float, typename TOutput = TInput>

using Mila::Dnn::Compute::UnaryOperation< TDeviceType, TInput, TOutput >::MR = std::conditional_t<TDeviceType == DeviceType::Cuda, CudaMemoryResource, HostMemoryResource>

Memory resource type based on device type.

This type alias automatically selects the appropriate memory resource type based on the device type. For CUDA devices, it uses CudaMemoryResource; for CPU devices, it uses CpuMemoryResource.

Constructor & Destructor Documentation

◆ UnaryOperation() [1/2]

template<DeviceType TDeviceType = DeviceType::Cuda, typename TInput = float, typename TOutput = TInput>

Mila::Dnn::Compute::UnaryOperation< TDeviceType, TInput, TOutput >::UnaryOperation ( OperationType operation_type )

inline

Constructs a UnaryOperation with the specified operation type.

Creates a compatible device context automatically based on TDeviceType.

Parameters

operation_type	The type of the operation.
precision_policy	The compute precision policy to use.

◆ UnaryOperation() [2/2]

template<DeviceType TDeviceType = DeviceType::Cuda, typename TInput = float, typename TOutput = TInput>

Mila::Dnn::Compute::UnaryOperation< TDeviceType, TInput, TOutput >::UnaryOperation	(	OperationType	operation_type,
		std::shared_ptr< DeviceContext >	context
	)

inline

Constructs a UnaryOperation with the specified operation type and device context.

Validates that the provided context is compatible with TDeviceType.

Parameters

operation_type	The type of the operation.
context	The device context to use for this operation.
precision_policy	The compute precision policy to use.

◆ ~UnaryOperation()

template<DeviceType TDeviceType = DeviceType::Cuda, typename TInput = float, typename TOutput = TInput>

virtual Mila::Dnn::Compute::UnaryOperation< TDeviceType, TInput, TOutput >::~UnaryOperation ( )

virtualdefault

Virtual destructor for proper cleanup of derived classes.

Member Function Documentation

◆ backward() [1/2]

template<DeviceType TDeviceType = DeviceType::Cuda, typename TInput = float, typename TOutput = TInput>

virtual void Mila::Dnn::Compute::UnaryOperation< TDeviceType, TInput, TOutput >::backward	(	const Tensor< TInput, MR > &	grad,
		const std::vector< std::shared_ptr< Tensor< TOutput, MR > > > &	parameters,
		std::vector< std::shared_ptr< Tensor< TOutput, MR > > > &	output_grads
	)		const

inlinevirtual

Executes the backward pass of a unary operation.

This is the simplified version of the backward pass that only computes gradients for the parameters, not for the input.

Derived classes may override this method to define the backward computation. The default implementation throws an exception, indicating that the operation does not support a backward pass.

Parameters

grad	The gradient tensor from the next layer in the network.
parameters	The parameters used during the forward pass.
output_grads	Output vector where parameter gradients will be stored.

Exceptions

std::runtime_error If the operation does not support backward pass.

◆ backward() [2/2]

template<DeviceType TDeviceType = DeviceType::Cuda, typename TInput = float, typename TOutput = TInput>

virtual void Mila::Dnn::Compute::UnaryOperation< TDeviceType, TInput, TOutput >::backward	(	const Tensor< TInput, MR > &	input,
		const Tensor< TOutput, MR > &	output_grad,
		const std::vector< std::shared_ptr< Tensor< TOutput, MR > > > &	parameters,
		std::vector< std::shared_ptr< Tensor< TOutput, MR > > > &	parameter_grads,
		Tensor< TInput, MR > &	input_grad,
		const OperationAttributes &	properties,
		const std::vector< std::shared_ptr< Tensor< TOutput, MR > > > &	output_state
	)		const

inlinevirtual

Executes the comprehensive backward pass of a unary operation.

This comprehensive backward method computes:

Gradients with respect to inputs (for backpropagation)
Gradients with respect to parameters (for optimization)

This version provides more flexibility for operations that need access to both the original input and the intermediates from the forward pass to compute gradients.

Parameters

input	The original input tensor from the forward pass.
output_grad	Gradient of the loss with respect to this operation's output.
parameters	The parameters used during the forward pass.
parameter_grads	Output vector where parameter gradients will be stored.
input_grad	Output tensor where input gradients will be stored.
properties	The same properties used during the forward pass.
output_state	Cached tensors from the forward pass.

Exceptions

std::runtime_error If the operation does not support this backward pass.

◆ forward()

template<DeviceType TDeviceType = DeviceType::Cuda, typename TInput = float, typename TOutput = TInput>

virtual void Mila::Dnn::Compute::UnaryOperation< TDeviceType, TInput, TOutput >::forward	(	const Tensor< TInput, MR > &	input,
		const std::vector< std::shared_ptr< Tensor< TOutput, MR > > > &	parameters,
		const OperationAttributes &	properties,
		Tensor< TOutput, MR > &	output,
		std::vector< std::shared_ptr< Tensor< TOutput, MR > > > &	output_state
	)		const

pure virtual

Executes the forward pass of a unary operation.

Derived classes must implement this method to define the forward computation.

Parameters

input	The input tensor to process.
parameters	The parameters for the operation (e.g., weights, biases).
properties	Additional properties that configure the operation's behavior.
output	The output tensor where results will be stored.
output_state	Cache for intermediate results needed in backward pass.

The documentation for this class was generated from the following file:

/home/runner/work/Mila/Mila/Mila/Src/Dnn/Compute/Operations/UnaryOperation.ixx

Public Types

Public Member Functions

Detailed Description

Member Typedef Documentation

◆ MR

Constructor & Destructor Documentation

◆ UnaryOperation() [1/2]

◆ UnaryOperation() [2/2]

◆ ~UnaryOperation()

Member Function Documentation

◆ backward() [1/2]

◆ backward() [2/2]

◆ forward()