Dnn.Modules.Residual Module Reference

Exported Modules
module	Compute.OperationAttributes
module	Dnn.TensorHelpers
module	Compute.DeviceContext
module	Dnn.Modules.Linear
module	Compute.CudaMemoryResource
module	Compute.OperationBase
module	Compute.CpuMemoryResource
module	Compute.MemoryResource
module	Compute.DeviceType
module	Dnn.Tensor
module	Dnn.Module
module	Compute.BinaryOperation
module	Dnn.TensorTraits
module	Compute.OperationRegistry
module	Compute.Precision
module	Serialization.ModelArchive
module	Compute.ComputeDevice

Classes
class	Mila::Dnn::Residual< TDeviceType, TInput, TOutput >
	A class implementing a residual connection module. More...
class	Mila::Dnn::ResidualConfig
	Configuration class for Residual connection module. More...

Typedefs
template<typename TInput = float, typename TOutput = TInput>
using	Mila::Dnn::CpuResidual = Residual< DeviceType::Cpu, TInput, TOutput >
	Type alias for CPU-based residual module with customizable tensor types.
template<typename TInput = float, typename TOutput = TInput>
using	Mila::Dnn::CudaResidual = Residual< DeviceType::Cuda, TInput, TOutput >
	Type alias for CUDA-based residual module with customizable tensor types.
using	ModuleBase = Module< TDeviceType, TInput, TOutput >
	Alias for base module type.
using	MR = std::conditional_t< TDeviceType==DeviceType::Cuda, CudaMemoryResource, CpuMemoryResource >
	Memory resource type used for tensors, selected based on device type.

Functions
	Residual (const std::string &device_name, const ResidualConfig &config)
	Constructs a new Residual module with a device name.
	Residual (std::shared_ptr< DeviceContext > device_context, const ResidualConfig &config)
	Constructs a new Residual module with a provided device context.
void	addTensors (const Tensor< TInput, MR > &a, const Tensor< TInput, MR > &b, Tensor< TInput, MR > &result)
	Adds two tensors element-wise.
void	backward (const Tensor< TInput, MR > &input, const Tensor< TOutput, MR > &output_grad, Tensor< TInput, MR > &input_grad)
	Performs the backward pass of the Residual connection.
static std::string	connectionTypeToString (ResidualConfig::ConnectionType type)
	Converts connection type enum to string for display purposes.
void	createOperation ()
	Creates an appropriate operation based on the connection type.
void	createProjection (const std::vector< size_t > &input_shape, const std::vector< size_t > &output_shape)
	Creates a projection layer when input and output dimensions don't match.
void	forward (const Tensor< TInput, MR > &input, Tensor< TOutput, MR > &output)
	Performs the forward pass of the Residual connection.
std::shared_ptr< Module< TDeviceType, TInput, TOutput > >	getInnerModule ()
	Gets the inner module.
void	initializeGateParameters (const std::vector< size_t > &shape)
	Initializes parameters for gated connections.
void	load (ModelArchive &archive) override
	Deserializes the module state from a ZIP archive.
size_t	parameterCount () const override
	Gets the number of trainable parameters in this module.
void	save (ModelArchive &zip) const override
	Serializes the module state to a ZIP archive.
bool	tensorShapesMatch (const Tensor< TInput, MR > &a, const Tensor< TOutput, MR > &b)
	Checks if two tensor shapes match for residual connection.
std::string	toString () const override
	Converts the module information to a human-readable string.

Variables
ResidualConfig	config_
	Configuration for the Residual module.
std::shared_ptr< Tensor< TOutput, MR > >	gate_weights_
	Learnable gate weights for gated residual connections.
std::shared_ptr< BinaryOperation< TDeviceType, TInput, TOutput, TOutput > >	gated_operation_
	Binary operation for gated residual connections.
Tensor< TInput, MR >	inner_input_grad_ {}
	Temporary tensor to store inner module gradients during backward pass.
std::shared_ptr< Module< TDeviceType, TInput, TOutput > >	inner_module_
	The inner module implementing the transformation F(x).
Tensor< TOutput, MR >	inner_output_ {}
	Temporary tensor to store inner module output during forward pass.
std::vector< std::shared_ptr< Tensor< TOutput, MR > > >	inner_parameter_grads_
	Gradients for inner parameters.
std::shared_ptr< BinaryOperation< TDeviceType, TInput, TOutput, TOutput > >	operation_
	Binary operation for standard and scaled residual connections.
std::vector< std::shared_ptr< Tensor< TOutput, MR > > >	output_state_
	Output state tensors for backward pass.
std::vector< std::shared_ptr< Tensor< TOutput, MR > > >	parameter_grads_
	Gradients for trainable parameters.
std::vector< std::shared_ptr< Tensor< TOutput, MR > > >	parameters_
	Collection of trainable parameters.
std::shared_ptr< Linear< TDeviceType, TInput, TOutput > >	projection_
	Optional projection layer for dimension matching.
Tensor< TOutput, MR >	projection_output_ {}
	Temporary tensor to store projection output during forward pass.
OperationAttributes	properties_
	Operation-specific attributes.
Tensor< TInput, MR >	temp_grad_ {}
	Temporary tensor for gradient accumulation.

Files
file	/home/runner/work/Mila/Mila/Mila/Src/Dnn/Modules/Layers/Residual.ixx
	Implementation of the Residual connection module for neural networks.
file	/home/runner/work/Mila/Mila/Mila/Src/Dnn/Modules/Layers/ResidualConfig.ixx
	Configuration interface for the Residual module in the Mila DNN framework.