Dnn.Modules.Encoder Module Reference

Exported Modules
module	Compute.MemoryResource
module	Compute.OperationAttributes
module	Dnn.TensorHelpers
module	Dnn.Tensor
module	Compute.OperationRegistry
module	Dnn.Module
module	Compute.UnaryOperation
module	Dnn.TensorTraits
module	Compute.CpuMemoryResource
module	Compute.OperationBase
module	Compute.DeviceType
module	Compute.CudaDevice
module	Serialization.ModelArchive
module	Compute.CpuDevice
module	Compute.CudaMemoryResource
module	Compute.DeviceContext

Classes
class	Mila::Dnn::Encoder< TDeviceType, TInput, TOutput >
	An encoder module that provides token and positional embeddings. More...
class	Mila::Dnn::Encoder< TDeviceType, TInput, TOutput >
	An encoder module that provides token and positional embeddings. More...
class	Mila::Dnn::EncoderConfig
	Configuration class for Encoder module. More...

Typedefs
template<typename TInput = int, typename TOutput = float>
using	Mila::Dnn::CpuEncoder = Encoder< DeviceType::Cpu, TInput, TOutput >
	Type alias for CPU-based encoder module with customizable tensor types.
template<typename TInput = int, typename TOutput = float>
using	Mila::Dnn::CudaEncoder = Encoder< DeviceType::Cuda, TInput, TOutput >
	Type alias for CUDA-based encoder module with customizable tensor types.
using	ModuleBase = Module< TDeviceType, TInput, TOutput >
	Alias for base module type.
using	MR = std::conditional_t< TDeviceType==DeviceType::Cuda, CudaMemoryResource, CpuMemoryResource >
	Memory resource type determined based on device type.

Functions
	Encoder (const std::string &device_name, const EncoderConfig &config)
	Constructs a new Encoder module with a device name.
	Encoder (std::shared_ptr< DeviceContext > device_context, const EncoderConfig &config)
	Constructs a new Encoder module with a provided device context.
void	createOperation ()
	Creates the computational operation based on current device context.
void	forward (const Tensor< TInput, MR > &input, Tensor< TOutput, MR > &output)
	Performs the forward pass of the encoder.
size_t	getChannels () const
	Gets the number of channels (embedding dimension).
size_t	getMaxSequenceLength () const
	Gets the maximum sequence length.
size_t	getVocabularyLength () const
	Gets the vocabulary length.
void	initializeTensors ()
	Initializes the token and positional embedding tensors.
void	load (ModelArchive &archive) override
	Loads the encoder parameters from a zip archive.
size_t	parameterCount () const override
	Gets the number of parameters in the module.
void	save (ModelArchive &archive) const override
	Saves the encoder parameters to a zip archive.
std::string	toString () const override
	Gets the module information as a string.

Variables
OperationAttributes	attributes_
	Operation-specific attributes and configuration.
EncoderConfig	config_
	Configuration for the Encoder module.
std::shared_ptr< UnaryOperation< TDeviceType, TInput, TOutput > >	operation_ { nullptr }
	The computational operation that implements the encoder logic.
std::vector< std::shared_ptr< Tensor< TOutput, MR > > >	output_state_
	Output state tensors used for intermediate values.
std::vector< std::shared_ptr< Tensor< TOutput, MR > > >	parameters_
	Vector of parameter tensors that will be used during forward/backward passes.
std::shared_ptr< Tensor< TOutput, MR > >	wpe_ { nullptr }
	Position embedding table with shape (maxT,C), encodes token position information.
std::shared_ptr< Tensor< TOutput, MR > >	wte_ { nullptr }
	Token embedding table with shape (V,C), maps token IDs to vector representations.

Files
file	/home/runner/work/Mila/Mila/Mila/Src/Dnn/Modules/Layers/Encoder.ixx
	Implementation of the Encoder module for token and positional embeddings in transformer models.
file	/home/runner/work/Mila/Mila/Mila/Src/Dnn/Modules/Layers/EncoderConfig.ixx
	Configuration interface for the Encoder module in the Mila DNN framework.