Inheritance diagram for Mila::Dnn::Compute::CpuGeluOp:

[legend]

Collaboration diagram for Mila::Dnn::Compute::CpuGeluOp:

[legend]

Public Types
using	MR = typename CpuDevice::MR

using	UnaryOperationBase = UnaryOperation< DeviceType::Cpu, float, float >

Public Types inherited from Mila::Dnn::Compute::UnaryOperation< DeviceType::Cpu, float, float >
using	MR = std::conditional_t< TDeviceType==DeviceType::Cuda, CudaMemoryResource, HostMemoryResource >
	Memory resource type based on device type.

Public Member Functions
	CpuGeluOp (const GeluConfig &config)
	Constructs a new CpuGeluOp with the default device context.

	CpuGeluOp (std::shared_ptr< DeviceContext > context, const GeluConfig &config)
	Constructs a new CpuGeluOp with a specific device context.

void	backward (float dinp, float inp, float *dout, int N)
	Performs the backward pass of the GELU activation function.

void	forward (const Tensor< float, MR > &input, const std::vector< std::shared_ptr< Tensor< float, MR > > > &parameters, const OperationAttributes &properties, Tensor< float, MR > &output, std::vector< std::shared_ptr< Tensor< float, MR > > > &output_state) const override
	Performs the forward pass of the GELU activation function.

std::string	getName () const override
	Gets the name of this operation.

Public Member Functions inherited from Mila::Dnn::Compute::UnaryOperation< DeviceType::Cpu, float, float >
	UnaryOperation (OperationType operation_type)
	Constructs a UnaryOperation with the specified operation type.

	UnaryOperation (OperationType operation_type, std::shared_ptr< DeviceContext > context)
	Constructs a UnaryOperation with the specified operation type and device context.

virtual	~UnaryOperation ()=default
	Virtual destructor for proper cleanup of derived classes.

virtual void	backward (const Tensor< float, MR > &grad, const std::vector< std::shared_ptr< Tensor< float, MR > > > &parameters, std::vector< std::shared_ptr< Tensor< float, MR > > > &output_grads) const
	Executes the backward pass of a unary operation.

virtual void	backward (const Tensor< float, MR > &input, const Tensor< float, MR > &output_grad, const std::vector< std::shared_ptr< Tensor< float, MR > > > &parameters, std::vector< std::shared_ptr< Tensor< float, MR > > > &parameter_grads, Tensor< float, MR > &input_grad, const OperationAttributes &properties, const std::vector< std::shared_ptr< Tensor< float, MR > > > &output_state) const
	Executes the comprehensive backward pass of a unary operation.

virtual void	forward (const Tensor< float, MR > &input, const std::vector< std::shared_ptr< Tensor< float, MR > > > &parameters, const OperationAttributes &properties, Tensor< float, MR > &output, std::vector< std::shared_ptr< Tensor< float, MR > > > &output_state) const=0
	Executes the forward pass of a unary operation.

Public Member Functions inherited from Mila::Dnn::Compute::OperationBase< TDeviceType, TInput1, TInput2, TOutput >
	OperationBase (OperationType operation_type, std::shared_ptr< DeviceContext > context)
	Constructs an OperationBase object with a specific device context and compute precision.

virtual	~OperationBase ()=default
	Virtual destructor for the OperationBase class.

std::shared_ptr< DeviceContext >	getDeviceContext () const
	Gets the device context associated with this operation.

DeviceType	getDeviceType () const
	Gets the device type for this operation.

OperationType	getOperationType () const
	Gets the operation type enumeration value.

Private Attributes
GeluConfig	config_
	CPU operations don't support mixed precision.

Member Typedef Documentation

◆ MR

using Mila::Dnn::Compute::CpuGeluOp::MR = typename CpuDevice::MR

◆ UnaryOperationBase

using Mila::Dnn::Compute::CpuGeluOp::UnaryOperationBase = UnaryOperation<DeviceType::Cpu, float, float>

Constructor & Destructor Documentation

◆ CpuGeluOp() [1/2]

Mila::Dnn::Compute::CpuGeluOp::CpuGeluOp ( const GeluConfig & config )

inline

Constructs a new CpuGeluOp with the default device context.

CPU operations always use full precision regardless of policy settings.

Parameters

precision_policy Ignored for CPU operations, as they always use full precision.

◆ CpuGeluOp() [2/2]

Mila::Dnn::Compute::CpuGeluOp::CpuGeluOp	(	std::shared_ptr< DeviceContext >	context,
		const GeluConfig &	config
	)

inline

Constructs a new CpuGeluOp with a specific device context.

CPU operations always use full precision regardless of policy settings.

Parameters

context	The device context to use for this operation.
precision_policy	Ignored for CPU operations, as they always use full precision.

Exceptions

std::runtime_error If the context is not for a CPU device.

Member Function Documentation

◆ backward()

void Mila::Dnn::Compute::CpuGeluOp::backward	(	float *	dinp,
		float *	inp,
		float *	dout,
		int	N
	)

inline

Performs the backward pass of the GELU activation function.

Computes the gradient of the GELU function with respect to its input.

Parameters

dinp	Pointer to gradient buffer for input.
inp	Pointer to original input values.
dout	Pointer to gradient from output.
N	Number of elements.

Here is the call graph for this function:

◆ forward()

void Mila::Dnn::Compute::CpuGeluOp::forward	(	const Tensor< float, MR > &	input,
		const std::vector< std::shared_ptr< Tensor< float, MR > > > &	parameters,
		const OperationAttributes &	properties,
		Tensor< float, MR > &	output,
		std::vector< std::shared_ptr< Tensor< float, MR > > > &	output_state
	)		const

inlineoverride

Performs the forward pass of the GELU activation function.

Implements the Gaussian Error Linear Unit (GELU) activation function: GELU(x) = 0.5 * x * (1 + tanh(sqrt(2/?) * (x + 0.044715 * x^3)))

Parameters

input	The input tensor.
parameters	Parameter tensors (not used in this operation).
properties	Additional operation attributes.
output	The output tensor.
output_state	Cache for intermediate results (not used).

Here is the call graph for this function:

◆ getName()

std::string Mila::Dnn::Compute::CpuGeluOp::getName ( ) const

inlineoverridevirtual

Gets the name of this operation.

Returns: std::string The name of the operation ("Cpu::GeluOp").

Implements Mila::Dnn::Compute::OperationBase< TDeviceType, TInput1, TInput2, TOutput >.

Member Data Documentation

◆ config_

GeluConfig Mila::Dnn::Compute::CpuGeluOp::config_

private

CPU operations don't support mixed precision.

This method overrides the base class implementation to always return false, indicating that mixed precision is not supported/enabled for CPU operations.

Returns: bool Always returns false. Configuration for the GELU operation.

The documentation for this class was generated from the following file:

/home/runner/work/Mila/Mila/Mila/Src/Dnn/Compute/Operations/Cpu/CpuGeluOp.ixx

Public Types

Public Member Functions

Private Attributes

Member Typedef Documentation

◆ MR

◆ UnaryOperationBase

Constructor & Destructor Documentation

◆ CpuGeluOp() [1/2]

◆ CpuGeluOp() [2/2]

Member Function Documentation

◆ backward()

◆ forward()

◆ getName()

Member Data Documentation

◆ config_