Inheritance diagram for Mila::Dnn::Compute::BinaryOperation< TDeviceType, TPrecision, TInputA, TInputB >:

Collaboration diagram for Mila::Dnn::Compute::BinaryOperation< TDeviceType, TPrecision, TInputA, TInputB >:

Public Types
using	MR = CpuMemoryResource
using	ParameterGradTensor = Tensor<TPrecision, MR>
using	ParameterTensor = Tensor<TPrecision, MR>
using	TensorLeftType = Tensor<TInputA, MR>
using	TensorOutputType = Tensor<TPrecision, MR>
using	TensorRightType = Tensor<TInputB, MR>
Public Types inherited from Mila::Dnn::Compute::Operation< TDeviceType, TPrecision >
using	DataTypeTraits

Public Member Functions
virtual	~BinaryOperation ()=default
virtual void	backward (const ITensor &input_a, const ITensor &input_b, const ITensor &output_grad, ITensor &input_a_grad, ITensor &input_b_grad) const =0
	Backward pass: compute gradients w.r.t.
virtual void	forward (const ITensor &input_a, const ITensor &input_b, ITensor &output) const =0
	Forward pass: output = f(input_a, input_b).
Public Member Functions inherited from Mila::Dnn::Compute::Operation< TDeviceType, TPrecision >
virtual	~Operation ()=default
virtual void	build (const BuildContext &build_context)
	Prepare the operation for a concrete input shape.
virtual void	clearGradients () noexcept
	Clear any cached gradient pointers held by the operation.
virtual TensorDataType	getDataType () const
	Tensor data type for this operation.
virtual DeviceType	getDeviceType () const
	Device type for this operation.
virtual std::string	getName () const=0
	Human-readable operation name.
virtual OperationType	getOperationType () const=0
	Operation type identifier.
virtual std::size_t	getStateMemorySize () const
	Returns the number of bytes of state memory allocated by this operation.
virtual bool	isBuilt () const
	Whether build() completed successfully for a concrete input shape.
virtual bool	isEvalMode () const
	Query whether operation is configured for training.
virtual void	setGradients (ITensor weight_grad, ITensor bias_grad)
	Bind module-owned gradient tensors to the operation.
virtual void	setParameters (ITensor weight, ITensor bias)
	Bind module-owned parameter tensors to the operation.
virtual void	setTrainingMode (TrainingMode training_mode)
	Configure operation training-mode behavior.

Static Protected Member Functions
static const TensorLeftType &	asLeftTensor (const ITensor &t)
static TensorOutputType &	asOutputTensor (ITensor &t)
static const TensorRightType &	asRightTensor (const ITensor &t)

Additional Inherited Members
Static Public Attributes inherited from Mila::Dnn::Compute::Operation< TDeviceType, TPrecision >
static constexpr TensorDataType	data_type
static constexpr DeviceType	device_type
Protected Attributes inherited from Mila::Dnn::Compute::Operation< TDeviceType, TPrecision >
bool	is_built_
TrainingMode	training_mode_

Detailed Description

template<DeviceType TDeviceType, TensorDataType TPrecision, TensorDataType TInputA = TPrecision, TensorDataType TInputB = TPrecision>
requires PrecisionSupportedOnDevice<TPrecision, TDeviceType>
class Mila::Dnn::Compute::BinaryOperation< TDeviceType, TPrecision, TInputA, TInputB >

Template Parameters

TDeviceType	Device type for this operation (Cpu, Cuda, ...)
TPrecision	Canonical element precision produced/consumed by the op (e.g. FP32)
TInputA	Element type for left input (defaults to TPrecision)
TInputB	Element type for right input (defaults to TPrecision)

Note: Operation precision (TPrecision) is constrained to be supported on the target device.

Member Typedef Documentation

◆ MR

template<DeviceType TDeviceType, TensorDataType TPrecision, TensorDataType TInputA = TPrecision, TensorDataType TInputB = TPrecision>

using Mila::Dnn::Compute::BinaryOperation< TDeviceType, TPrecision, TInputA, TInputB >::MR = CpuMemoryResource

◆ ParameterGradTensor

template<DeviceType TDeviceType, TensorDataType TPrecision, TensorDataType TInputA = TPrecision, TensorDataType TInputB = TPrecision>

using Mila::Dnn::Compute::BinaryOperation< TDeviceType, TPrecision, TInputA, TInputB >::ParameterGradTensor = Tensor<TPrecision, MR>

◆ ParameterTensor

template<DeviceType TDeviceType, TensorDataType TPrecision, TensorDataType TInputA = TPrecision, TensorDataType TInputB = TPrecision>

using Mila::Dnn::Compute::BinaryOperation< TDeviceType, TPrecision, TInputA, TInputB >::ParameterTensor = Tensor<TPrecision, MR>

◆ TensorLeftType

template<DeviceType TDeviceType, TensorDataType TPrecision, TensorDataType TInputA = TPrecision, TensorDataType TInputB = TPrecision>

using Mila::Dnn::Compute::BinaryOperation< TDeviceType, TPrecision, TInputA, TInputB >::TensorLeftType = Tensor<TInputA, MR>

◆ TensorOutputType

template<DeviceType TDeviceType, TensorDataType TPrecision, TensorDataType TInputA = TPrecision, TensorDataType TInputB = TPrecision>

using Mila::Dnn::Compute::BinaryOperation< TDeviceType, TPrecision, TInputA, TInputB >::TensorOutputType = Tensor<TPrecision, MR>

◆ TensorRightType

template<DeviceType TDeviceType, TensorDataType TPrecision, TensorDataType TInputA = TPrecision, TensorDataType TInputB = TPrecision>

using Mila::Dnn::Compute::BinaryOperation< TDeviceType, TPrecision, TInputA, TInputB >::TensorRightType = Tensor<TInputB, MR>

Constructor & Destructor Documentation

◆ ~BinaryOperation()

template<DeviceType TDeviceType, TensorDataType TPrecision, TensorDataType TInputA = TPrecision, TensorDataType TInputB = TPrecision>

virtual Mila::Dnn::Compute::BinaryOperation< TDeviceType, TPrecision, TInputA, TInputB >::~BinaryOperation ( )

virtualdefault

Member Function Documentation

◆ asLeftTensor()

template<DeviceType TDeviceType, TensorDataType TPrecision, TensorDataType TInputA = TPrecision, TensorDataType TInputB = TPrecision>

const TensorLeftType & Mila::Dnn::Compute::BinaryOperation< TDeviceType, TPrecision, TInputA, TInputB >::asLeftTensor ( const ITensor & t )

inlinestaticprotected

◆ asOutputTensor()

template<DeviceType TDeviceType, TensorDataType TPrecision, TensorDataType TInputA = TPrecision, TensorDataType TInputB = TPrecision>

TensorOutputType & Mila::Dnn::Compute::BinaryOperation< TDeviceType, TPrecision, TInputA, TInputB >::asOutputTensor ( ITensor & t )

inlinestaticprotected

◆ asRightTensor()

template<DeviceType TDeviceType, TensorDataType TPrecision, TensorDataType TInputA = TPrecision, TensorDataType TInputB = TPrecision>

const TensorRightType & Mila::Dnn::Compute::BinaryOperation< TDeviceType, TPrecision, TInputA, TInputB >::asRightTensor ( const ITensor & t )

inlinestaticprotected

◆ backward()

template<DeviceType TDeviceType, TensorDataType TPrecision, TensorDataType TInputA = TPrecision, TensorDataType TInputB = TPrecision>

virtual void Mila::Dnn::Compute::BinaryOperation< TDeviceType, TPrecision, TInputA, TInputB >::backward	(	const ITensor &	input_a,
		const ITensor &	input_b,
		const ITensor &	output_grad,
		ITensor &	input_a_grad,
		ITensor &	input_b_grad ) const

pure virtual

Backward pass: compute gradients w.r.t.

inputs.

Parameter gradients are written via pointers bound by setParameterGradients().

Parameters

input_a	First input from forward pass
input_b	Second input from forward pass
output_grad	Gradient w.r.t. output (dL/dOutput)
input_a_grad	Gradient w.r.t. first input (dL/dInputA)
input_b_grad	Gradient w.r.t. second input (dL/dInputB)

Implemented in Mila::Dnn::Compute::CpuResidualOp, Mila::Dnn::Compute::CpuSoftmaxCrossEntropyOp< TPrecision, TLogits, TTargets >, Mila::Dnn::Compute::Cuda::Residual::CudaResidualOp< TInputA, TInputB, TPrecision >, Mila::Dnn::Compute::Cuda::Residual::CudaResidualOp< TensorDataType::BF16 >, Mila::Dnn::Compute::Cuda::Residual::CudaResidualOp< TensorDataType::FP32 >, Mila::Dnn::Compute::Cuda::SoftmaxCrossEntropy::CudaSoftmaxCrossEntropyOp< TPrecision, TLogits, TTargets >, Mila::Dnn::Compute::Cuda::SoftmaxCrossEntropy::CudaSoftmaxCrossEntropyOp< TensorDataType::BF16 >, and Mila::Dnn::Compute::Cuda::SoftmaxCrossEntropy::CudaSoftmaxCrossEntropyOp< TensorDataType::FP32 >.

◆ forward()

template<DeviceType TDeviceType, TensorDataType TPrecision, TensorDataType TInputA = TPrecision, TensorDataType TInputB = TPrecision>

virtual void Mila::Dnn::Compute::BinaryOperation< TDeviceType, TPrecision, TInputA, TInputB >::forward	(	const ITensor &	input_a,
		const ITensor &	input_b,
		ITensor &	output ) const

pure virtual

Forward pass: output = f(input_a, input_b).

Implementations should accept polymorphic ITensor references and may use the typed helpers below to obtain concrete Tensor<T,...> references.

Implemented in Mila::Dnn::Compute::CpuResidualOp, Mila::Dnn::Compute::CpuSoftmaxCrossEntropyOp< TPrecision, TLogits, TTargets >, Mila::Dnn::Compute::Cuda::Residual::CudaResidualOp< TInputA, TInputB, TPrecision >, Mila::Dnn::Compute::Cuda::Residual::CudaResidualOp< TensorDataType::BF16 >, Mila::Dnn::Compute::Cuda::Residual::CudaResidualOp< TensorDataType::FP32 >, Mila::Dnn::Compute::Cuda::SoftmaxCrossEntropy::CudaSoftmaxCrossEntropyOp< TPrecision, TLogits, TTargets >, Mila::Dnn::Compute::Cuda::SoftmaxCrossEntropy::CudaSoftmaxCrossEntropyOp< TensorDataType::BF16 >, and Mila::Dnn::Compute::Cuda::SoftmaxCrossEntropy::CudaSoftmaxCrossEntropyOp< TensorDataType::FP32 >.

The documentation for this class was generated from the following file:

/__w/Mila/Mila/Mila/Src/Dnn/Compute/Operations/BinaryOperation.ixx

Public Types

Public Member Functions

Static Protected Member Functions

Additional Inherited Members

Detailed Description

Member Typedef Documentation

◆ MR

◆ ParameterGradTensor

◆ ParameterTensor

◆ TensorLeftType

◆ TensorOutputType

◆ TensorRightType

Constructor & Destructor Documentation

◆ ~BinaryOperation()

Member Function Documentation

◆ asLeftTensor()

◆ asOutputTensor()

◆ asRightTensor()

◆ backward()

◆ forward()