CpuSoftmaxCrossEntropyOp.ixx File Reference

Fused CPU implementation of Softmax + CrossEntropy loss operation. More...

#include <memory>
#include <vector>
#include <string>
#include <stdexcept>
#include <cmath>
#include <limits>
#include <cstdint>
import Compute.CpuMemoryResource;
import Compute.MemoryResource;
import Compute.OperationRegistry;
import Dnn.ITensor;
import Dnn.TensorDataType;
import Dnn.ComponentConfig;
import Dnn.Tensor;
import Dnn.TensorHostTypeMap;
import Compute.DeviceType;
import Dnn.Components.SoftmaxCrossEntropy;
import Compute.ExecutionContext;
import Compute.BinaryOperation;
import Compute.OperationType;
import Dnn.TensorTypes;
import Dnn.TensorDataTypeTraits;
import Compute.IExecutionContext;
import Compute.OperationBase;

Classes
class	Mila::Dnn::Compute::CpuSoftmaxCrossEntropyOp< TPrecision, TLogits, TTargets >
	Fused CPU implementation of Softmax + CrossEntropy using abstract TensorDataType API. More...
class	Mila::Dnn::Compute::CpuSoftmaxCrossEntropyOpRegistrar
	Registrar for fused Softmax+CrossEntropy operation. More...

Namespaces
namespace	Mila
	Mila main API namespace.
namespace	Mila::Dnn
namespace	Mila::Dnn::Compute

Detailed Description

Fused CPU implementation of Softmax + CrossEntropy loss operation.

Combines softmax and cross-entropy into a single numerically stable operation following the ExecutionContext / TensorDataType BinaryOperation interface.

Key advantages over separate Softmax + CrossEntropy:

• Numerical stability: Uses log-sum-exp trick throughout
• Performance: Single pass, no materialized probability distribution
• Simplified gradient: dL/dlogits = softmax(logits) - one_hot(targets)
• Memory efficiency: No intermediate probability tensor exposed in API