LayerNorm.Config.ixx File Reference

Configuration for Layer Normalization component. More...

#include <stdexcept>
#include <string>
#include <sstream>
#include <optional>
#include <utility>
import Dnn.TensorTypes;
import Serialization.Metadata;
import Dnn.ComponentConfig;
import Dnn.Component;

Classes
class	Mila::Dnn::LayerNormConfig

Namespaces
namespace	Mila
	Mila main API namespace.
namespace	Mila::Dnn

Detailed Description

Configuration for Layer Normalization component.

Design principle (Mila-wide):

• Constructor parameters are structurally required — no sensible default exists.
• Fluent setters are reserved for optional behavioural parameters that have well-known defaults. There are no fluent overrides for constructor parameters.

LayerNormConfig supports two mutually exclusive normalization modes selected by constructor overload:

• Shape mode: LayerNormConfig( shape_t ) — normalize over a trailing shape.
• Axis mode: LayerNormConfig( int64_t ) — normalize over a single axis.

The overloads are unambiguous — shape_t and int64_t cannot collide.

Typical usage:

// Shape mode (most common for transformers)
auto cfg = LayerNormConfig( shape_t{ model_dim } )
    .withEpsilon( 1e-5f )
    .withBias( false );
 
// Axis mode
auto cfg = LayerNormConfig( int64_t{ -1 } )
    .withBias( false );