Swiglu.Config.ixx File Reference

Configuration for the SwiGLU activation component. More...

#include <stdexcept>
#include <string>
#include <string_view>
#include <sstream>
import Serialization.Metadata;
import Dnn.ApproximationMethod;
import Dnn.ComponentConfig;
import Dnn.Component;

Classes
class	Mila::Dnn::SwigluConfig

Namespaces
namespace	Mila
	Mila main API namespace.
namespace	Mila::Dnn

Detailed Description

Configuration for the SwiGLU activation component.

Design principle (Mila-wide):

• Constructor parameters are structurally required — no sensible default exists.
• Fluent setters are reserved for optional behavioural parameters that have well-known defaults. There are no fluent overrides for constructor parameters.

SwigluConfig has no structurally required parameters — hidden_dim is determined from the input tensor shape at build time. The default constructor is correct.

Optional (fluent): inner_gelu_method (default: Tanh).

Typical usage:

// Default — Tanh approximation.
auto cfg = SwigluConfig();
 
// Explicit approximation method.
auto cfg = SwigluConfig()
    .withInnerGeluMethod( ApproximationMethod::Exact );