Rope.Config.ixx File Reference

Configuration for Rotary Position Embedding (RoPE) component. More...

#include <stdexcept>
#include <string>
#include <sstream>
#include <utility>
import Serialization.Metadata;
import Dnn.ComponentConfig;
import Dnn.Component;

Classes
class	Mila::Dnn::RopeConfig

Namespaces
namespace	Mila
	Mila main API namespace.
namespace	Mila::Dnn

Detailed Description

Configuration for Rotary Position Embedding (RoPE) component.

Provides construction, validation and serialization for RoPE configuration.

Design principle (Mila-wide):

• Constructor parameters are structurally required — no sensible default exists.
• Fluent setters are reserved for optional behavioural parameters that have well-known defaults. There are no fluent overrides for constructor parameters.

Required (constructor): channels, n_heads, n_kv_heads, max_seq_len. Optional (fluent): base (default 10000.0f), rotary_dim (default 0 = full head_dim).

Typical usage:

auto cfg = RopeConfig( model_dim, n_heads, n_kv_heads, max_seq_len )
    .withBase( 500000.0f );  // Llama 3 theta