Common attributes for neural network operations. More...
Public Member Functions | |
| OperationAttributes ()=default | |
| Default constructor with default values. | |
| OperationAttributes (int64_t axisVal, float epsilonVal=1e-5f, float temperatureVal=1.0f) | |
| Constructor with common attributes. | |
| size_t | customPropertyCount () const |
| Get the number of custom properties. | |
| template<typename T > | |
| T | get (const std::string &key, const T &defaultValue) const |
| Helper method to get values from the propsMap with type safety. | |
| bool | has (const std::string &key) const |
| Check if a custom property exists in the propsMap. | |
| template<typename T > | |
| bool | isType (const std::string &key) const |
| Check if a property exists and is of the expected type. | |
| bool | remove (const std::string &key) |
| Remove a custom property from the propsMap. | |
| template<typename T > | |
| void | set (const std::string &key, const T &value) |
| Helper method to set a value in the propsMap. | |
| std::string | toString () const |
| Serialize attributes to a string representation. | |
Public Attributes | |
| int64_t | axis = -1 |
| Axis along which the operation is applied. | |
| float | clipThreshold = std::numeric_limits<float>::max() |
| Gradient clipping threshold. | |
| float | dropoutProb = 0.0f |
| Dropout probability. | |
| float | epsilon = 1e-5f |
| Small constant added for numerical stability. | |
| int | num_heads = 0 |
| Number of attention heads for operations like MultiHeadAttention. | |
| std::unordered_map< std::string, std::any > | propsMap |
| Extensible property storage for operation-specific attributes. | |
| float | temperature = 1.0f |
| Temperature parameter for Softmax-like operations. | |
| bool | trainingMode = false |
| Indicates whether the operation is in training mode. | |
Detailed Description
Common attributes for neural network operations.
This structure provides a centralized way to manage operation attributes, combining frequently used properties as direct members for efficiency and an extensible map for less common properties.
Constructor & Destructor Documentation
◆ OperationAttributes() [1/2]
|
default |
Default constructor with default values.
◆ OperationAttributes() [2/2]
|
inline |
Constructor with common attributes.
- Parameters
-
axisVal The axis parameter value. epsilonVal The epsilon parameter value. temperatureVal The temperature parameter value.
Member Function Documentation
◆ customPropertyCount()
|
inline |
Get the number of custom properties.
- Returns
- size_t The number of properties in the propsMap.
◆ get()
|
inline |
Helper method to get values from the propsMap with type safety.
- Template Parameters
-
TElementType The expected type of the property.
- Parameters
-
key The property key to retrieve. defaultValue The default value to return if the property doesn't exist or has the wrong type.
- Returns
- TElementType The property value or defaultValue if not found.
◆ has()
|
inline |
Check if a custom property exists in the propsMap.
- Parameters
-
key The property key to check.
- Returns
- bool True if the property exists.
◆ isType()
|
inline |
Check if a property exists and is of the expected type.
- Template Parameters
-
TElementType The expected property type.
- Parameters
-
key The property key to check.
- Returns
- bool True if the property exists and is of type TElementType.
◆ remove()
|
inline |
Remove a custom property from the propsMap.
- Parameters
-
key The property key to remove.
- Returns
- bool True if the property was found and removed.
◆ set()
|
inline |
Helper method to set a value in the propsMap.
- Template Parameters
-
TElementType The type of the property to set.
- Parameters
-
key The property key. value The property value.
◆ toString()
|
inline |
Serialize attributes to a string representation.
Useful for debugging, logging, and potentially for simple serialization.
- Returns
- std::string A string representation of the attributes.
Member Data Documentation
◆ axis
| int64_t Mila::Dnn::Compute::OperationAttributes::axis = -1 |
◆ clipThreshold
| float Mila::Dnn::Compute::OperationAttributes::clipThreshold = std::numeric_limits<float>::max() |
Gradient clipping threshold.
Used to prevent exploding gradients during training by clipping gradient values that exceed this threshold.
◆ dropoutProb
| float Mila::Dnn::Compute::OperationAttributes::dropoutProb = 0.0f |
◆ epsilon
| float Mila::Dnn::Compute::OperationAttributes::epsilon = 1e-5f |
Small constant added for numerical stability.
Used by operations like LayerNorm, BatchNorm, and some activation functions to prevent division by zero or to stabilize gradients.
◆ num_heads
| int Mila::Dnn::Compute::OperationAttributes::num_heads = 0 |
Number of attention heads for operations like MultiHeadAttention.
This is a common attribute for attention-based operations.
◆ propsMap
| std::unordered_map<std::string, std::any> Mila::Dnn::Compute::OperationAttributes::propsMap |
Extensible property storage for operation-specific attributes.
Allows storing arbitrary attributes that aren't common enough to warrant direct member variables.
◆ temperature
| float Mila::Dnn::Compute::OperationAttributes::temperature = 1.0f |
Temperature parameter for Softmax-like operations.
Controls the "peakiness" of probability distributions. Higher values produce more uniform distributions, while lower values make distributions more peaked.
◆ trainingMode
| bool Mila::Dnn::Compute::OperationAttributes::trainingMode = false |
Indicates whether the operation is in training mode.
Some operations like BatchNorm and Dropout behave differently during training versus inference.
The documentation for this struct was generated from the following file:
- /home/runner/work/Mila/Mila/Mila/Src/Dnn/Compute/Operations/OperationAttributes.ixx
