Mila::Dnn::CosineLRScheduler Class Reference

Cosine annealing scheduler. More...

Inheritance diagram for Mila::Dnn::CosineLRScheduler:

Collaboration diagram for Mila::Dnn::CosineLRScheduler:

Public Member Functions
	CosineLRScheduler (double initial_lr, double final_lr, std::size_t total_steps)
double	getLearningRate (std::size_t step) const override
	Get the learning rate for the provided (zero-based) step.
std::string	toString () const override
	Human-readable description of the scheduler.
Public Member Functions inherited from Mila::Dnn::LearningRateScheduler
virtual	~LearningRateScheduler ()=default

Private Attributes
double	final_lr_
double	initial_lr_
std::size_t	total_steps_

Detailed Description

Cosine annealing scheduler.

Uses cosine schedule from initial_lr to final_lr over total_steps. For step >= total_steps the scheduler returns final_lr.

Formula (t in [0,1]): lr = final + 0.5*(initial - final)*(1 + cos(pi * t))

Preconditions:

• total_steps > 0
• initial_lr >= 0, final_lr >= 0

Constructor & Destructor Documentation

CosineLRScheduler()

Mila::Dnn::CosineLRScheduler::CosineLRScheduler ( double initial_lr, double final_lr, std::size_t total_steps )

inline

Member Function Documentation

getLearningRate()

double Mila::Dnn::CosineLRScheduler::getLearningRate ( std::size_t step ) const

inlineoverridevirtual

Get the learning rate for the provided (zero-based) step.

Parameters

step	Zero-based step index.

Returns

Scalar learning rate to use at this step.

Implements Mila::Dnn::LearningRateScheduler.

toString()

std::string Mila::Dnn::CosineLRScheduler::toString ( ) const

inlineoverridevirtual

Human-readable description of the scheduler.

Implements Mila::Dnn::LearningRateScheduler.

Member Data Documentation

final_lr_

double Mila::Dnn::CosineLRScheduler::final_lr_

private

initial_lr_

double Mila::Dnn::CosineLRScheduler::initial_lr_

private

total_steps_

std::size_t Mila::Dnn::CosineLRScheduler::total_steps_

private

---

The documentation for this class was generated from the following file:

• /__w/Mila/Mila/Mila/Src/Dnn/Core/LearningRateScheduler.ixx