Mila::Dnn::Gpt2::DSReaderConfig Struct Reference

Public Attributes
size_t	max_queue_size {10}
	Maximum number of batches to queue in memory.
size_t	read_chunk_size {1024 * 1024}
	Size of chunks when reading large files (1MB default)
size_t	token_window_size {0}
	Size of token window in number of tokens (0 for auto-size)
bool	verbose_logging {false}
	Whether to output detailed performance logs.

Member Data Documentation

max_queue_size

size_t Mila::Dnn::Gpt2::DSReaderConfig::max_queue_size {10}

Maximum number of batches to queue in memory.

read_chunk_size

size_t Mila::Dnn::Gpt2::DSReaderConfig::read_chunk_size {1024 * 1024}

Size of chunks when reading large files (1MB default)

token_window_size

size_t Mila::Dnn::Gpt2::DSReaderConfig::token_window_size {0}

Size of token window in number of tokens (0 for auto-size)

verbose_logging

bool Mila::Dnn::Gpt2::DSReaderConfig::verbose_logging {false}

Whether to output detailed performance logs.

---

The documentation for this struct was generated from the following file:

• /home/runner/work/Mila/Mila/Mila/Src/Dnn/Models/Gpt2/Gpt2DatasetReader.ixx