Mila::Dnn::BufferedTokenStreamer< Sink, BufSize > Class Template Reference

Buffers BufSize tokens before forwarding a contiguous span to Sink. More...

Public Member Functions
	BufferedTokenStreamer (Sink &sink)
void	finish ()
void	flush ()
void	operator() (int32_t tok)

Private Attributes
std::array< int32_t, BufSize >	buf_ {}
size_t	count_ = 0
Sink &	sink_

Detailed Description

template<class Sink, size_t BufSize = 8>
requires TokenSink<Sink>
class Mila::Dnn::BufferedTokenStreamer< Sink, BufSize >

Buffers BufSize tokens before forwarding a contiguous span to Sink.

Reduces per-token overhead when Sink involves cross-thread posting or I/O. Satisfies TokenStreamer via operator()(int32_t). Call finish() after generation completes to flush any partial buffer and invoke Sink::on_finish() if present.

Sink is held by reference; the caller must ensure Sink outlives this object.

Template Parameters

BufSize	Tokens to buffer before each flush. Defaults to 8.
Sink	A TokenSink callable.

Constructor & Destructor Documentation

BufferedTokenStreamer()

template<class Sink, size_t BufSize = 8>

Mila::Dnn::BufferedTokenStreamer< Sink, BufSize >::BufferedTokenStreamer ( Sink & sink )

inlineexplicit

Member Function Documentation

finish()

template<class Sink, size_t BufSize = 8>

void Mila::Dnn::BufferedTokenStreamer< Sink, BufSize >::finish ( )

inline

Here is the call graph for this function:

flush()

template<class Sink, size_t BufSize = 8>

void Mila::Dnn::BufferedTokenStreamer< Sink, BufSize >::flush ( )

inline

Here is the caller graph for this function:

operator()()

template<class Sink, size_t BufSize = 8>

void Mila::Dnn::BufferedTokenStreamer< Sink, BufSize >::operator() ( int32_t tok )

inline

Here is the call graph for this function:

Member Data Documentation

buf_

template<class Sink, size_t BufSize = 8>

std::array<int32_t, BufSize> Mila::Dnn::BufferedTokenStreamer< Sink, BufSize >::buf_ {}

private

count_

template<class Sink, size_t BufSize = 8>

size_t Mila::Dnn::BufferedTokenStreamer< Sink, BufSize >::count_ = 0

private

sink_

template<class Sink, size_t BufSize = 8>

Sink& Mila::Dnn::BufferedTokenStreamer< Sink, BufSize >::sink_

private

---

The documentation for this class was generated from the following file:

• /__w/Mila/Mila/Mila/Src/Dnn/Core/TokenStreamer.ixx