TokenStreamer.ixx File Reference

Token streaming abstractions for autoregressive generation. More...

#include <span>
#include <cstdint>
#include <concepts>
#include <array>

Classes
class	Mila::Dnn::BufferedTokenStreamer< Sink, BufSize >
	Buffers BufSize tokens before forwarding a contiguous span to Sink. More...

Namespaces
namespace	Mila
	Mila main API namespace.
namespace	Mila::Dnn

Concepts
concept	Mila::Dnn::TokenSink
	Satisfied by any callable accepting a span of decoded tokens.
concept	Mila::Dnn::TokenStreamer
	Satisfied by any callable accepting a single decoded token.

Detailed Description

Token streaming abstractions for autoregressive generation.

Provides TokenStreamer (per-token callback), TokenSink (batched span callback), and BufferedTokenStreamer (batches tokens before forwarding to a TokenSink).