pub const BLOCK_SIZE: usize = 256;
Number of tokens per block in the paged KV cache. 256 balances fragmentation overhead vs block reuse granularity.