Skip to main content

ANCHOR_TOKEN

Constant ANCHOR_TOKEN 

Source
pub const ANCHOR_TOKEN: &str;
Expand description

The learned special anchor token string. Byte-identical to [serialize::ANCHOR_PLACEHOLDER]: the placeholder in the frozen serialization is the special token’s text, so the tokenizer’s added-token matcher maps every placeholder occurrence to the anchor id.