Expand description
OpenTelemetry token-burn metering for inference.
Token burn — prompt tokens consumed plus generated tokens produced per request — is accounting- and ops-relevant, so it is emitted as OpenTelemetry metrics rather than process logs (#1253, #1261). The authoritative spend ledger lives in #1264; the instruments here are the metering/ops signal.
§Instruments
inference_tokens_total—Counter<u64>(unit{token}, UCUM custom-unit form), cumulative tokens consumed, with attributeskind=prompt|generated,model, and an opaque keyed-hash tenant id.inference_request_tokens—Histogram<u64>(unit{token}), total tokens (prompt + generated) per request, withmodel/tenantattributes only.inference_speculative_tokens_total—Counter<u64>(unit{token}), MTP self-speculative draft outcomes, with attributekind=accepted|rejectedplusmodel/ opaque tenant. Integer draft counts only — never token IDs or text.
§Security
The raw request subject is never used as a metric attribute: metrics are
aggregatable across tenants and must not carry tenant-identifying content. The
tenant is reduced to an opaque keyed hash via opaque_tenant_id before it
becomes an attribute. Prompt text, token text, generated text, and token IDs
are likewise never in metrics — only integer counts. See the no-leak canary
test in crate::runtime::torch_engine which guards the same property for
logs.
§Platform gating
The opentelemetry dependency family is not wasm32-compatible, so the crates
are target-gated in Cargo.toml and the instruments here compile only for
all(feature = "otel", not(target_arch = "wasm32")). Everywhere else a no-op
TokenBurnMeter keeps the call sites compiling — the wasm/browser client
pays no metric cost and non-otel builds meter nothing.
Structs§
- Token
Burn Meter - Token-burn instruments built from an OTel [
Meter].
Functions§
- global_
meter - Shared process-global token-burn instruments.
- opaque_
tenant_ id - Reduce a raw tenant subject to an opaque 64-bit id for metric attrs and tenant-safe logs.