Expand description
Metrics engine: storage, aggregation, and query execution.
This crate is the serving/engine half of the metrics substrate
(AGPL-3.0-only): DuckDB and DataFusion backends, the query planner and
orchestrator, metric aggregation execution, and checkpointing. The public
record types, schema, producer API, and the calibration / decision-batch
stores live in the Apache-2.0 hyprstream-metrics-api crate; everything
consumers need is re-exported here so existing hyprstream_metrics::...
paths keep working.
Re-exports§
pub use checkpoint::Checkpoint;pub use checkpoint::CheckpointConfig;pub use checkpoint::CheckpointManager;pub use checkpoint::CheckpointMetadata;pub use checkpoint::RecoveryManager;pub use checkpoint::RecoveryStatus;pub use checkpoint::RegistryClient;pub use checkpoint::RegistryError;pub use query::CachedStatement;pub use query::DataFusionExecutor;pub use query::DataFusionPlanner;pub use query::ExecutorConfig;pub use query::Query;pub use query::QueryOrchestrator;pub use storage::context::context_schema;pub use storage::context::ContextRecord;pub use storage::context::ContextStore;pub use storage::context::SearchResult;pub use storage::context::DEFAULT_EMBEDDING_DIM;pub use duckdb::arrow;
Modules§
- aggregation
- Core aggregation framework for time-series data.
- checkpoint
- Checkpoint system for versioned metrics storage.
- config
- Configuration management for hyprstream-metrics library.
- error
- metrics
- Metric record types, stores, and metric-specific aggregation.
- query
- Query planning and execution module.
- storage
- Storage backends for SQL database operations.
- utils
Structs§
- Aggregate
Result - Result of an aggregation operation
- Calibration
Params - One calibration parameter row.
- Calibration
Store - Calibration parameter store over any
StorageBackend: the writer API for the P2.2 re-fit loop and the reader API for serving. - Conformal
Quantile - One coverage level → nonconformity quantile (APS/RAPS-style).
- Decision
Batch Meta - Row metadata for one stored decision batch (everything but the payload).
- Decision
Batch Store - Decision-batch store over any
StorageBackend: the write/read API for Arrow-native model output (eval harness, distillation corpora, re-fit input). - Dirichlet
Params - Dirichlet-calibration parameters (Kull et al. 2019) for high-cardinality
choice questions, where scalar temperature underfits: calibrated log-probs
are
W · ln(p) + bwithweightsrow-major K×K andbiasof length K. - GroupBy
- Grouping specification for aggregation
- Metrics
Producer - Producer of schema-v2 metric rows over any
StorageBackend. - Ordinal
Interval - One coverage level → ordinal contiguous interval half-widths, in level units, applied around the predicted level (min-CPS / RPS-based nonconformity for the score primitive).
- Version
Triple - The (schema, model, calib) version triple keying every calibration row.
Enums§
- Aggregate
Function - Generic aggregation functions that can be applied to time-series data
- Conformal
Params - Conformal parameters for one primitive. Score questions are ordinal, so their prediction sets are contiguous intervals; noul/choice use APS/RAPS-style quantiles.
- Decision
Primitive - Decision primitive a calibration row applies to (jev-1 profile superset).
- Time
Window - Time window for aggregation
Constants§
- CALIBRATION_
TABLE - Calibration parameter table name.
- DECISION_
BATCH_ TABLE - Decision batch table name.
- METRICS_
SCHEMA_ VERSION - Version of the metrics table schema emitted by
get_metrics_schema. v1: five running-window columns; v2: addslabelsandtenant_id. - METRICS_
TABLE - Primary metrics table name.
Traits§
- Storage
Backend - Storage backend trait for SQL database operations.
Functions§
- calibration_
params_ schema - Arrow schema of the
calibration_paramstable. Vector/composite params are stored as JSON in nullable VARCHAR columns (the crate’s existing convention for composite values, queryable via DuckDB JSON functions). - create_
schema_ v2_ tables - Creates all schema-v2 tables and migrates pre-v2
metricstables in place. - decision_
batch_ schema - Arrow schema of the
decision_batchestable. - mint_
batch_ id - Mint a new batch id.
- mint_
calib_ version - Mint a new calibration version id. Re-fits call this (or supply their own) and insert a new row; existing rows are never mutated.