Skip to main content

Module selective

Module selective 

Source
Expand description

Selective prediction: risk–coverage analysis over confidence-thresholded answers.

Abstention is a first-class answer value in the jev-1 profile (null = abstained), and selective accuracy is the metric that prices it: accuracy among the answers a threshold policy would have kept, as a function of coverage. The curve is computed exactly (every distinct threshold), not sampled.

Structs§

RiskCoverage
One point on the risk–coverage curve.

Functions§

accuracy_at_coverage
Accuracy at the largest coverage <= target_coverage (step semantics: a threshold policy realizes only the curve’s own points). Returns None if the target is below the smallest realized coverage.
risk_coverage_curve
Exact risk–coverage curve: sort by confidence descending; at each distinct threshold, coverage = kept fraction and accuracy = mean correctness among kept.
selective_auc
Area under the risk–coverage curve (coverage on x, accuracy on y), trapezoidal over the realized points with the left edge extended to coverage 0 at the first point’s accuracy. Higher is better; 1.0 is an oracle ranker.