Calibration Error

Task

Implement calibration_error(labels: list[int], probabilities: list[float], num_bins: int) -> float. Group predicted positive-class probabilities into equal-width bins and return the expected calibration error across non-empty bins.

Requirements

• Labels are binary and probabilities are positive-class probabilities in [0, 1].
• Use num_bins equal-width bins over [0, 1].
• A probability p belongs to bin floor(p * num_bins), except p = 1.0 belongs to the last bin.
• For each non-empty bin, compute abs(mean_probability - positive_rate).
• Weight each bin's gap by that bin's fraction of all examples.
• Skip empty bins and return 0.0 for empty inputs or when num_bins <= 0.

Example

labels = [1, 0, 1, 0]
probabilities = [0.9, 0.8, 0.2, 0.1]

calibration_error(labels, probabilities, num_bins=2)
# 0.35