Calibration by Bucket

Task

Implement calibration_by_bucket(labels: np.ndarray, probabilities: np.ndarray, num_bins: int) -> np.ndarray. Debug the bucketed-calibration diagnostic so it returns a stable per-bin table of counts, mean confidence, empirical accuracy, and signed calibration gap.

Requirements

• labels is a one-dimensional array of binary values (0 or 1), probabilities is a one-dimensional array of finite positive-class probabilities, and the arrays have equal length.
• Every probability is in the closed interval [0, 1], and num_bins is a positive integer.
• Use equal-width bins [b / num_bins, (b + 1) / num_bins) for b < num_bins - 1; the final bin is [(num_bins - 1) / num_bins, 1] and therefore includes probability 1.
• Equivalently, assign probability p to min(floor(p * num_bins), num_bins - 1). A probability exactly equal to an internal boundary belongs to the bin on its right.
• Return a floating-point array with exactly num_bins rows and four columns in this order: count, mean_confidence, empirical_accuracy, signed_gap.
• count is the number of examples in the bin, mean_confidence is the mean positive-class probability, and empirical_accuracy is the mean binary label in that bin.
• signed_gap is empirical_accuracy - mean_confidence. Positive values indicate under-confidence and negative values indicate over-confidence.
• Represent every empty bin by [0.0, 0.0, 0.0, 0.0]; never omit empty bins or produce NaN.
• Empty labels and probabilities are valid and return a num_bins by 4 all-zero array. Do not round results or mutate either input.

Example

calibration_by_bucket(
    np.array([1, 1, 0, 0]),
    np.array([0.1, 0.4, 0.6, 0.9]),
    2,
)
# array([[2.0, 0.25, 1.0, 0.75], [2.0, 0.75, 0.0, -0.75]])