recall_at_k
Measure how many relevant items were recovered in the top k results.
Problem Set
Starter ML coding exercises.
The first batch maps out the core implementation tasks for metrics, training loops, data pipelines, retrieval, ranking, and debugging.
precision_at_k
Rank examples by score and compute relevant labels in the top k positions.
ndcg
Compute discounted cumulative gain and normalize it by the ideal ranking.
map
Average precision across ranked results and aggregate across queries.
auc
Compute pairwise ranking quality for positive and negative examples.
logloss
Score probabilistic binary predictions with numerical stability.
calibration_error
Bucket predictions and measure probability calibration error.
grouped_auc
Compute AUC per group and aggregate over uneven group sizes.
binary_cross_entropy
Implement numerically stable BCE with optional sample weights.
softmax_cross_entropy
Use log-sum-exp to compute multiclass cross entropy safely.
pairwise_ranking_loss
Compare positive and negative items with a margin-based objective.
hinge_loss
Implement max-margin binary classification loss.
focal_loss
Downweight easy examples for imbalanced binary classification.
ips_weighting
Apply inverse propensity weights for counterfactual training data.
minibatch_iterator
Yield stable mini-batches with optional shuffling and drop-last behavior.
feature_normalization
Normalize numeric features while handling constants and missing values.
categorical_encoding
Map categories to stable ids with unknown and rare value handling.
sequence_padding
Pad and truncate variable-length sequences for batched model input.
time_window_aggregation
Aggregate events over rolling historical windows without leakage.
deduplication
Remove duplicate records using keys, timestamps, and deterministic tie-breaks.
train_test_split_by_time
Split events chronologically while avoiding leakage from future data.
top_k_heap
Maintain the best k candidates from a stream using a heap.
cosine_similarity
Compute cosine similarity with zero-vector handling.
brute_force_nearest_neighbor
Search all vectors and return nearest neighbors with deterministic ordering.
inverted_index
Build a token-to-document index and retrieve matching candidates.
bm25
Score documents with term frequency, inverse document frequency, and length normalization.
two_stage_ranker
Combine candidate retrieval and a second-pass scoring interface.
compare_two_datasets
Check row counts, schemas, duplicate keys, and changed values.
detect_null_drift
Compare null rates across datasets and flag suspicious feature changes.
feature_distribution_comparison
Compare two datasets and surface meaningful feature drift.
label_distribution_by_segment
Aggregate label rates by segment and highlight skewed groups.
calibration_by_bucket
Bucket predictions and compare average score to observed label rate.
confusion_matrix_by_group
Compute confusion matrices per segment for binary classifiers.
No problems match the current filters.