ML Models From Scratch ยท Medium
Logistic Regression Trainer
Train a numerically stable binary classifier with gradient descent.
Task
Implement class LogisticRegression(initial_weights: np.ndarray, initial_bias: float, learning_rate: float). Implement an end-to-end binary logistic regression trainer with stable loss, explicit gradients, training state, probabilities, and class predictions.
Requirements
- Copy the initial weights so training does not mutate the caller's array.
- forward(x) returns logits x @ weights + bias, not probabilities.
- loss(logits, targets) computes numerically stable mean binary cross-entropy with logits.
- backward(x, logits, targets) stores weight_grad and bias_grad without using autograd.
- step() applies one gradient-descent update using learning_rate.
- fit(x, targets, epochs) records the loss before each update in loss_history.
- predict_proba(x) returns sigmoid probabilities and predict(x) uses probability >= 0.5.
Example
model = LogisticRegression(
np.array([0.0]),
initial_bias=0.0,
learning_rate=0.5,
)
model.fit(
np.array([[-2.0], [-1.0], [1.0], [2.0]]),
np.array([0.0, 0.0, 1.0, 1.0]),
epochs=2,
)
model.predict(np.array([[-2.0], [-1.0], [1.0], [2.0]]))
# array([0, 0, 1, 1])