All problems
MediumML Models From Scratchv2026-06-29

ML Models From Scratch ยท Medium

Principal Component Analysis

Learn deterministic principal directions and use them for dimensionality reduction and reconstruction.

Task

Implement class PCA(n_components: int). Implement principal component analysis from covariance construction through deterministic eigendecomposition, projection, explained variance, and inverse reconstruction.

Requirements

  • Require n_components to be a positive integer.
  • fit(x) computes and stores the feature-wise mean, then centers x without mutating the caller's array.
  • Build the population covariance matrix as centered.T @ centered divided by the number of rows.
  • Use a symmetric eigendecomposition and order principal directions by descending eigenvalue.
  • Clamp eigenvalues to zero before reporting variance to remove tiny negative roundoff artifacts.
  • Resolve eigenvector sign ambiguity by making the largest-absolute loading non-negative; an absolute-value tie uses the lowest feature index.
  • Store components with shape (n_components, n_features), along with explained_variance and explained_variance_ratio.
  • Compute each explained variance ratio against the sum of all covariance eigenvalues, not only the selected components.
  • transform(x) centers rows with the learned mean and projects them onto components.
  • inverse_transform(transformed) maps projected rows back to feature space and restores the mean.
  • fit_transform(x) fits once and returns the transformed training rows.
  • Assume fit receives at least one row, n_components does not exceed min(n_samples, n_features), and the data has positive total variance.

Example

model = PCA(n_components=1)
transformed = model.fit_transform(
    np.array([[-2.0, 0.0], [0.0, -1.0], [0.0, 1.0], [2.0, 0.0]])
)
model.components
# array([[1.0, 0.0]])
transformed
# array([[-2.0], [0.0], [0.0], [2.0]])
solution.pySign in to save
Public tests run locally in your browser.
Run your code to see public test results.