ML Models From Scratch ยท Medium
Principal Component Analysis
Learn deterministic principal directions and use them for dimensionality reduction and reconstruction.
Task
Implement class PCA(n_components: int). Implement principal component analysis from covariance construction through deterministic eigendecomposition, projection, explained variance, and inverse reconstruction.
Requirements
- Require n_components to be a positive integer.
- fit(x) computes and stores the feature-wise mean, then centers x without mutating the caller's array.
- Build the population covariance matrix as centered.T @ centered divided by the number of rows.
- Use a symmetric eigendecomposition and order principal directions by descending eigenvalue.
- Clamp eigenvalues to zero before reporting variance to remove tiny negative roundoff artifacts.
- Resolve eigenvector sign ambiguity by making the largest-absolute loading non-negative; an absolute-value tie uses the lowest feature index.
- Store components with shape (n_components, n_features), along with explained_variance and explained_variance_ratio.
- Compute each explained variance ratio against the sum of all covariance eigenvalues, not only the selected components.
- transform(x) centers rows with the learned mean and projects them onto components.
- inverse_transform(transformed) maps projected rows back to feature space and restores the mean.
- fit_transform(x) fits once and returns the transformed training rows.
- Assume fit receives at least one row, n_components does not exceed min(n_samples, n_features), and the data has positive total variance.
Example
model = PCA(n_components=1)
transformed = model.fit_transform(
np.array([[-2.0, 0.0], [0.0, -1.0], [0.0, 1.0], [2.0, 0.0]])
)
model.components
# array([[1.0, 0.0]])
transformed
# array([[-2.0], [0.0], [0.0], [2.0]])