Unsupervised Learning & Manifold Learning

Welcome to the definitive guide to Unsupervised Learning and Manifold Learning. When data arrives without ground truth labels, extracting latent structure, discovering underlying clusters, and modeling low-dimensional geometries requires rigorous mathematical principles rather than black-box approximations.

This curriculum derives every fundamental unsupervised algorithm from mathematical first principles and implements each from scratch in both pure Python and pure Rust without third-party dependencies.

Explore lessons from the sidebar to inspect step-by-step mathematical proofs, zero-dependency implementations, interactive zoomable diagrams, and knowledge verification checkpoints.

Curriculum Architecture at a Glance#

This course provides comprehensive mathematical foundations and production-grade implementations across the unsupervised learning lifecycle:

1. Partition-Based Clustering (Lloyd's k-Means): Voronoi tessellations, coordinate descent proofs of monotonic inertia convergence, $k$-Means++ probabilistic distance seeding ($D^2$ weighting), and computational complexity boundaries.
2. Density and Hierarchical Clustering (DBSCAN & HDBSCAN): $\varepsilon$-neighborhoods, MinPts core density connectivity, border point propagation, noise rejection, and agglomerative linkage criteria (single, complete, Ward variance minimization).
3. Probabilistic Density Estimation (Gaussian Mixture Models): Soft cluster assignments via Bayes' theorem, Expectation-Maximization (EM) derivation, Jensen's inequality for the Evidence Lower Bound (ELBO), and singularity prevention via covariance regularization.
4. Linear Dimensionality Reduction (PCA & Truncated SVD): Maximal variance projections, reconstruction error minimization, covariance eigendecomposition, Singular Value Decomposition (SVD: $X = U \Sigma V^T$), and Power Iteration for high-dimensional spectra.
5. Nonlinear Manifold Learning (t-SNE & UMAP): High-dimensional Gaussian affinities, low-dimensional Student-$t$ distributions resolving the crowding problem, Kullback-Leibler divergence gradient derivations, and fuzzy simplicial set approximations.
6. Unsupervised Anomaly Detection (Isolation Forests & LOF): Random recursive axis-aligned hyperplanes, average path length anomalies, harmonic number normalization $c(n)$, and Local Outlier Factor (LOF) local reachability densities.
7. Cluster Validation & Quantitative Evaluation: Silhouette coefficients, Davies-Bouldin index, Calinski-Harabasz score, Adjusted Rand Index (ARI), and comprehensive technical interview matrices.

Core Pedagogical Standard#

• Step-by-Step Algebraic Derivations: Every objective function, Expectation-Maximization step, and gradient update is derived algebraically step by step.
• Zero-Dependency Implementations: Every algorithm includes complete, runnable implementations in pure Python (standard library only) and pure Rust (std only).
• Interactive Visual Architecture: Explore interactive flowcharts, mathematical formulas rendered with KaTeX, expandable diagrams, and knowledge verification checkpoints.

The complete course source code and implementations are open on the Source Repository.