Mathematical Foundations
Linear algebra, calculus, probability, and statistics for ML.
Derivatives and Gradients
The mathematical machinery for measuring how outputs change with inputs – the foundation of all learning algorithms.
Information Theory
Entropy, KL divergence, and mutual information – quantifying uncertainty, surprise, and the difference between distributions.
Matrix Decompositions
Eigendecomposition, SVD, and Cholesky – factoring matrices to reveal structure, compress data, and solve systems efficiently.
Maximum Likelihood Estimation
Finding the parameter values that make observed data most probable – the dominant paradigm for fitting ML models.
Norms and Distance Metrics
Measuring size and similarity in feature space – L1, L2, cosine, Mahalanobis, and when each is appropriate.
Optimization and Gradient Descent
Iteratively adjusting parameters to minimize a loss function – the engine that drives model training.
Probability Fundamentals
Random variables, distributions, Bayes’ theorem, and conditional probability – the language of uncertainty in ML.
Statistical Inference
Drawing conclusions about populations from samples – hypothesis testing, confidence intervals, and the frequentist-Bayesian divide.
Vectors and Matrices
The fundamental data structures of ML – representing data as points in high-dimensional space and transformations as matrices.