Batch 04: Mathematics for AI | Codebitbybit × Tavro Studio

You know how to code. Now learn why your code works. Linear Algebra, Calculus, and Probability are not prerequisites to learn before AI — they are AI, written in the language of mathematics. Every weight update, attention score, and loss function has an equation behind it. This batch makes you fluent in all of them.

You've Come Far. This Is the Turn.

Batches 01–03 gave you history, Python, and DSA. Batch 04 is the mathematical engine that makes everything in Batches 07–11 (ML, Deep Learning, LLMs) fully understandable — not as a black box, but from first principles.

The Mathematical Foundations of AI

Every AI technique roots in one or more of these three pillars. Understanding them isn't optional for a serious AI engineer — it's what separates those who use tools from those who build them.

Vectors, matrices, and tensors are how AI stores and transforms information. Every neural network forward pass is a sequence of matrix multiplications. Embeddings, attention, SVD compression — all linear algebra.

Derivatives measure how functions change — and gradient descent uses derivatives to minimize loss. Backpropagation is the chain rule of calculus applied to neural networks. Optimizers are calculus algorithms.

AI models are probability machines. Softmax outputs are probability distributions. Loss functions are probabilistic. Bayes' theorem underlies everything from spam filters to diffusion models. Data itself is statistical.

Reading Mathematical Notation

Chapter 1 (3 episodes, ~1.2 hrs) demystifies the symbols. Every Greek letter, every operator — explained with context. After Chapter 1, math papers stop looking foreign.

Sigma — Sum

Σᵢ xᵢ = x₁+x₂+...+xₙ

Pi — Product

∏ᵢ pᵢ = joint probability

Partial Derivative

∂f/∂x — change in f w.r.t. x

Nabla — Gradient

∇f = vector of all partials

Element Of

x ∈ ℝⁿ — x is in n-dim space

Expected Value

𝔼[X] = ∑ xᵢP(xᵢ)

Proportional To

P(A|B) ∝ P(B|A)P(A)

Norm

‖𝐯‖₂ = √(v₁²+v₂²+...)

This is the gradient of mean squared error loss with respect to weight w. By end of Chapter 1, this sentence makes complete sense. That is the goal of the notation chapter.

Linear Algebra — The Language of Data

Every piece of data in AI — an image, a sentence, a sound clip — becomes a vector. Every layer of a neural network is a matrix transformation. PCA, SVD, embeddings, attention — pure linear algebra. This is the most important mathematical pillar for AI engineers.

Scalars (single numbers), vectors (ordered lists of numbers), matrices (2D grids of numbers), and tensors (n-dimensional arrays). The hierarchy from Python float → NumPy 1D array → 2D array → 3D tensor. We animate what each structure looks like in memory — connecting to Batch 01's memory chapter.

Vector addition and scalar multiplication (geometrically animated), dot product formula and its geometric meaning (projection), vector norms (L1, L2, Linf), unit vectors, cosine similarity as normalized dot product, and cross product. Every concept demonstrated with NumPy, then animated geometrically in 2D and 3D.

Matrix addition, scalar multiplication, matrix-matrix multiplication (shape rules!), transpose, identity matrix, matrix inverse (when it exists), determinant, and trace. We animate every matrix multiply as inner products between rows and columns. Then implement a neural network forward pass as three matrix multiplies in NumPy.

Matrices as functions that transform space (rotation, scaling, shearing, reflection, projection), the column space and null space, rank and linear independence, visualizing transforms in 2D and 3D animations, and why "neural network layers are nonlinear transformations stacked on linear transformations."

What eigenvectors and eigenvalues mean geometrically (directions that only get scaled, not rotated), computing them from the characteristic polynomial, eigendecomposition 𝐀=𝐐𝚲𝐐⁻¹, symmetric matrices have orthogonal eigenvectors, and the power iteration method. Animated with 2D transformations showing the eigenvectors staying fixed in direction.

SVD factorization A = U𝚺Vᵀ, geometric interpretation (three simple transforms), singular values as importance ranking, low-rank approximation for compression, relationship to eigendecomposition, and computing SVD with NumPy. We compress a grayscale image using rank-5, rank-20, rank-50 SVD and watch quality improve.

Principal Component Analysis from scratch: center the data, compute covariance matrix, find eigenvectors, project data onto top-k components. Visualizing 2D → 1D and 3D → 2D reduction. Explained variance ratio, scree plot, and the curse of dimensionality. Implemented in NumPy and compared to sklearn PCA.

The complete NumPy linear algebra toolkit: np.dot, @, np.linalg.eig/svd/solve/norm/det/inv, broadcasting for batch matrix ops, einsum notation (the most powerful NumPy function), and building a complete neural network forward pass using only NumPy. Performance comparison: pure Python loops vs NumPy vs PyTorch.

Calculus & Optimization — Following the Gradient Downhill

Training a neural network means minimizing a loss function — and calculus tells us which direction to move. Derivatives, the chain rule, and gradient descent are not just mathematical curiosities. They are the algorithms that make every AI model learn. Backpropagation is the chain rule, automated.

What a function is (input → output mapping, like a Python function), limits intuitively (approaching without reaching), left and right limits, continuity (no jumps or holes), and why AI activation functions are designed to be continuous and differentiable. We graph ReLU, sigmoid, and tanh — and identify their continuity properties.

The limit definition of a derivative (watching h shrink to 0 in slow animation), differentiation rules (power rule, product rule, quotient rule), common derivatives (polynomials, exp, log, trig), numerical differentiation vs symbolic, and why d/dx[eˣ]=eˣ is special. We compute d/dx[sigmoid(x)] — used in every logistic regression.

Chain rule for composite functions (d/dx[f(g(x))] = f'(g(x))·g'(x)), chaining multiple levels deep, multivariate chain rule, and the full manual backpropagation derivation for a 2-layer network — computing ∂L/∂W₁, ∂L/∂W₂ step by step. This is the episode that makes PyTorch's .backward() fully transparent.

Partial derivatives ∂f/∂x (treating all other variables as constants), the gradient as a vector of all partial derivatives, gradient of scalar functions (points uphill), Jacobian matrix of vector functions, Hessian matrix (second-order, used in Newton's method), and directional derivatives. Visualized as 3D loss surfaces with gradient arrows.

Gradient descent intuition (ball rolling downhill on a loss surface), learning rate α (step size), batch vs mini-batch vs stochastic GD, the convexity assumption, local vs global minima, saddle points, gradient explosion/vanishing, learning rate schedules (warmup, cosine decay), and why SGD noise can help escape local minima. Fully animated loss curves.

SGD with momentum (exponential moving average of gradients), Nesterov momentum (look-ahead gradient), AdaGrad (adaptive learning rate per parameter), RMSProp (fix AdaGrad decay), Adam (momentum + RMSProp combined) — each derived from first principles with the math shown. We compare convergence speed on a simulated loss surface.

Definite integrals as area under a curve (Riemann sums animated), the fundamental theorem of calculus (integration is the reverse of differentiation), common antiderivatives, and why probability density functions must integrate to 1. We compute the area under the Gaussian curve — and discover it equals exactly 1 (proof why it's a valid distribution).

Probability & Statistics — The Language of Uncertainty

AI models don't output certainties — they output probabilities. Every softmax is a probability distribution. Every loss function is a probabilistic measure. Bayes' theorem, information theory, and maximum likelihood estimation are the statistical engines powering modern AI. This chapter makes uncertainty mathematical and beautiful.

Sample spaces, events, Kolmogorov's three axioms of probability, complement rule, addition rule, multiplication rule (for independent events), conditional probability P(A|B), and the total probability theorem. Animated with coin flips, dice rolls, and card draws — building intuition before formalism.

Bayes' theorem derived and explained, prior/likelihood/posterior/evidence, Bayesian updating with sequential evidence (how beliefs change), naive Bayes classifier (built from scratch), and the Bayesian vs frequentist interpretation of probability. Classic examples: medical test false positives, spam filter, treasure hunter analogy.

Discrete distributions (Bernoulli, Binomial, Categorical/Multinomial, Poisson), continuous distributions (Uniform, Gaussian/Normal, Exponential), PMF vs PDF vs CDF, sampling from distributions, and the Central Limit Theorem. Every distribution plotted and animated with scipy.stats — showing which one each AI application uses.

Expected value (weighted average), variance (spread), standard deviation, covariance and correlation, the linearity of expectation, Jensen's inequality, and why these matter for AI: computing mean loss, understanding model uncertainty, and why batch normalization subtracts the mean and divides by std.

Likelihood function, log-likelihood (for numerical stability), MLE as an optimization problem, deriving the MLE for Gaussian parameters (mean and variance), MLE for logistic regression, and the key insight: minimizing cross-entropy loss IS maximizing log-likelihood. This is why cross-entropy is used everywhere in classification AI.

Shannon entropy H(p) as average surprise, binary entropy, cross-entropy H(p,q) as average bits using wrong distribution, the relation CE = H(p) + KL(p||q), KL divergence as a measure of distribution distance (not symmetric!), mutual information, and why KL divergence appears in VAEs, RL policy gradient, and fine-tuning.

Bias (systematic error from wrong assumptions), variance (sensitivity to training data fluctuations), the bias-variance decomposition of MSE, the tradeoff (reducing one increases the other), regularization as variance reduction, ensemble methods as variance reducers, and double descent (why very large models break the classic tradeoff).

Null and alternative hypotheses, p-values (correctly interpreted!), t-tests, confidence intervals, statistical significance vs practical significance, A/B testing methodology, paired vs independent tests, and multiple hypothesis correction (Bonferroni). How AI teams decide if model A is genuinely better than model B — not just lucky sampling.

Math in AI Practice — Equations Made Code

Four closing episodes that apply all three pillars directly to AI. We derive the math of a forward pass, backpropagation, transformer attention, and then implement a complete neural network using only NumPy and the mathematics from this batch. No PyTorch. Pure math → pure Python → working AI.

Walk through a 3-layer neural network equation by equation: input → linear transform → activation → linear transform → softmax output → cross-entropy loss. Every symbol defined. Every matrix dimension tracked. Every numpy operation named. By end: you can write and trace any feedforward network manually on paper.

Complete, step-by-step derivation of backpropagation for a 2-layer network: compute output gradient, propagate backwards through softmax+CE, through layer 2 weights, through activation, through layer 1 weights. Every ∂ term computed explicitly. Then vectorized with matrix notation. This is the single most important derivation in all of deep learning education.

The Query-Key-Value attention framework: what Q, K, V matrices represent, why QKᵀ computes similarity scores (dot product!), why we scale by √dₖ (variance control), softmax as probability normalization, the weighted sum of values, and multi-head attention as parallel attention from different "perspectives." Implemented in NumPy.

Build a complete 3-layer MLP from mathematical equations: initialize weights (Xavier), forward pass (matrix multiply + ReLU), compute cross-entropy loss, backprop (chain rule), Adam optimizer step — all in NumPy. Train on MNIST digit classification. Reach 97%+ accuracy. 250 lines of math-first Python. No PyTorch. No sklearn. Just the math you learned in Batch 04.

What You'll Know Cold

Concrete, testable mathematical skills that every serious AI engineer needs — and that make every batch from 07 onward fully comprehensible from first principles.

Read & Write Matrix Equations

See W @ x + b and immediately know the shapes, the computation, and what it represents geometrically in a neural network.

Compute Gradients Manually

Given a loss function, apply the chain rule layer by layer to compute ∂L/∂W — without PyTorch. Understand what .backward() does.

Interpret Every Loss Function

Cross-entropy, KL divergence, MSE, binary cross-entropy — know what each measures probabilistically and why it's used for a given task.

Understand the Attention Equation

Read Softmax(QKᵀ/√dₖ)V and explain every symbol, every operation, and why each design choice was made.

Think Probabilistically About AI

Know that model outputs are probability distributions. Apply Bayes' theorem to understand RLHF, diffusion, and model uncertainty.

Build Any Model From Scratch

Implement a training loop — forward, loss, backward, optimizer step — in pure NumPy. Achieve competitive accuracy without any ML framework.

Batch 04 Completes the Foundation Layer

After Batch 04, you have Python, DSA, and Math. Batches 05–06 add systems knowledge. Batches 07–11 are where everything you've built is applied to AI.

Batch 01–03

CS · Python · DSA

Batch 04: Math

Linear Algebra · Calculus · Probability

Batch 05: SQL

Databases for AI datasets

Networking

APIs, REST, AI inference

Machine Learning

Math applied to models

Deep Learning

PyTorch + full architecture

Generative AI

LLMs · Diffusion · RAG

AI Products

Build & ship with AI

Math Is Not the Barrier . It's the Key.

30 episodes. 5 chapters. One NumPy neural network that achieves 97% accuracy. From "what is a vector?" to implementing the attention equation that powers every LLM — free, animated, and built for beginners with ambition.