Learning Tracks

Optimizers from Scratch

Build the workhorse optimizers byte by byte. Start with vanilla SGD and end at Adam.

Tensor Foundations

The basics of working with tensors — shapes, indexing, broadcasting, and the operations you'll reach for daily.

Activations

From the classic non-linearities to modern gated activations. End with implementing a custom gradient.

Loss Functions

The training objectives that shape every model — regression, classification, distribution-matching, and contrastive.

Regression from Scratch

Linear, polynomial, regularized, and logistic — the classics every interview revisits.

Classifiers from Scratch

Build the simplest classifiers end-to-end — binary, multi-class, and sequence.

Normalization

BatchNorm, LayerNorm, GroupNorm, RMSNorm, dropout — when to use which, and how each shifts gradients.

CNNs from Scratch

Build convolutional networks one block at a time — from raw conv to skip connections and squeeze-excitation.

Recurrent Networks

RNN, LSTM, GRU, bidirectional — the architectures that ruled NLP before transformers.

Tokenization & Embeddings

From raw text to token tensors — BPE, subword, and the embedding matrices that turn ids into vectors.

Attention 101

From dot products to multi-head transformers. Each step composes onto the next.

Attention Variants

After Attention 101 — the real-world variants that actually run in modern LLMs.

Position Encodings

Sinusoidal, learned, relative, RoPE, ALiBi — how transformers know where tokens live.

Decoding Strategies

Greedy through speculative — every way to turn logits into tokens.

Parameter-Efficient FT

Adapt large models without touching most of their weights — LoRA, adapters, prefix-tuning.

Production ML — Training Stack

From minimal training loops to gradient accumulation, EMAs, distributed primitives, and checkpoints. Train models at scale.

Metrics & Evaluation

The numbers that tell you whether your model is actually working — accuracy, F1, AUC, perplexity, BLEU.

Generative Models

From autoencoders through VAEs to modern diffusion. The math behind generating images, audio, and text.