PyTorch learning tracks

Build the workhorse optimizers byte by byte. Start with vanilla SGD and end at Adam.

The basics of working with tensors — shapes, indexing, broadcasting, and the operations you'll reach for daily.

From the classic non-linearities to modern gated activations. End with implementing a custom gradient.

The training objectives that shape every model — regression, classification, distribution-matching, and contrastive.

Linear, polynomial, regularized, and logistic — the classics every interview revisits.

Build the simplest classifiers end-to-end — binary, multi-class, and sequence.

BatchNorm, LayerNorm, GroupNorm, RMSNorm, dropout — when to use which, and how each shifts gradients.

Build convolutional networks one block at a time — from raw conv to skip connections and squeeze-excitation.

RNN, LSTM, GRU, bidirectional — the architectures that ruled NLP before transformers.

From raw text to token tensors — BPE, subword, and the embedding matrices that turn ids into vectors.

From dot products to multi-head transformers. Each step composes onto the next.

After Attention 101 — the real-world variants that actually run in modern LLMs.

Sinusoidal, learned, relative, RoPE, ALiBi — how transformers know where tokens live.

Greedy through speculative — every way to turn logits into tokens.

Adapt large models without touching most of their weights — LoRA, adapters, prefix-tuning.

From minimal training loops to gradient accumulation, EMAs, distributed primitives, and checkpoints. Train models at scale.

The numbers that tell you whether your model is actually working — accuracy, F1, AUC, perplexity, BLEU.

From autoencoders through VAEs to modern diffusion. The math behind generating images, audio, and text.