PyTorch learning tracks

The basics of working with tensors — shapes, indexing, broadcasting, and the operations you'll reach for daily.

From the classic non-linearities to modern gated activations. End with implementing a custom gradient.

The training objectives that shape every model — regression, classification, distribution-matching, and contrastive.

Build the workhorse optimizers byte by byte. Start with vanilla SGD and end at Adam.

Linear, polynomial, regularized, and logistic — the classics every interview revisits.

Build the simplest classifiers end-to-end — binary, multi-class, and sequence.

BatchNorm, LayerNorm, GroupNorm, RMSNorm, dropout — when to use which, and how each shifts gradients.

Build convolutional networks one block at a time — from raw conv to skip connections and squeeze-excitation.

RNN, LSTM, GRU, bidirectional — the architectures that ruled NLP before transformers.

From raw text to token tensors — BPE, subword, and the embedding matrices that turn ids into vectors.

From dot products to multi-head transformers. Each step composes onto the next.

The numbers that tell you whether your model is actually working — accuracy, F1, AUC, perplexity, BLEU.

After Attention 101 — the real-world variants that actually run in modern LLMs.

Sinusoidal, learned, relative, RoPE, ALiBi — how transformers know where tokens live.

Greedy through speculative — every way to turn logits into tokens.

Adapt large models without touching most of their weights — LoRA, adapters, prefix-tuning.

From minimal training loops to gradient accumulation, EMAs, distributed primitives, and checkpoints. Train models at scale.

From autoencoders through VAEs to modern diffusion. The math behind generating images, audio, and text.