Optimizers from Scratch

Build the workhorse optimizers byte by byte. Start with vanilla SGD and end at Adam.

1. 1. ○ Implement Gradient Descent Step
2. 2. ○ Implement Momentum Update
3. 3. ○ Implement Adam Optimizer Step
4. 4. ○ Train with Adam End-to-End