JAX Autodiff

Forward and reverse mode, custom derivatives, gradient checkpointing, per-example gradients. The autodiff toolbox.

1. 1. ○ jvp Basics
2. 2. ○ jacfwd vs jacrev
3. 3. ○ jvp for Sensitivity Analysis
4. 4. ○ vjp Basics
5. 5. ○ Jacobian via Batched vjp
6. 6. ○ grad vs vjp
7. 7. ○ Hessian of a Quadratic
8. 8. ○ HVP via grad-of-grad
9. 9. ○ HVP via jvp-of-grad
10. 10. ○ Custom VJP: Stable log1pexp
11. 11. ○ Custom JVP: Clip with Pass-through Gradient
12. 12. ○ Custom VJP: Implicit Function Theorem
13. 13. ○ stop_gradient: Target Network
14. 14. ○ Straight-Through Estimator
15. 15. ○ Gradient Checkpointing: Basics
16. 16. ○ Checkpoint with Save Policy
17. 17. ○ Checkpointed Deep Stack via scan
18. 18. ○ Per-Example Gradients via vmap(grad(...))
19. 19. ○ vmap(grad) vs grad(sum(vmap))
20. 20. ○ Microbatched Gradient Accumulation via scan
21. 21. ○ jax.linearize Primitive
22. 22. ○ Jacobian via Mixed-Mode (jvp+vjp)
23. 23. ○ Higher-Order custom_vjp
24. 24. ○ Saved Residuals in custom_vjp
25. 25. ○ Grad through stop_gradient