JAX Performance & Distributed

jit details, memory and device control, sharding APIs (Mesh, PartitionSpec, shard_map), mixed precision. The performance and parallelism toolbox.

1. 1. ○ jit with static_argnums
2. 2. ○ jit with static_argnames
3. 3. ○ jit with donate_argnums
4. 4. ○ jit with Pytree Input
5. 5. ○ Explicit device_put
6. 6. ○ block_until_ready for Sync
7. 7. ○ Host Roundtrip via device_get
8. 8. ○ make_jaxpr Inspection
9. 9. ○ jit Retrace on Shape Change
10. 10. ○ disable_jit Context Manager
11. 11. ○ Mesh Creation
12. 12. ○ PartitionSpec Basics
13. 13. ○ NamedSharding (Replicated)
14. 14. ○ shard_map Basics
15. 15. ○ shard_map with psum Collective
16. 16. ○ shard_map vs vmap
17. 17. ○ bfloat16 Mixed Precision
18. 18. ○ Explicit Dtype Promotion
19. 19. ○ Fused Elementwise Ops
20. 20. ○ with_sharding_constraint
21. 21. ○ pjit Basics
22. 22. ○ Compilation Cache Key
23. 23. ○ compilation_cache.set_cache_dir
24. 24. ○ Shape Polymorphism (Concept)
25. 25. ○ Multi-Host JAX (Conceptual)