Skip to content

Transformer practice problems in JAX

Difficulty:
Category:
Tag:

Showing 26 of 26

Status Title Difficulty
Adapter Layer Medium
ALiBi Position Bias Medium
Causal LM Forward Pass Medium
Causal Self-Attention Block Medium
Cross Attention Medium
Cross-Attention Block Medium
Encoder-Decoder Transformer Forward Pass Hard
GELU Activation Easy
Grouped-Query Attention Hard
Implement Layer Normalization Medium
Implement Positional Encoding Hard
Implement Scaled Dot-Product Attention Hard
KV Cache for Autoregressive Decoding Hard
Learned Absolute Position Embedding Easy
MLM Forward Pass Medium
Multi-Head Attention Block Hard
Multi-Query Attention Medium
Patch Embedding with CLS Token Medium
Rotary Position Embeddings Hard
Self-Attention Layer Hard
Tied Input/Output Embeddings Medium
Transformer Block with RoPE Hard
Transformer Decoder Block Hard
Transformer Encoder Block Hard
ViT Classification Forward Hard
ViT Encoder Block Medium