Skip to content
← All tracks

Parameter-Efficient FT

PyTorch

Adapt large models without touching most of their weights — LoRA, adapters, prefix-tuning.

0 / 4 solved
Continue →
  1. 1. Not solved yet. LoRA Update
  2. 2. Not solved yet. Adapter Layer
  3. 3. Not solved yet. Prefix Tuning
  4. 4. Not solved yet. Train LoRA Adapter End-to-End

Check yourself

4 questions · one attempt each

These do not count toward finishing the track. They are here to catch the things that are easy to read past.

0 / 4

LoRA replaces a 4096x4096 update with two matrices of rank 8. How many trainable parameters is that? 

# full update:  W   (4096 x 4096)
# LoRA:         A (4096 x 8), B (8 x 4096)
Question 1 of 4