Skip to content
← All PyTorch tracks

Parameter-Efficient FT

PyTorch

Adapt large models without touching most of their weights — LoRA, adapters, prefix-tuning.

4 exercises

Explore freely. Sign in to keep your progress.

Start track

Your learning sequence

  1. 1 Not solved yet. LoRA Update Practice · Medium
  2. 2 Not solved yet. Adapter Layer Practice · Medium
  3. 3 Not solved yet. Prefix Tuning Practice · Hard
  4. 4 Not solved yet. Train LoRA Adapter End-to-End Practice · Medium

Check yourself

4 questions · not saved while you're signed out

Use these questions to check your understanding and compare explanations. Your score does not affect track completion.

0 / 4 correct

A frozen 4096 by 4096 weight matrix gets an adapter update A @ B, with A shaped (4096, 8) and B shaped (8, 4096). Excluding biases, how many trainable adapter parameters are there?

Choose one answer to reveal the explanations. Answers are kept only while this page is open. 

width, rank = 4096, 8
full = width * width
adapter = width * rank + rank * width
print(f"full: {full:,}")
print(f"adapter: {adapter:,}")
print(f"percentage: {100 * adapter / full:.2f}%")
Question 1 of 4