Skip to content
← All PyTorch tracks

Decoding Strategies

PyTorch

Greedy through speculative — every way to turn logits into tokens.

8 exercises

Explore freely. Sign in to keep your progress.

Start track

Your learning sequence

  1. 1 Not solved yet. Greedy Decoding Practice · Easy
  2. 2 Not solved yet. Temperature Scaling Practice · Easy
  3. 3 Not solved yet. Top-k Sampling Practice · Medium
  4. 4 Not solved yet. Nucleus (Top-P) Filtering Practice · Medium
  5. 5 Not solved yet. Beam Search Practice · Hard
  6. 6 Not solved yet. Speculative Decoding Accept/Reject Practice · Hard
  7. 7 Not solved yet. Nucleus Sampling Generation Loop Practice · Hard
  8. 8 Not solved yet. Causal LM with KV Cache Generation Practice · Hard

Check yourself

4 questions · not saved while you're signed out

Use these questions to check your understanding and compare explanations. Your score does not affect track completion.

0 / 4 correct

In this two-step example, what better-scoring sequence can a beam of width two retain that greedy decoding misses?

Choose one answer to reveal the explanations. Answers are kept only while this page is open. 

first = {"A": 0.6, "B": 0.4}
second = {"A": {"x": 0.5, "y": 0.5},
          "B": {"x": 0.9, "y": 0.1}}
first_choice = max(first, key=first.get)
next_choice = max(second[first_choice], key=second[first_choice].get)
greedy_score = first[first_choice] * second[first_choice][next_choice]
# Width two retains both first-step prefixes, then ranks their extensions.
candidates = [(p * q, a + b) for a, p in first.items()
              for b, q in second[a].items()]
best_score, best_sequence = max(candidates)
print(first_choice + next_choice, f"{greedy_score:.2f}")
print(best_sequence, f"{best_score:.2f}")
Question 1 of 4