Skip to content

← JAX Stochasticity step 13 of 25

Easy Primitives

Log Acceptance Ratio

Jump to the task

Why this matters

A Metropolis-Hastings step decides whether to move from a current state x to a proposed state x'. Its ratio includes both target densities and the probability of proposing the move in each direction:

ratio = p(x') * q(x | x') / (p(x) * q(x' | x))

Here q(x' | x) is the forward proposal density, while q(x | x') is the backward density. They cancel for a symmetric proposal. For an asymmetric proposal, keeping both terms corrects for the proposal's preference for one direction.

This exercise returns the raw log ratio, not a probability. The actual acceptance probability is min(1, ratio), whose logarithm is min(0, log_ratio).

Worked mini-example

log_p_current = -2.0
log_p_proposal = -1.0
log_q_forward = -0.5
log_q_backward = -2.0

log_ratio = log_p_proposal + log_q_backward - log_p_current - log_q_forward
# -1 - 2 + 2 + 0.5 = -0.5

Although the proposed state has higher target density, the proposal correction makes its acceptance probability exp(-0.5), about 0.607. With equal forward and backward log densities instead, the raw log ratio would be 1 and the proposal would always be accepted.

Problem

Implement log_acceptance(log_p_current, log_p_proposal, log_q_forward, log_q_backward) to return:

log_p_proposal + log_q_backward - log_p_current - log_q_forward

All inputs are finite scalar log densities. Assume their arithmetic remains finite. Return a scalar number without clamping or exponentiating. Positive results are valid. No proposal or random draw is needed here.

Common pitfalls

  • Add the proposed target and backward proposal terms. Subtract the current target and forward proposal terms. Swapping proposal terms generally changes the chain's stationary distribution.
  • The target may be known only up to a normalization constant: the same additive constant in both target log densities cancels.
  • Log densities can be positive for continuous distributions; they are not log probabilities of individual points.
  • Subtracting supplied log densities avoids exponentiating tiny densities before dividing. It does not eliminate rounding or overflow in all possible floating-point arithmetic.