Evaluate without recording gradient history

If a calculation is only for reporting, you may not need to differentiate it. Disable gradient recording before computing it, rather than detaching the result afterward:

import torch

x = torch.tensor([0., 1.], requires_grad=True)
with torch.no_grad():
    metric = (x * 2).exp().sum()
detached = (x * 2).exp().sum().detach()
print(torch.equal(metric, detached))  # True
print(metric.requires_grad, detached.requires_grad)  # False False
print(metric.grad_fn, detached.grad_fn)  # None None

The numerical values and final gradient flags agree. The work done during the two forward calculations differs.

Detaching does not undo earlier recording

While gradient recording is enabled, the second expression constructs backward nodes and saves values needed to differentiate its operations. Detaching the final result removes its gradient connection; it does not undo the recording work or the memory used while the expression ran.

Autograd saves values according to each operation's backward requirements, not every intermediate indiscriminately. Saved tensors can remain alive through graph references. Once those references are no longer needed, their storage can be released or reused; allocator caching and other references also affect observed memory use.

For this expression, no-grad prevents the backward saves entirely. It still performs the arithmetic and allocates its numerical results. The memory and speed benefit depends on the computation and its lifetime.

Use the right boundary

Use no-grad around calculations that should not contribute to gradients. Use detach when a value comes from a graph you do need, but a particular consumer should not backpropagate through it, such as metric logging. No-grad does not change the existing input's requires_grad flag.

It also does not put a model into evaluation mode. model.eval() changes training-dependent module behavior, such as dropout; disabling gradient recording is a separate choice. Some evaluation procedures intentionally need gradients.

Inference mode has additional restrictions

torch.inference_mode() also removes autograd overhead such as view tracking and version-counter updates for inference tensors. Those tensors have restrictions when used later in recorded computations, especially when backward would need to save them. It is too broad to say every later use in a graph fails.

Choose the mode based on how the outputs will be used. No-grad results can be useful as constants in later training computations. This exercise only needs no-grad for a scalar reporting calculation.

def evaluate(x: torch.Tensor) -> torch.Tensor