Register a sequence of model layers

A Python list can hold layers and call them successfully, while keeping their parameters out of the parent module's parameter list:

import torch
import torch.nn as nn

class Net(nn.Module):
    def __init__(self, depth):
        super().__init__()
        self.layers = [nn.Linear(4, 4) for _ in range(depth)]

    def forward(self, x):
        for layer in self.layers:
            x = layer(x)
        return x

net = Net(3)
print(net(torch.ones(4)).shape)       # torch.Size([4])
print(len(list(net.parameters())))   # 0
print(len(net.state_dict()))         # 0

Each Linear has a weight and a bias, so we wanted six parameter tensors. The layers exist, but they are not registered as children of Net.

What registration changes

Assigning a module directly to an attribute registers it as a child module. Assigning a plain list does not recursively register the modules inside it. Use nn.ModuleList when you need an indexed collection of registered layers:

# Replace the assignment inside Net.__init__:
self.layers = nn.ModuleList([nn.Linear(4, 4) for _ in range(depth)])

The existing forward loop still works. Now the parent can find the layers when collecting parameters, saving a state_dict, changing device or dtype, and switching training mode.

How the bug shows up

For the original Net, constructing torch.optim.SGD(net.parameters(), lr=0.1) raises ValueError because the parameter list is empty. If the model has other registered parameters, optimizer construction can succeed while these layers are omitted from updates.

Autograd can still compute gradients for the unregistered layers when you call them. The problem is that an optimizer built from the parent's parameters() does not receive those tensors. Similarly, parent.eval() cannot reach an unregistered Dropout child, although Linear itself behaves the same in training and evaluation modes.

Choose a container for the job

• ModuleList registers an indexed collection; you write the forward logic.
• ModuleDict does the same for a collection with string keys.
• Sequential registers modules and passes each output to the next module.
• ParameterList registers a collection of parameter tensors rather than layers.

Sequential can contain a custom residual block. Use a custom forward method when the connections between your chosen modules need branching or other logic beyond passing one result to the next.

def net_facts(depth: int) -> dict