GLU Activation

A Gated Linear Unit combines two projections of the same input. One produces values; the other controls how much of each value passes through.

In this exercise, w is the gate matrix and v is the value matrix:

gate = sigmoid(x @ w)
value = x @ v
output = gate * value

The @ operations are matrix products. The final multiplication is elementwise. Sigmoid maps gate inputs to numbers between zero and one, so a gate near zero suppresses its value and a gate near one preserves it. At a gate input of zero, sigmoid is 0.5, so the corresponding value is halved.

For example, let x=[2.0, -4.0], w be a 2-by-2 zero matrix, and v be the 2-by-2 identity matrix. The gate is [0.5, 0.5], the value projection is [2.0, -4.0], and the result is [1.0, -2.0]. A negative value can remain negative.

Your task

Implement glu(x, w, v) as sigmoid(x @ w) * (x @ v), without bias terms.

• x has shape (..., d_in), where ... represents any leading batch dimensions.
• w has shape (d_in, d_out) and produces the gate inputs.
• v has shape (d_in, d_out) and produces the values.
• Return shape (..., d_out), preserving x's leading dimensions.

Do not use torch.nn.functional.glu or jax.nn.glu. Those convenience functions split an already-projected input into halves; this exercise instead supplies the two projection matrices explicitly. Sigmoid itself is allowed.

The curve below shows only the sigmoid gate, not the complete GLU mapping.