arXiv:2610.31943v1 Announce Type: new Abstract: Looped language models apply one block of layers repeatedly, and the design pays off only if depth is reusable: loop steps beyond the trained budget must remain usable at inference. We ask whether that rests on normalising the state between loop steps, as the released Ouro models do, or on agreement between the loop a model is trained with and the loop it is run with. On released weights this loop-boundary normalisation can be removed without adding parameters and with the readout path bit-identical, and the operation is an identity at one loop step, which serves as an implementation check. Removing it takes Ouro-1.4B and Ouro-2.6B to zero accuracy by the trained budget on a balanced binary task (chance 0.500).
Machine Learning · 2026-10-26
Agreement rather than normalisation keeps depth reusable in looped language models
arXiv:2610.31943v1 Announce Type: new Abstract: Looped language models apply one block of layers repeatedly, and the design pays off only if depth is reusable: loop steps beyond the trained budget must remain usable at inference. We ask whether that rests on normalising the state between loop steps, as the released Ouro models do, or on agreement between the loop a model is trained with and the loop it is run with.
Source and attribution
This is an attributed editorial digest. For the complete study and original reporting, follow the source link.
Reader feedback
0 likes · 0 dislikes