arXiv Artificial Intelligence

Recurrent Looped Transformer

1 hour ago 07.10.2026 07:00 👁 2 views
Recurrent Looped Transformer

Quick summary

arXiv:2610.07591v1 Announce Type: cross Abstract: State tracking requires an update at every input, but the depth a Transformer applies to each token is fixed regardless of sequence length. We introduce the Recurrent Looped Transformer (RLT), which splits its layers between a parallel causal encoder and a recurrent decoder. At each token, the decoder merges the encoder output with the previous token's final decoder state, so the computation path grows with sequence length at a fixed per-token cost. On six algorithmic tasks, we compare five splits of eight layers with an eight-layer Transformer

Key takeaways

  • arXiv:2610.07591v1 Announce Type: cross Abstract: State tracking requires an update at every input, but the depth a Transformer applies to each token is fixed regardless of sequence length.
  • We introduce the Recurrent Looped Transformer (RLT), which splits its layers between a parallel causal encoder and a recurrent decoder.
  • At each token, the decoder merges the encoder output with the previous token's final decoder state, so the computation path grows with sequence length at a fixed per-token cost.

Why it matters

“Recurrent Looped Transformer” is a product decision that may change how people work with AI. Its value depends on task completion, correction effort and data handling—not simply the presence of a new feature.

Kaynak sitede devamını oku: arXiv Artificial Intelligence ↗