LoopICL: Looping a single transformer block to solve tabular tasks
Quick summary
arXiv:2609.36108v1 Announce Type: cross Abstract: Tabular foundation models using in-context learning have recently surpassed gradient-boosted trees on predictive tabular tasks. However, recent mechanistic insights suggest that parameters in these models are largely redundant. We introduce LoopICL, a looped transformer whose core design decouples parameter count from computational depth. LoopICL consists of a single block, processing data through two coupled streams: a cell stream capturing per-cell feature representations and a row stream capturing in-context example representations, jointly
Key takeaways
- arXiv:2609.36108v1 Announce Type: cross Abstract: Tabular foundation models using in-context learning have recently surpassed gradient-boosted trees on predictive tabular tasks.
- However, recent mechanistic insights suggest that parameters in these models are largely redundant.
- We introduce LoopICL, a looped transformer whose core design decouples parameter count from computational depth.
Why it matters
“LoopICL: Looping a single transformer block to solve tabular tasks” is a product decision that may change how people work with AI. Its value depends on task completion, correction effort and data handling—not simply the presence of a new feature.
Member comments