arXiv Artificial Intelligence

Answer First, Reason Later: Commitment Order in Diffusion LLMs

1 hour ago 07.08.2026 07:00 👁 2 views
Answer First, Reason Later: Commitment Order in Diffusion LLMs

Quick summary

arXiv:2608.05687v1 Announce Type: cross Abstract: Masked diffusion language models (dLLMs) can commit tokens in any order -- a freedom marketed as their core advantage over autoregressive decoding. We show that on reasoning tasks this freedom is instead the axis of failure. Logging every commitment during decoding of LLaDA-8B on GSM8K, we find that unconstrained (pure) decoding commits the final answer at 15-24% of the trajectory while half the reasoning region is still masked, and collapses to answer-only outputs on up to 90% of problems as the canvas grows. The cause is not the model's termi

Key takeaways

  • arXiv:2608.05687v1 Announce Type: cross Abstract: Masked diffusion language models (dLLMs) can commit tokens in any order -- a freedom marketed as their core advantage over autoregressive decoding.
  • We show that on reasoning tasks this freedom is instead the axis of failure.
  • Logging every commitment during decoding of LLaDA-8B on GSM8K, we find that unconstrained (pure) decoding commits the final answer at 15-24% of the trajectory while half the reasoning region is still masked, and collapses to answer-only outputs on up to 90% of problems as the canvas grows.

Why it matters

“Answer First, Reason Later: Commitment Order in Diffusion LLMs” should be evaluated beyond branding and benchmark scores. Its practical importance will emerge in task accuracy, latency, unit cost, safety and integration with real workflows.

Kaynak sitede devamını oku: arXiv Artificial Intelligence ↗