LLM Probability Concentration: How Alignment Shrinks the Generative Horizon
Quick summary
arXiv:2506.17871v4 Announce Type: replace-cross Abstract: Despite their impressive capabilities, aligned large language models (LLMs) often generate outputs that lack diversity. What drives this consistency in the generation? We investigate this phenomenon through the lens of probability concentration in the model's output distribution. To quantify it, we use the Branching Factor (BF)--the exponentiated length-averaged entropy of the output distribution, interpreted as the effective number of plausible next steps during generation. Our empirical analysis reveals two key findings: (1) BF often
Key takeaways
- arXiv:2506.17871v4 Announce Type: replace-cross Abstract: Despite their impressive capabilities, aligned large language models (LLMs) often generate outputs that lack diversity.
- What drives this consistency in the generation?
- We investigate this phenomenon through the lens of probability concentration in the model's output distribution.
Why it matters
“LLM Probability Concentration: How Alignment Shrinks the Generative Horizon” should be evaluated beyond branding and benchmark scores. Its practical importance will emerge in task accuracy, latency, unit cost, safety and integration with real workflows.
Member comments