Self-Improvement as Coherence Optimization: A Theoretical Account
Quick summary
arXiv:2601.13566v2 Announce Type: replace-cross Abstract: Can language models improve their accuracy without external supervision? Methods such as debate, bootstrap, and internal coherence maximization achieve this surprising feat, even matching golden finetuning performance. Yet why they work remains theoretically unclear. We show that they can all be understood as coherence optimization, the search for a context-to-behavior mapping that is most compressible and jointly predictable, with debate an exact instance and bootstrap and internal coherence maximization closely related to it. We prove
Key takeaways
- arXiv:2601.13566v2 Announce Type: replace-cross Abstract: Can language models improve their accuracy without external supervision?
- Methods such as debate, bootstrap, and internal coherence maximization achieve this surprising feat, even matching golden finetuning performance.
- Yet why they work remains theoretically unclear.
Why it matters
The importance of “Self-Improvement as Coherence Optimization: A Theoretical Account” will be measured by what changes in practice. User behavior, access conditions, verifiable performance and responsible-use outcomes are the signals worth following.
Member comments