Small Models Scout Bottleneck Order for Large-Model Data Control
Quick summary
arXiv:2608.14936v1 Announce Type: new Abstract: Small proxy models are commonly used to identify data mixtures for larger-scale training. We ask whether their training trajectories reveal another transferable structure: the order in which larger models should resolve skill bottlenecks. We formulate first-passage skill training, where each monitored skill has a target floor and the objective is to minimize the tokens required to reach all floors. We introduce LogFloor, a closed-loop controller that directs each round toward current bottlenecks, producing phase-ordered resolution trajectories. A
Key takeaways
- arXiv:2608.14936v1 Announce Type: new Abstract: Small proxy models are commonly used to identify data mixtures for larger-scale training.
- We ask whether their training trajectories reveal another transferable structure: the order in which larger models should resolve skill bottlenecks.
- We formulate first-passage skill training, where each monitored skill has a target floor and the objective is to minimize the tokens required to reach all floors.
Why it matters
“Small Models Scout Bottleneck Order for Large-Model Data Control” should be evaluated beyond branding and benchmark scores. Its practical importance will emerge in task accuracy, latency, unit cost, safety and integration with real workflows.
Member comments