A Generalisation Signal Need Not Be a Model-Selection Signal
Quick summary
arXiv:2609.39099v1 Announce Type: cross Abstract: Model selection in computational biology often relies on validation data drawn from the training regime, even when deployment lies outside it. When validation no longer preserves which model is best, a natural alternative is to rank candidates using properties of the trained network itself. We test this idea using a novel, forward-only proxy motivated by the norm of the Hessian, alongside common Hessian measures, across molecular property, protein fitness, and drug-response tasks. Contrary to our hypothesis, geometry does not become more useful
Key takeaways
- arXiv:2609.39099v1 Announce Type: cross Abstract: Model selection in computational biology often relies on validation data drawn from the training regime, even when deployment lies outside it.
- When validation no longer preserves which model is best, a natural alternative is to rank candidates using properties of the trained network itself.
- We test this idea using a novel, forward-only proxy motivated by the norm of the Hessian, alongside common Hessian measures, across molecular property, protein fitness, and drug-response tasks.
Why it matters
“A Generalisation Signal Need Not Be a Model-Selection Signal” should be evaluated beyond branding and benchmark scores. Its practical importance will emerge in task accuracy, latency, unit cost, safety and integration with real workflows.
Member comments