Teacher Geometry Shapes Learnability in Teacher-Student Networks
Quick summary
arXiv:2609.09595v1 Announce Type: cross Abstract: Teacher-student systems, in which a teacher neural network generates training labels so that a student neural network can learn to implement the same function, are widely used as an abstract setting to study learning. However, the structure of the teachers is often overlooked by assuming randomly-generated, normally-distributed parameters. This hides substantial variation in how learnable different teachers are. We formalize learnability as the success rate of converging to the global minimum, as a function of overparameterization, learning alg
Key takeaways
- arXiv:2609.09595v1 Announce Type: cross Abstract: Teacher-student systems, in which a teacher neural network generates training labels so that a student neural network can learn to implement the same function, are widely used as an abstract setting to study learning.
- However, the structure of the teachers is often overlooked by assuming randomly-generated, normally-distributed parameters.
- This hides substantial variation in how learnable different teachers are.
Why it matters
“Teacher Geometry Shapes Learnability in Teacher-Student Networks” highlights the need for repeatable measurement rather than a single impressive demonstration. Independent validation across datasets and clearly stated limitations determine whether a result can guide product decisions.
Member comments