Hidden Gauge Controls Feature Specialization in ReLU Networks
Quick summary
arXiv:2608.06766v1 Announce Type: cross Abstract: Training changes a network's predictions while allocating task-relevant structure across its internal units. In an overparameterized ReLU network, several neurons can begin with exactly the same functional role, yet one may acquire a teacher feature while the others become redundant. We call the identity of that neuron feature ownership and ask whether it can be controlled by a parameter choice invisible to the initial predictor. In a tractable Gaussian teacher--student model, we fix the complete initial function and vary only a positive-homoge
Key takeaways
- arXiv:2608.06766v1 Announce Type: cross Abstract: Training changes a network's predictions while allocating task-relevant structure across its internal units.
- In an overparameterized ReLU network, several neurons can begin with exactly the same functional role, yet one may acquire a teacher feature while the others become redundant.
- We call the identity of that neuron feature ownership and ask whether it can be controlled by a parameter choice invisible to the initial predictor.
Why it matters
“Hidden Gauge Controls Feature Specialization in ReLU Networks” is a product decision that may change how people work with AI. Its value depends on task completion, correction effort and data handling—not simply the presence of a new feature.
Member comments