The Sequential Price of Continual Learning
Quick summary
arXiv:2609.29674v1 Announce Type: cross Abstract: Sequential task updates are fundamental to continual learning, but their recency bias can impose a lasting performance cost. We study this cost in an overparameterized linear-regression model with i.i.d. task sampling. We prove that distribution-level forgetting and population loss converge to the same stationary limit. This common limit separates exactly into the intrinsic loss asymptotically attained by joint training and an additional sequential price, and in more homogeneous task geometries the two terms coincide, making the total loss twic
Key takeaways
- arXiv:2609.29674v1 Announce Type: cross Abstract: Sequential task updates are fundamental to continual learning, but their recency bias can impose a lasting performance cost.
- We study this cost in an overparameterized linear-regression model with i.i.d.
- We prove that distribution-level forgetting and population loss converge to the same stationary limit.
Why it matters
The value of this work lies as much in how it was tested as in the claim itself. Sample design, baselines, uncertainty and replication help separate a laboratory result from real-world impact.
Member comments