Scale-Invariant Training for Time Series Foundation Models
Quick summary
arXiv:2610.07324v1 Announce Type: cross Abstract: Time series foundation models (TSFMs) are trained on large collections of time series datasets that span various morphologies and domains. This setting exposes models to series whose scales -- typical magnitudes of their values -- can differ substantially. Affine scaling methods such as Reversible Instance Normalization (ReVIN) scale model inputs and reverse the transform before computing the loss. We show that this inversion multiplies each series' gradient by $b^p$ relative to loss on scaled targets, where $b$ is the scaling denominator (e.g.
Key takeaways
- arXiv:2610.07324v1 Announce Type: cross Abstract: Time series foundation models (TSFMs) are trained on large collections of time series datasets that span various morphologies and domains.
- This setting exposes models to series whose scales -- typical magnitudes of their values -- can differ substantially.
- Affine scaling methods such as Reversible Instance Normalization (ReVIN) scale model inputs and reverse the transform before computing the loss.
Why it matters
This model development creates a new option for users and a new testing obligation for developers. A fixed evaluation set comparing quality, cost and failure behavior is more useful than launch claims.
Member comments