FlexiFlow: Bandit-based Model Switching in ML Workflows
Quick summary
arXiv:2610.07286v1 Announce Type: cross Abstract: Model optimizations help improve inference performance and accuracy of ML workflows. However, relying on a single model to perform inference across all data batches often fails to maximize accuracy and thus overall performance. In many cases, alternate models could perform better on specific subsets of data where a primary model underperforms. Our experiments with real ML workflows indeed show that switching models improves workflow accuracy by up to 23%. Yet, current systems lack the ability to adaptively switch between models based on perform
Key takeaways
- arXiv:2610.07286v1 Announce Type: cross Abstract: Model optimizations help improve inference performance and accuracy of ML workflows.
- However, relying on a single model to perform inference across all data batches often fails to maximize accuracy and thus overall performance.
- In many cases, alternate models could perform better on specific subsets of data where a primary model underperforms.
Why it matters
This model development creates a new option for users and a new testing obligation for developers. A fixed evaluation set comparing quality, cost and failure behavior is more useful than launch claims.
Member comments