Model selection is now a engineering problem for us

When we started building with AI choosing a model felt like a one time decision cause we'd evaluate a few options then pick the one that fit the use case and move on. That hasn't really been the case anymore cause every new model release sparks another round of testing + every team has slightly different priorities and before long we're maintaining integrations with providers we never planned on supporting. The engineering work isn't really about the models themselves but I think it's everything around them. Keeping integrations consistent, making sure behavior doesn't change unexpectedly, keeping track of different APIs and understanding where requests are going is now a big part of the job which I didn't think it would be like not something I thought of. It also means we can't treat AI as one part of the stack anymore cause when someone wants to swap a model for a new release or try a different provider we have to make sure existing workflows still behave the same way and check that we haven't introduced regressions and then make sure another team isn't relying on the same implementation. Im very curious how other teams handle this. Are you standardizing on one provider, building an internal abstraction layer or maybe accepting that model selection is going to stay messy like that? In hindsight we planned for choosing models but we never really planned for living with all of them. submitted by /u/Ok_Obligation_3681 [link] [comments]