Building products around AI can be a tricky endeavor, especially if AI sits at the core of your user experience. Not only do you rely on the accuracy of the model, but you also rely on its availability.
As luck would have it, on the day I’m writing this article, Claude, ChatGPT, and Grok all went offline at pretty much the same time, highlighting the clear need for contingency planning in product design. How does your application behave when a model isn’t available? Can it fail over to another model? Can a user still use your application with no model at all?
These are all critical questions. Also, keep in mind that OpenAI, Anthropic, and xAI aren’t the only model providers out there. That’s why services like OpenRouter exist: to help direct requests to an available model. There are thousands of models offered by hundreds of providers. Beyond gains in uptime and stability, looking at other providers can often unlock greater efficiency and cost savings. Unless you’re truly leveraging the full power of a top-tier model like Opus 5.1, there is likely a suitable alternative available.
When building your service, even if one feature requires a specific top-spec model, other features might need less compute, faster performance, or simpler logic. How often do you switch models in Claude Desktop or ChatGPT to run a task on a smaller model? Be honest! Probably not that often, yet from a product perspective, optimizing model assignment per feature is where huge gains can be made.
So as you plan your LLM architecture and prepare to roll out AI-enabled features, take a step back. Don’t default to the most obvious providers or the latest flagship models. Ask yourself: is there a cheaper, more cost-effective way to achieve this that also provides greater stability for your end users?
Ex-NASA engineer and cloud architect with over a decade of experience building scalable systems for startups and enterprises.
Work with Tom →
