Enterprises often choose the most powerful AI model for every task, reasoning that a cheaper alternative might make mistakes. It is a simple strategy, but an expensive one. It also avoids the deeper question: How can an organization know when a less capable model is enough, or whether a different model should be used at all?
Model selection is not the real challenge
During a recent conversation with Leo Zheng, head of marketing at Fireworks AI, the AI infrastructure company that runs and improves open-weight models, an unexpected answer emerged. The key to effective AI isn't picking the perfect model in advance. Fireworks wants to enable every company to own the continual learning loop within its four walls. The idea is to abstract away model updates while companies feed new signals as customer behavior changes. That framing shifts the conversation from a one-time model decision to a continuous process.
Model choice then becomes simply one important action the system can take, not the be-all and end-all decision a developer must get right before launch. The harder problem is building a feedback loop that tells a company what actually worked. Arguably, the more important component is integrating enterprise data into that continual learning loop.
The need for outcome data
Proprietary data does not automatically guarantee AI success. Neither does dumping that data into a retrieval-augmented generation (RAG) system. Competitors can do the same. The durable advantage comes from outcome data: traces that connect what the system saw, what it did, and whether the business result was acceptable. Those traces become valuable when a company uses them to improve the system and carries the learning to the next model.
Public benchmarks are useful, but they describe how a model performed on someone else's test. They cannot tell a bank whether a fraud alert was correct, inform a retailer whether an AI agent resolved a return, or tell a software company whether AI-generated code survived review. Enterprises already have pieces of those answers buried in support transcripts, accepted patches, abandoned carts, and closed tickets. The trick is turning that mess into something a system can learn from.
Consider a software company. It already collects product analytics to improve its product. AI can extend that flywheel. Application signals can improve the system powering the product by capturing user intent, engagement, and preferences.
Avoid weak proxies
However, this is not foolproof. A fast learning loop built on a weak proxy is simply a fast way to make the system worse. Suppose a customer support agent named Toni accepts a suggested response. Was the suggestion good? Maybe. But what if Toni was rushing and the customer reopened the ticket the next day? It is not enough to act on inputs alone. The company needs to connect the suggestion to the eventual outcome or risk rewarding the wrong behavior.
Acceptable or ideal outcomes must be defined by the enterprise. A platform can optimize through learning loops, but only if desired outcomes are clearly defined. This changes the cost calculation. The cheapest model by token can be the most expensive model by outcome if people must review, retry, and repair its work. A company can be losing money on AI failures while congratulating itself for lowering inference costs.
Start with one workflow
None of this requires open weights, of course. A company can own its evaluations, improve prompts and retrieval, and route among closed APIs and open models without changing a single parameter. For many workloads, that will be enough.
This generally won't start with reinforcement learning. It starts by picking one workflow with an observable outcome and logging the request, model, cost, answer, and what happened next. Do that for a quarter and patterns should emerge. A cheaper model will prove sufficient for some work, while a proprietary frontier model will earn its price elsewhere. In a third category, every model may fail in the same repeatable way, and it is this last category that makes the serious case for changing the weights.
Open weights turn models into raw material
A few years ago, the common argument was that open source succeeds when it makes developers' lives easier, not when it wins an ideological purity contest. The same applies to open weights. The importance of open weights is not that they are a cheaper facsimile of whatever the largest AI labs ship. Instead, they matter because they shift leverage away from the models and toward users with proprietary outcome data. They let enterprises turn what they learn into something they can shape, keep, and carry forward as models change.
Open weights turn model renters into model owners. That is a very big deal.
Consider Cursor's Composer 2. The company started with the open-weight Kimi K2.5 model, then specialized it through continued pretraining and reinforcement learning in environments built to resemble Cursor itself. The interesting thing is not that Cursor picked the right model. It is that Cursor can use what it learns from developers to keep making that model more right for them. Fireworks ran the reinforcement learning inference, while Cursor supplied what only Cursor could: its product environment and definition of good. Cursor subsequently built Composer 2.5 on the same Kimi K2.5 checkpoint and said 85% of its compute went into additional training and reinforcement learning.
The base model was the starting point, not the finished product.
This is why open weights matter. They turn the model from a finished product into raw material. Foundation labs have broad knowledge, far more than any normal company could hope to collect. What they don't know is whether your patch shipped, your fraud was stopped, your customer stayed, or whether Toni's response made matters worse. You do.
RAG lets a model consult your data. A learning loop lets the system learn from it. One helps answer today's question, but the other helps the company answer tomorrow's question better.
Control must be convenient
Raw weights alone will not deliver that control. Open source did not take over the enterprise because CIOs wanted tarballs. It won because distributors such as Red Hat, and especially cloud providers, turned theoretical control into everyday convenience. Open-weight AI needs the same translation.
Enterprises want the convenience of a closed API without surrendering control over the intelligence their data creates. That is the real Fireworks opportunity. It is not faster tokens, but a shorter distance between learning something about your customers and making the product smarter because of it. Fireworks CEO Lin Qiao calls this application-specific intelligence. Enterprises don't need to outbuild OpenAI or Anthropic. Even if they wanted to, they couldn't. Instead, they need to turn what they know about their customers and workflows into intelligence those labs cannot supply.
Fireworks is not alone. Together AI also spans training, evaluation, and serving, while Baseten now talks about connecting production feedback to model training. This is becoming a category, not simply a Fireworks feature. Fireworks' particular bet is that the entire learning loop should belong to the customer.
Rethinking model selection
Back to the question that started the conversation: How do you pick the right model in advance? The answer is that you can't. Neither can anyone else, and that is fine, because it is not really the right question.
Once the enterprise owns the test, it no longer has to trust a vendor's benchmark or brand. New models must prove themselves against the work that matters to that company, then be promoted, specialized, or discarded accordingly. Model choice becomes a consequence of learning, not an act of faith. In effect, the models audition for the enterprise, not the other way around.
No one can predict whether OpenAI, Anthropic, Google, or the open-weight community will lead six months from now. The sensible response is to stop making that prediction central to the strategy. The enterprise's own data should decide which model to pick. That is the bet in Fireworks: make open-weight models as easy to improve as APIs are to call, so enterprises can stop treating their data as context for somebody else's model and start using it to build intelligence they control.
Source: InfoWorld News