Key points
- Open weight models now deliver 90% of the effectiveness of closed frontier models.
- Token costs for open models are 10% to 25% of those for leading closed alternatives.
- Enterprises face a shortage of kernel engineers to manage on-prem AI hardware.
The short version
- IBM CEO Arvind Krishna stated that open weight models are now economically attractive for on-premises deployment.
- The performance gap between closed and open models has narrowed significantly since 2024.
- Running these models locally requires specialized engineering skills that many organizations lack.
The details
IBM CEO Arvind Krishna told investors during the company’s Q3 earnings call that open weight models have become a practical choice for enterprise use on-premises. According to the Register, Krishna noted that in 2024, closed frontier models were clearly superior to open alternatives in broad-based assessments. He described a scenario where a top closed model might score 70 on a benchmark while an open model scored only 20 or 30. This large performance delta made closed models the default choice for high-stakes tasks.
By late 2025, Krishna observed that the performance gap began to shrink. He said the comparison shifted from 70 to 20 or 30, to roughly 70 to 50. This improvement in open model quality, combined with drastically lower costs, changed the economic calculation for businesses. Krishna emphasized that open weight models now deliver anywhere from 10% to 25% of the token cost of frontier models. He argued that receiving 90% of the effectiveness at 10% of the price is a hard value proposition to beat.
Krishna framed the decision as a simple cost-benefit analysis for any enterprise. He asked whether organizations wanted something that cost 10% as much but performed 90% as well, or if they preferred to pay ten times higher for marginal gains. He concluded that the vast majority of tasks, such as summarizing emails, examining contracts, or composing responses, can be handled effectively by the cheaper open models. This shift is driving booming interest in on-premises AI infrastructure, with Krishna suggesting most businesses can build useful setups in just two or three server racks.
The context
The move toward on-premises AI is not just about software costs but also infrastructure complexity. Krishna warned that while the hardware footprint is small, getting the systems to run efficiently is difficult. Many enterprises struggle to find the necessary technical talent to manage these deployments. He specifically mentioned the lack of kernel engineers and network specialists who can optimize model clustering on GPUs. Ensuring proper network proximity and result caching is critical for performance, yet these skills are scarce in many organizations.
This trend coincides with broader industry shifts, including Nvidia’s efforts to woo European customers with more open weight models and reports of significant spending on AI tokens by developers. The Register noted that while AI is expected to consume a fifth of tech budgets by 2027, it may not pay for itself immediately. However, the immediate appeal of low-cost, high-performance open models is accelerating adoption. Companies are weighing the operational burden of managing local hardware against the financial savings of reduced token spending.
What to watch
- Monitor the continued narrowing of the performance gap between open and closed models.
- Track the availability of skilled engineers capable of managing on-prem AI infrastructure.
- Watch for shifts in enterprise spending as organizations migrate workloads to local systems.
- Observe how vendors like Nvidia and IBM adjust their offerings to support this trend.
What it means for organisations
Organizations should assess their current AI workflows to determine which tasks can tolerate slightly lower performance metrics. Leadership needs to evaluate whether the cost savings from open models outweigh the operational complexity of on-premises management. Teams usually require new skill sets to maintain local GPU clusters, so hiring or training plans must be reviewed. Check whether your IT staff can handle kernel-level optimizations and network proximity issues. Communicate these technical requirements to procurement and finance teams early. Ensure that data governance policies account for local model hosting.
What to do and how to stay safe: IBM
- Audit your current AI usage to identify tasks that do not require top-tier model accuracy.
- Evaluate the technical skills within your IT team to manage on-premises GPU infrastructure.
- Compare the total cost of ownership for open models, including hardware and engineering labor.
- Review data security policies to ensure they support local model deployment and caching.
Step-by-step guide: Secure Home Routers for Small Business Networks
General security guidance from the TechSec Headlines newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
How do open weight models compare to closed models in performance?
Open weight models now deliver about 90% of the effectiveness of closed frontier models, according to IBM CEO Arvind Krishna.
What is the cost difference between open and closed AI models?
Open weight models cost 10% to 25% of the token cost of frontier models, making them significantly cheaper to run.
What challenges do enterprises face when deploying on-prem AI?
Many organizations struggle to find kernel engineers and network specialists needed to optimize GPU clustering and caching.
