Apple and Omdia Release New Report: On-Device AI Is Becoming a New Fulcrum for Enterprise AI Strategy
Recently, Omdia, a globally recognized technology market research firm, officially released a major industry report, Rethinking Critical AI Infrastructure. Commissioned by Apple and based on in-depth surveys of more than 1,500 enterprise technology decision-makers (CIOs, CTOs, AI leads, and others), it is the broadest-sample study of enterprise AI infrastructure published in the past year.
For the past two years, the dominant narrative around AI has been almost entirely about "the cloud" and "hundred-billion-parameter large models." But with hard data, the Omdia report puts the untold "real story" inside enterprises on the table — security anxiety, runaway costs, and workload mismatch are quietly pushing more and more companies toward "on-device." These three dimensions are exactly where on-device AI's core strengths lie: through architectural design, it addresses enterprises' security, economic, and workload challenges.
Below is our read of the report's key findings.
On-device becomes a new consideration in enterprise AI planning
The biggest change in AI infrastructure over the past few years is not compute, but that enterprises are rethinking where compute should live. Omdia's research reveals a fact: pure cloud solutions are hitting bottlenecks at a large number of companies.
01 Hybrid deployment is now mainstream
The report shows that 41% of enterprises have adopted a "cloud + on-premises + on-device" hybrid deployment strategy, 32% use cloud only, 18% use on-premises only, and 8% use on-device only.
Moreover, the higher an enterprise's AI maturity, the more likely it is to use hybrid deployment — among companies that have deployed AI at scale, about half use a hybrid strategy, compared with about one-third of companies still in the pilot phase.
02 Enterprises are not very satisfied with their current cloud providers
Omdia delivers a number that makes many cloud vendors uncomfortable: only 9% of enterprises say their current strategic AI partner "has no obvious weaknesses." The top three complaints are:
Insufficient data privacy and security controls (32%)
Unpredictable or persistently high costs (32%)
Poor integration with existing systems (28%)
Together, these three amount to the "three mountains" standing in the way of enterprise AI adoption.
03 A clear intent to move toward on-device
More than one-third of enterprises say they will move more AI workloads to on-device devices over the next 12 months. Broken down by current primary deployment model:
65% of enterprises already using on-device plan to increase their on-device investment
51% of enterprises using on-premises deployment plan to add on-device capabilities
33% of enterprises using a hybrid strategy plan to shift further toward on-device
26% of cloud-only enterprises plan to migrate toward on-device
Worth noting: in this chapter Omdia highlights a commonly overlooked "cost visibility bias" — 72% of enterprises keep a close eye on their monthly cloud bill, but fewer than 50% systematically track hidden costs such as security compliance, staffing audits, and process delays. In other words, when assessing TCO, enterprises often underestimate the value of on-device.
Security — from process compliance to architectural privacy
Two numbers in the report are worth pondering:
99% of enterprises handle proprietary or sensitive data in their AI workflows;
76% of enterprises express clear concern about data breaches caused by cloud services.
It is not that enterprises distrust cloud vendors. A cloud vendor's security engineering is almost certainly stronger than the in-house data centers of 99% of enterprises. The real problem is that AI has turned "data transmission" from a one-time action into a continuous one.
In the past, moving data to the cloud was a one-time compliance decision. But AI applications mean that every user question, every upload, and every tool call continuously sends internal enterprise data outward.
Omdia uses a very precise concept to describe the advantage of on-device: architectural privacy. Its logic is simple but powerful: no transmission, no transmission risk to manage.
Development teams can work directly on local datasets without accessing third-party infrastructure. Models are trained where the data lives, so there is no data transfer cost to bear, no transfer encryption to verify, no cloud provider controls to audit, and no cross-border data transfer paperwork to handle.
At the same time, with on-device AI, development teams can use AI tools locally to speed up cleaning and preparation — using models to identify sensitive fields, automatically mask data, validate data quality, and transform datasets, all before the data leaves the secure environment.
This is not "better security." It is "no longer needing that part of security management." The two are entirely different categories.
Economics — the "experimentation tax" and runaway inference bills
The report uses a concept in this chapter called the "Experimentation Tax."
01 The "experimentation tax" quietly paid
Doing AI development in the cloud means every failed experiment, like every successful one, accumulates costs in compute time, data transfer, and storage. Over time, R&D teams subconsciously "run fewer experiments."
The report puts it bluntly: this is fundamentally a tax on innovation. What most distinguishes AI R&D from traditional software development is precisely its extremely high demand for trial-and-error density. Suppress trial and error, and you suppress iteration speed; suppress iteration speed, and you cap the product's ceiling.
On-device development releases this variable again — the hardware is a sunk cost, and once deployed, each additional iteration, test run, and validation cycle incurs no extra cost. A hundred experiments cost the same as one.
02 The "linear curse" of inference cost
Cloud API costs grow linearly with user volume — the more successful an AI application, the more money it burns.
On-device inference costs are fixed: ten thousand calls or a million, the hardware cost stays the same. On-device makes unpredictable costs predictable.
03 The difference in usage models
Cloud-based AI mainly follows two pricing models: pay per API call, which accumulates with every request; or per-seat subscription, which stays fixed regardless of usage intensity. Both models create economic challenges that grow over time.
What stings even more is the waste in the subscription model: if 1,000 cloud AI licenses sell at $30 per user per month, even if only 10% go unused, that is $3,000 wasted every month.
The report shows that 62% of enterprises are dissatisfied with end-user adoption of AI tools, with many purchased cloud seats sitting idle for long periods.
For on-device AI, even when the AI workload is light, the device itself remains a productivity tool — it never becomes an "idle asset."
Workload effectiveness — the myth of the "giant model"
The single most striking data point in the entire report is this:
57% of enterprises actually deploy AI models with fewer than 10 billion parameters.
For years, the entire public conversation has chased "bigger" — 100 billion, 1 trillion, 10 trillion — as if there were a simple linear relationship between parameter count and "AI capability." But enterprises vote with their feet: more than half of real deployments are below 10 billion.
Behind that gap lies a fact many overlook — frontier research and industrial deployment are moving along two different curves. The frontier curve pursues the "capability ceiling"; the industrial curve pursues a "value loop." The two overlap in AI's early days, but once capability spreads far enough, they diverge. Enterprises no longer need "the strongest on earth"; they need "good enough, controllable, cheap, and secure."
And 10 billion parameters happens to be the range that modern high-end on-device devices can comfortably carry. According to the report, a 10-billion-parameter model quantized to 8-bit needs about 10-12 GB of memory — more than enough for modern PCs and workstations with 16 GB, 24 GB, or more.
Unified memory architecture: the underestimated hardware variable
The report also has a meaty passage on the Unified Memory Architecture (UMA). This architecture — Apple Silicon being the typical example — lets the CPU and GPU share one large memory pool, trading space for topological simplicity. A laptop with 128 GB of unified memory can, in theory, run quantized 70-billion or even 100-billion-parameter models locally.
The significance goes far beyond the conclusion that "on-device can run large models." It points to a larger possibility — the inference hardware of the future may not all look like "GPU + HBM." As model weights grow larger and inference becomes the dominant workload, the "large memory + moderate compute" architecture may move from an alternative to one of the mainstream options.
The report also has a counterintuitive finding worth special attention:
Enterprises already using on-device AI are nearly twice as likely as others to deploy models with more than 100 billion parameters.
In other words — it is not that on-device cannot hold large models; it is that enterprises with on-device capability are more willing to use them.
Hybrid intelligence — the standard answer for the future
Emphasizing the value of on-device is not about setting it against the cloud. In its conclusion, Omdia states clearly: the enterprise AI architecture of the future will inevitably be hybrid.
Cloud — for ultra-large-scale training, public API services, and tasks that need massive elastic compute;
Enterprise internal clusters — for medium-to-heavy workloads such as batch data processing and private model training;
On-device devices — for high-frequency, low-latency, privacy-sensitive personalized inference and development debugging.
A typical future workflow might look like this: prototype and fine-tune quickly with local data on an on-device workstation; call on the cloud or an internal cluster for final large-scale training; deploy the optimized model back to the on-device devices of thousands of employees for secure, low-cost everyday inference.
The report's conclusion also offers four clear recommendations:
First, match infrastructure to workload
Mandate workload-infrastructure alignment based on "model size, data sensitivity, usage pattern, and user needs." Researchers working on frontier models and the majority of enterprises deploying sub-10B models have completely different optimal infrastructure; development and production stages have different needs too. Beware one-size-fits-all default infrastructure choices.
Second, solve security problems architecturally wherever possible
For projects involving proprietary or sensitive information, prioritize solutions that eliminate transmission risk through architecture rather than process. On-device is not the only answer, but for development workflows and small-model inference, it eliminates an entire class of compliance reviews and process delays.
Third, make unpredictable costs predictable
Extend the scope of TCO measurement to easily overlooked areas — security review delays, compliance overhead, the productivity impact of resource constraints, and the marginal cost of experimentation under usage-based billing. Enterprises always optimize what they can measure; incomplete cost visibility inevitably leads to suboptimal infrastructure decisions.
Fourth, empower technical teams to build hybrid solutions
Give in-house teams the freedom to bring in multiple infrastructure types. The teams doing the most complex AI work are already building hybrid portfolios. Procurement frameworks and technology policies should support, not restrict, this strategic diversification.
Final thoughts
As this shows, the Omdia report is not out to reject the cloud. It highlights a truth that public discourse has oversimplified —
The optimal answer for AI infrastructure is not "all cloud" or "all on-device," but running every workload where it fits best.
The enterprises in the survey that have already built on-device capabilities have not given up the cloud. They have simply found that when on-device capability is strong enough, the cloud can be used more intelligently, local clusters can be used more focused, and the development pace can be unleashed more fully.
As AI capability keeps evolving and workload patterns keep changing, enterprises that build infrastructure flexibility today will reap compounding returns tomorrow.
Whoever recognizes this earlier will save themselves trouble and gain efficiency sooner.
References:
Omdia, "Rethinking Critical AI Infrastructure: How on-device infrastructure is emerging as a platform for enterprise AI development and deployment"