Air-Gapped vs Cloud AI: Enterprise Cost Comparison

TL;DR – Our analysis finds that air-gapped on-prem AI deployments are more cost-effective for any organisation with >200 developers and can drive significant cost savings for large (2000-person) orgs.

Three major developments in recent weeks have rocked considerations around model deployment. Firstly, reports are emerging of enterprises going heavily over budget on cloud compute spend. Recent reporting from TechCrunch and CIO highlights that autonomous agents are burning through cloud tokens and API budgets much faster than initially forecast. Secondly, the US government’s decision to impose sudden export controls on Anthropic’s Fable and the staggered release of OpenAI’s GPT-5.6 shifted infrastructure sovereignty directly into the spotlight.

Enterprises are now weighing the pros and cons of owning their own GPUs and hosting models directly on them. While this approach may address the resilience question posed by the Fable ban and GPT-5.6’s staggered release, uncertainty still remains surrounding the cost.

We have evaluated the cost of a standard external SaaS cloud agent against Cosine’s enterprise coding agent running on a large, enterprise-controlled, sovereign GPU infrastructure. The comparison focuses specifically on a coding agent use case and the underlying hardware economics.

Headline comparison

While economics are broadly comparable at smaller deployment sizes (using a fully loaded infrastructure view), the sovereign GPU model becomes materially more attractive at larger scales. Because the same infrastructure can be reused across multiple workloads, enterprise teams can scale without every developer interaction generating additional external token spend.

Note that the fully loaded basis for “Sovereign GPU” includes hardware amortisation (capex), electricity, cooling, and operational support costs. In contrast, "Cloud coding agent" costs are based on projected API token usage driven by agentic workflows. Furthermore, GPU numbers above are a conservative procurement view. If the H200 estate is already being provisioned as part of a broader enterprise sovereign AI infrastructure strategy, the incremental cost is even lower.

On a fully loaded basis, the sovereign GPU model becomes economically superior once deployment scale exceeds approximately 200 developers. By excluding amortisation – since the hardware is not solely attributable to the coding agent – the sovereign strategy is consistently more cost-efficient across all deployment sizes.

Cloud coding agent cost framework

The cloud coding agent estimates reflect realistic enterprise agent usage rather than static seat licensing. Because autonomous agents execute parallel background workflows and process massive volumes of tokens, actual production costs are driven by usage rather than by flat seat licenses.

Core model assumptions:

• Standard profile (80% of engineering): budgeted at ~$200/user/month. Comprises the standard enterprise base seat license supplemented by everyday conversational and agentic assistance (averaging ~$13/active day in tokens).
• Power profile (20% of engineering): budgeted at ~$750/user/month. Driven by advanced engineers running autonomous, long-context repository refactoring loops and multi-repo code generation, which incurs heavy API token usage.
• Total portfolio: settles at a blended run-rate of ~$310/user/month.
• Exchange rate: $1 = £0.74.

This produces the following indicative annualised spend:

These assumptions are broadly aligned with public reporting on large-scale enterprise deployments, where usage intensity increases materially once engineering teams adopt agentic workflows at scale.

Sovereign GPU infrastructure breakdown

The sovereign GPU model assumes that the coding agent is deployed on a large, enterprise-controlled, company-operated cluster of 8x NVIDIA H200 nodes.

The infrastructure modelling assumes an indicative purchase cost of ~£360k per node, amortised over three years, alongside electricity, cooling, and operational support. That produces an indicative fully loaded annual infrastructure cost of approximately ~£167k/year per node.

Calculating concurrency and active requests

To estimate the required number of nodes for a given developer count, we consider three key factors:

1. What proportion of a developer’s time is spent on active concurrency (typically 13-20%)
2. How many concurrent users can be supported per node (~16)
3. What GPU utilisation is attributable to coding agent workloads (assuming that GPUs are also used for other tasks)

That produces the following indicative infrastructure sizing:

The annual sovereign GPU estimates are therefore derived by multiplying the indicative node count by the approximate fully loaded annual node cost of ~£167k/year.

For example:

• 500 developers: 5-7 nodes × ~£167k/year = ~£835k-£1.17M/year
• 2,000 developers: 17-25 nodes × ~£167k/year = ~£2.84M-£4.18M/year

Other considerations

Beyond direct infrastructure costs, enterprises should evaluate the following strategic factors:

• Speed of onboarding and scaling: cloud solutions are far more agile at scaling rapidly, effectively bypassing current GPU supply chain bottlenecks.
• Cost stability: on-premise infrastructure offers a fixed, predictable ceiling on compute costs, enabling better budget control despite energy price volatility. In contrast, cloud costs can be harder to manage as usage scales.
• Dependency risk: on-premise deployments eliminate reliance on external cloud providers, mitigating external dependency risks.
• Security and data egress: on-premise hosting provides superior control over data, significantly reducing security and data egress risks.
• Model accessibility: cloud-based models currently provide broader access to leading frontier models (e.g. from OpenAI and Anthropic). Air-gapped on-premise environments typically cannot access these external models, limiting model flexibility.

Key assumptions and limitations

When making your evaluation, keep these final considerations in mind:

1. The sovereign GPU figures above represent indicative infrastructure economics only and do not include software licence fees. The comparison is therefore intended to show the relative economics of external API-based coding agents versus internally provisioned sovereign GPU capacity, rather than a full comparison of commercial proposals.
2. The cloud coding agent estimates are based on current publicly available pricing and observed enterprise usage patterns as of July 2026. The frontier AI market is evolving rapidly, and current pricing may not reflect long-term steady-state economics, particularly given the degree of competitive subsidisation among leading AI vendors.
3. This analysis assumes physical on-premise ownership of GPU hardware (e.g. 8xH200 nodes). Organisations should consider operational factors such as hardware longevity, interconnect reliability, and rated service hours over the asset's lifecycle. Furthermore, while sovereign infrastructure provides long-term cost benefits, smaller enterprises or those with variable workloads may prefer long-term cloud compute contracts to mitigate expenditure and hardware maintenance requirements.

Looking to explore the economic benefits of sovereign AI deployment for your engineering team? Contact the Cosine team today to discuss your infrastructure needs.

__________________

¹ Based on an illustrative maximum system draw of ~10.2kW per 8x H200 node, ~1.3x PUE/facility overhead, and an average electricity pricing for UK businesses of ~£0.23/kWh. This produces an indicative annual electricity and cooling/facility cost of approximately ~£27k/year assuming continuous operation.

² Represents indicative annual support, rack space, networking, maintenance, orchestration, monitoring, and operational overhead associated with running sovereign GPU infrastructure in an enterprise environment. Actual operational costs will vary depending on deployment model and internal support structure.