On-Premise vs Private Cloud AI in Europe

On-premise AI runs models on servers the organisation owns and operates; private cloud AI runs them on dedicated capacity at a cloud provider. For regulated sectors the deciding factors are who can access the data, which jurisdiction the provider falls under, and how the organisation exits. A European-owned private cloud often gives most of the control with less capital.

This guide compares the options for AI workloads in Europe and maps them to sector rules. For the technical side of running models yourself, see the self-hosted LLM guide.

What is private AI?

Private AI means running AI models in an environment the organisation controls, so that prompts, documents and outputs do not pass through a shared public AI service. VMware, which uses the term for a product line, describes it as "an architectural approach that aims to balance the business gains from AI" against privacy and compliance requirements.1

In practice private AI takes one of two forms. On-premise: the organisation owns the servers and GPUs, in its own data centre or in a rented rack at a colocation site, and its own staff run them. Private cloud: a provider owns and runs the hardware, and the organisation rents dedicated capacity under a contract. Both keep open-weight models, data and logs under the organisation's policies. They differ in who carries the capital, the operational work and the legal exposure of the operator.

On-premise, colocation or private cloud: what is the difference?

Four options cover almost every European deployment. The table compares them on the questions a risk committee asks first.

Exhibit 1
Only ownership of the operator changes the legal reach over the data.

Operator under EU jurisdiction only Operator with a US parent

Sources: 18 U.S.C. 2713; provider terms; accessed 28 September 2026. Colocation: the site operator may have a US parent, but the hardware and access stay with the customer. Qualitative comparison; see the text for costs.

Colocation is often overlooked. The organisation owns the servers and holds the keys, while the colocation provider supplies power, cooling and connectivity. It avoids building or upgrading a data centre for GPU servers and keeps operational control in-house.

Among cloud options, ownership varies. The European GPU providers and their ownership are listed in the Lindstead price index: Scaleway and OVHcloud in France, Hetzner, IONOS and STACKIT in Germany, Verda in Finland and Nebius in the Netherlands. Some, such as Hetzner, rent dedicated GPU servers rather than shared, on-demand capacity.

Why does the provider's jurisdiction matter?

Under the US CLOUD Act, a provider must disclose data in its "possession, custody, or control", regardless of whether the data is located inside or outside the United States.2 The obligation follows the company, not the data centre. In June 2025, Microsoft France's director of public and legal affairs told a French Senate inquiry under oath that he could not guarantee that French citizens' data would never be transferred to US authorities without French consent, while adding that it had not happened.3

Transfers to the United States currently rest on the EU-US Data Privacy Framework. The General Court upheld it in September 2025, and an appeal is pending before the Court of Justice as case C-703/25 P.4,5 As of September 2026 no judgment has been found. The accurate description is legal uncertainty, not a ban.

For public procurement, the European Commission published a Cloud Sovereignty Framework in October 2025. It scores cloud offers against eight sovereignty objectives, with assurance levels from 0, no sovereignty, to 4, full digital sovereignty.10 It was written for the Commission's own cloud tenders, but it gives other buyers a structured way to compare "sovereign" claims.

What do sector rules require?

No EU rule requires AI to run on-premise. Sector rules require control, documented risk management and a way out. The table shows what that means for hosting.

Exhibit 3
Each sector has a different reason to keep AI under control.

Sources: Regulation (EU) 2022/2554; Regulation (EU) 2016/679; Rijksoverheid, 7 July 2026; accessed 28 September 2026.

• Financial services. DORA has applied since 17 January 2025. For ICT services supporting critical or important functions, financial entities must put exit strategies in place.6 On 18 November 2025 the European supervisors designated 19 critical ICT third-party providers, including AWS, Google Cloud and Microsoft.7
• Healthcare. Data concerning health is a special category under Article 9 of the GDPR; processing is prohibited unless one of the listed exceptions applies.8 Sending patient records to an external model adds a processor and often a transfer to assess.
• Energy and critical infrastructure. The Dutch Cyberbeveiligingswet, which implements NIS2, applies from 15 August 2026 and brings new obligations to more than 8,000 organisations, including supply-chain security.9
• Public sector and legal services. Citizen data, procurement rules and professional secrecy all favour infrastructure where the organisation decides who has access.

The AI Act is the exception: its obligations depend on role and use case, not on hosting. A model used for credit scoring is high-risk from 2 December 2027 whether it runs on-premise or in the cloud.14 The AI Act timeline lists every date. Lindstead maps DORA, GDPR and NIS2 obligations to your AI systems before a hosting decision is made.

What does each option cost and how long does it take?

Own hardware turns an operating cost into a capital investment, and 2026 made that investment more expensive. The list price of an NVIDIA RTX PRO 6000 Blackwell on NVIDIA's US marketplace rose to $16,000, about €14,000, in August 2026.11 Renting a dedicated server with one such GPU from Hetzner in Germany costs €1,199 a month plus a one-off €599 setup fee.12 On-demand cloud GPU prices rose too; current figures are in the European GPU price index.

Independent academic research (Pan et al., arXiv 2509.18101) finds that owned hardware pays back within about three months for small models, within 6 to 24 months for medium models and often only after more than two years for large ones, and is mainly viable at high, steady volumes or under strict residency requirements.13 The Lindstead break-even calculator shows the monthly token volume at which own GPUs beat an API.

Beyond the hardware, three costs decide the comparison:

• Staff. On-premise and colocation need people who run GPU servers, inference software, security patching and model updates. In a private cloud the provider carries the hardware layer.
• Utilisation. Owned GPUs cost the same whether they run at 10 or 90 percent. Rented capacity can follow demand.
• Time. Cloud capacity is available once the contract is signed. Own hardware depends on supplier delivery and data-centre readiness; ask suppliers for dated delivery commitments before planning around them.

How do you decide? Five questions in order

1. How sensitive is the data? Classify each use case. Public or internal data can often use an API. Special category data, privileged files and trade secrets point to private deployment.
2. Which jurisdiction may the operator fall under? If exposure to non-EU disclosure orders is unacceptable, the options narrow to on-premise, colocation or an EU-owned cloud.
3. How large and steady is the volume? High, predictable utilisation favours owned hardware. Variable or uncertain demand favours rental.
4. Can the organisation operate GPU infrastructure? Without a team for servers, security and model updates, a private cloud is the realistic start.
5. What does exit require? Regulated entities need a tested exit plan. Open-weight models and standard inference software make moving between on-premise and cloud a contract question rather than a rebuild.

For most regulated organisations the answers lead to the same sequence: prove the use case in an EU-owned private cloud, then move steady, sensitive workloads onto owned hardware when utilisation justifies it. Lindstead delivers a first production use case on-premise or in a European cloud, with the logging and access controls auditors expect.

Can you combine on-premise and private cloud?

Yes, and most mature set-ups do. Three patterns recur:

• Sensitive on-premise, the rest in the cloud. Client files and health data stay on owned hardware; development, testing and burst capacity run in an EU-owned cloud.
• Private models, external APIs for public data. Marketing copy or public research can use a commercial API, while confidential work runs on a private model. A routing policy decides per request type.
• Cloud first, hardware later. The same open-weight model and inference stack run in a rented environment until volume and certainty justify buying servers.

A hybrid set-up only works with one policy for data classification, logging and model versions across all environments. For the regulatory and cost picture behind these choices, see the 2026 European briefing on where AI should run.

Frequently asked questions

• Is private cloud AI compliant with DORA?

  It can be. DORA does not prohibit cloud services; it requires financial entities to manage ICT third-party risk, include specific contract terms and put exit strategies in place for ICT services that support critical or important functions. A private cloud contract that meets those requirements, with a tested exit plan, is compatible with DORA. The choice of provider also matters: 19 providers, including AWS, Google Cloud and Microsoft, are designated as critical and overseen directly by the European supervisors.

• Does hosting AI in an EU region of a US cloud avoid the CLOUD Act?

  No. The CLOUD Act obliges US providers to disclose data in their possession, custody or control whether it is stored inside or outside the United States. Choosing an EU region fixes where the data sits, not which law can compel the provider. Encryption with keys the customer alone holds reduces the exposure; an EU-owned provider or own hardware removes the question.

• What is the difference between private AI and sovereign AI?

  Private AI describes where and how models run: in an environment the organisation controls, so data and prompts do not go to a shared public service. Sovereign AI adds a jurisdictional condition: the infrastructure, operator and often the model are subject only to European law. A private deployment at a US-owned cloud is private but not sovereign in that sense.

• Can a bank run an LLM on its own servers?

  Yes. Open-weight models such as Mistral Small, gpt-oss or Gemma run on a single server with one or a few GPUs, and larger models on an eight-GPU node. The bank then carries the operational work itself: security, monitoring, model updates and the model risk management that supervisors expect. The regulatory case is strongest for client data and confidential documents.

• Does on-premise hosting reduce AI Act obligations?

  No. AI Act duties depend on the organisation's role and the use case, not on where the model runs. An organisation that uses a model for internal work is usually a deployer with the same obligations on-premise as in the cloud. Hosting affects GDPR, DORA, NIS2 and data-access exposure, not the AI Act classification.

Lindstead advises regulated organisations on where their AI should run and deploys a first use case on infrastructure they control. The self-hosted LLM guide covers models, hardware and the step-by-step plan; schedule a meeting to discuss your options.

Sources

1. VMware, VMware Private AI: privacy and security best practices, technical white paper, May 2024. www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/techpaper/vmware-private-ai-foundation-privacy-security.pdf
2. 18 U.S.C. 2713, Required preservation and disclosure of communications and records (CLOUD Act), Cornell Legal Information Institute. www.law.cornell.edu/uscode/text/18/2713
3. The Register, Microsoft admits it 'cannot guarantee' data sovereignty, 25 July 2025. www.theregister.com/2025/07/25/microsoft_admits_it_cannot_guarantee/
4. Court of Justice appeal in Latombe v Commission, case C-703/25 P, notice in the Official Journal, 22 December 2025. eur-lex.europa.eu/eli/C/2025/6610/oj/eng
5. WilmerHale, European Court of Justice to review challenge to EU-U.S. Data Privacy Framework, 1 December 2025. www.wilmerhale.com/en/insights/blogs/wilmerhale-privacy-and-cybersecurity-law/20251201-european-court-of-justice-to-review-challenge-to-eu-us-data-privacy-framework
6. Regulation (EU) 2022/2554 (DORA), Articles 28 and 64. eur-lex.europa.eu/eli/reg/2022/2554/oj/eng
7. European Supervisory Authorities, designation of critical ICT third-party providers under DORA, 18 November 2025. www.eiopa.europa.eu/european-supervisory-authorities-designate-critical-ict-third-party-providers-under-digital-2025-11-18_en
8. Regulation (EU) 2016/679 (GDPR), Article 9. eur-lex.europa.eu/eli/reg/2016/679/oj/eng
9. Rijksoverheid, Cyberbeveiligingswet en Wet weerbaarheid kritieke entiteiten vanaf 15 augustus 2026 van kracht, 7 July 2026. www.rijksoverheid.nl/actueel/nieuws/2026/07/07/cyberbeveiligingswet-en-wet-weerbaarheid-kritieke-entiteiten-vanaf-15-augustus-2026-van-kracht
10. European Commission, Cloud Sovereignty Framework, version 1.2.1, October 2025. commission.europa.eu/document/download/09579818-64a6-4dd5-9577-446ab6219113_en
11. Tom's Hardware, NVIDIA RTX PRO 6000 Blackwell list price on NVIDIA's US marketplace, 12 August 2026. www.tomshardware.com/pc-components/gpus/nvidia-doubles-rtx-pro-6000-blackwells-msrp-to-a-staggering-usd16-000-96gb-card-started-pre-orders-below-usd8-000-last-year
12. Hetzner, GEX131 dedicated GPU server, product page. www.hetzner.com/dedicated-rootserver/gex131/
13. Pan et al. (Carnegie Mellon University and independent researchers), cost-benefit analysis of on-premise LLM deployment, arXiv 2509.18101. arxiv.org/abs/2509.18101
14. Regulation (EU) 2026/1744 (Digital Omnibus on AI), Official Journal, 24 July 2026. eur-lex.europa.eu/legal-content/EN/TXT/?uri=OJ:L_202601744

Method: desk research of public sources, accessed 28 September 2026. Dollar prices converted at the ECB reference rate of 25 September 2026. This guide is general information, not legal advice. Corrections: contact@lindstead.com.