Cloud AI or local compute: what should a business compare?

AI compute is the processing capacity used to run or develop a model. For a business choosing a tool, the practical issue is how that capacity is supplied, paid for and operated. Buying a powerful machine is only one option, and often not the first decision to make.

Separate training a model from using one. A team summarising approved documents has different requirements from a research group developing a new model. Start with the work, the volume, the acceptable delay and the information involved. Those requirements are more useful than a headline about the size of a data centre.

Compare the operating options

A managed business application

A hosted application may be the simplest route when it already performs the required task. The supplier operates the infrastructure while your business manages users, inputs and the workflow. Check the specific plan's limits, retention, connected services, export options and support arrangements. A familiar product name does not answer those questions.

A hosted model API

An API can support a custom workflow without your team running the model infrastructure. You still need to manage application access, usage costs, error handling and the systems around the model. Include development and support effort in the comparison. A low price for a single request says little about a complete working service.

A privately operated model

Running a model on equipment you control gives the team additional infrastructure responsibilities. Someone must maintain the operating system, model files, software dependencies, access controls, storage and recovery. Check the model licence and the performance of the actual workload. Keeping a machine in an office is not, by itself, a complete privacy or security design.

A mixed arrangement

Different tasks may justify different routes. A team might use a controlled local process for one document task and a hosted application for another. Document which information moves between them. A hybrid label is not useful unless the data paths and operational responsibilities are clear.

Map the infrastructure around the model

Model execution is one part of a working service. Record who operates the application, model runtime, document storage, search index, network connections and backups. For each part, identify its administrator, update owner, access boundary and recovery route. A managed model API does not also manage the business application built around it.

Check shared dependencies before describing a setup as resilient. Two applications may rely on the same model provider, cloud account or identity service. Ask what happens if that dependency is unavailable or its terms change. Keep the information needed for a manual fallback and test an export of non-sensitive sample work. A second supplier is useful only if the alternative can perform the required task and the team knows how to switch.

The NCSC cloud security principles provide a framework for assessing the provider, its supply chain and the controls around the service. Apply those questions to the actual configuration and evidence, not just a supplier's assurance statement.

Build a workload record

• Task: the output the user needs and the quality checks it must pass.
• Inputs: typical file sizes, formats, sensitivity and permitted processing locations.
• Demand: average and peak workload, including several people using it together.
• Response time: what users can tolerate, and what happens when the service is slow.
• Availability: when the workflow must run and the acceptable interruption.
• Recovery: the manual alternative, retained records and person responsible for repair.

Use representative examples rather than the smallest demonstration file. Include a long document, missing information and a busy period. Measure whether the output remains useful, how long it takes to review and what it costs to repeat a failed task.

Count more than the hardware or licence

For hosted services, record subscription or usage charges, storage, integrations and administration. For local equipment, include purchase, electricity, cooling, replacement, backups and the time of the people operating it. Both options need testing and support. Treat a cost estimate as an assumption until the pilot supplies real usage.

A fictional business comparing document-summary options could test the same approved sample through a managed application and a small internal prototype. It would record useful summaries, unsupported statements, review effort, waiting time and total cost. It would not choose the local machine merely because its monthly cloud bill looked lower.

Data location is one part of the decision

Ask where input, output, logs and backups are processed or stored, who can access them, and which services receive information. Check the contract and the configuration together. “UK hosted”, “private” and “sovereign” need a precise explanation for the actual product; they do not replace a review of access, retention or legal obligations.

The NCSC cloud security collection is a useful starting point for assessing cloud services. Use it alongside the controls and obligations relevant to your organisation.

Sovereignty and control: separate the questions

Agree what “sovereign” is intended to mean for this project. Consider each of these independently:

• Location: where the model runs and where inputs, conversation history, outputs, logs and backups are held.
• Access: which staff, administrators, providers and subcontractors can retrieve or change that material.
• Jurisdiction: which organisations and legal arrangements are involved, assessed with appropriate advice where needed.
• Operation: who can change the model, update the software, stop processing and recover the service.
• Portability: what the business can export, which formats it receives, and what happens to retained copies when the arrangement ends.

A local model does not prove that the complete application works offline or sends no external traffic. Check connected tools, remote model calls, updates and telemetry in the proposed setup. Test the agreed offline mode using non-sensitive material rather than relying on the location of the machine. Local hosting alone does not establish security or compliance.

Use the same questions for private cloud and managed services. The right comparison is the control the business needs and can maintain, including conversation-history access, export and deletion, rather than the deployment label alone.

Make an operational decision

Choose the route your team can support. A local system with no maintainer can be a worse choice than a managed service with well-defined controls. Equally, an API integration can be a poor choice if its data handling or recovery route does not fit the work.

For an independent starting point, define the workflow through an AI consultancy discussion. Ask for a comparison of practical options and their limitations, not a recommendation to buy infrastructure before the task is understood.