Private AI for an enterprise should begin with a clear account of where information goes and who can use it. A product name, a private-cloud label or a promise that data is not used for model training does not answer every privacy and security question.
The practical aim is to choose controls that fit the information and the business task. That may mean a managed business service, a cloud application with restricted access, a locally hosted model or a combination. This guide compares those choices and gives you a worksheet for reviewing the whole workflow.
Write down the job the system must do, the information it needs and the consequences of a wrong answer or disclosure. An assistant answering questions about a public product manual has a different boundary from one searching customer contracts. Neither should receive additional records just because a connector makes them available.
Turn broad demands into testable requirements. “Keep it private” might mean approved users only, a specified processing region, no external network access, a defined retention period or an exportable audit record. Those are different requirements. Identify who can confirm each one and what evidence they need.
| Pattern | What to investigate | Work the business still owns |
|---|---|---|
| Managed business AI service | Account type, processing terms, connected sources, access controls, sharing, retention and service dependencies. | Approve use cases, configure access, manage staff changes and check outputs and incidents. |
| An application in a controlled cloud environment | Which parts you operate, which model/API providers receive information, network routes, service identities and logging. | Secure the application and integrations, manage configuration and verify the complete data path. |
| Model hosted on your own infrastructure | Hardware needs, model licence, runtime, updates, backups, remote administration and outbound connections. | Operate, patch, monitor and recover the service, as well as govern the data and users. |
A hybrid design can use more than one pattern. Draw each hand-off: a local document store may still send selected text to an external model. The NCSC cloud guidance provides a useful structure for assessing shared responsibility and provider controls. Compare actual configurations, not only sales descriptions.
Hosting is where the components run and who operates them. It does not establish which records a user may see. A model on an office server can still be exposed through an overly broad account, an insecure application or an external integration.
Retrieval finds relevant material and supplies it as context for an answer. Microsoft's retrieval-augmented generation documentation describes this pattern and the need for granular access control. A knowledge assistant should retrieve only material the requesting user is entitled to use. Adding documents to a search index is not the same as making every document safe for every employee.
Fine-tuning trains a model using examples to change how it behaves on a task. It is not a substitute for a current, permission-controlled document library. Separate the training dataset, trained model, evaluation records and inference service in the review. OpenAI's model optimisation guidance explains the distinction between context, prompting, evaluation and training; check current provider and model availability before proposing a fine-tuning route.
Begin with a simpler prompt or retrieval test if it can meet the requirement. Do not commission training merely to make a system sound more bespoke. Whatever approach you choose, a fluent answer still needs appropriate checking.
Use this worksheet with the supplier or technical owner. Keep it alongside a diagram showing the services and connections. An unanswered cell is a question to resolve, not permission to assume the safest behaviour.
| Stage | Record these details | Evidence to request |
|---|---|---|
| Input | Allowed document types, sensitivity, owner and minimum fields needed. | Approved sample and written exclusions. |
| Ingestion | Uploads, connectors, accounts, permissions and temporary copies. | Connection configuration and an access test. |
| Index or storage | Documents, extracted text, embeddings, location and access rules. | Storage configuration and permission mapping. |
| Model processing | Provider, exact service, deployment type and data sent per request. | Applicable terms and documented processing route. |
| Output | Who sees answers, source excerpts, exports and shared links. | Tests using different user roles. |
| Logs and support | Prompts, responses, diagnostics, access, retention and support visibility. | Log settings and a representative redacted record. |
| Deletion and exit | Originals, indexes, histories, caches, backups and exports. | Documented retention rules and a tested removal/export procedure. |
Treat search indexes and logs as information stores in their own right. A document disappearing from a source folder does not by itself prove that every derived copy, conversation or backup has disappeared. Establish what deletion means in each component and which retention exceptions apply.
OpenAI's API data controls state that API data is not used for model training by default unless a customer opts in. The same documentation separately describes abuse-monitoring logs and application state. Enhanced retention controls require eligibility and have endpoint or feature limitations. A no-training default therefore must not be described as a universal no-storage promise.
Microsoft's Models sold by Azure in Microsoft Foundry distinguishes prompts, uploaded material, stored state and training data. It also explains that processing location depends on deployment type, including different treatment for Global and DataZone deployments. Confirm the actual resource and feature configuration rather than assuming every Azure deployment has the same boundary.
These are examples of what to check, not endorsements or a complete comparison of the services. Record the exact product, account, feature and terms you are assessing. Third-party connectors and tools may add separate providers and retention arrangements to the path.
Ask for evidence appropriate to the risk: documentation, configuration, an agreement, a demonstration or an independently assessed control. A security certificate can inform the assessment, but check its scope and the service it covers. It does not establish that your particular workflow is correctly configured or meets every obligation.
The NCSC's secure-use principle makes customer configuration a continuing responsibility. Involve the people accountable for security, information governance and the business process before giving the system sensitive records.
Imagine an internal assistant containing general procedures and two fictional customer folders. The sales test account may read the procedures and Customer A; the operations test account may read both customers. Use synthetic records throughout the exercise.
Ask both accounts the same questions. Test a direct request for Customer B's information from the restricted account, an ambiguous customer name, a document whose permissions have changed and an answer that quotes a restricted source indirectly. Inspect both the retrieved material and final answer. A refusal written by the model is not sufficient if restricted text already reached the wrong user or log.
Also include an outdated procedure, a missing answer and a document containing instructions to ignore the task rules. Check that the application treats document text as source material and that the approved access boundary still holds. This is a proposed test, not evidence that a Blue Canvas deployment passed it.
For a local or private-cloud system, name the people responsible for patches, model and software updates, credentials, backups, monitoring and recovery. Check whether the runtime, extensions, telemetry, remote support or update mechanisms contact external services. If offline operation is a requirement, test it under the intended network restrictions.
Include the time and skills needed to maintain the system in the cost comparison. The NCSC secure-development principle covers maintained dependencies, secrets and controlled development. These remain practical concerns when a system runs on equipment you own.
Agree the pilot's allowed inputs, expected answers, failure cases, reviewer and stop conditions. Measure factual errors, inappropriate access, review effort, response time and operating cost separately. Keep a record of what has been tested and what remains unresolved.
A new connector, model, document collection or user role can change the data path. Review material changes and repeat the relevant tests. The NCSC AI development guidance treats operation and maintenance as part of the security lifecycle, alongside design and deployment.
For broader hosting economics, see our cloud AI and local compute checklist. To discuss a specific workflow and the evidence needed before implementation, arrange a free conversation with Blue Canvas.


Talk to Blue Canvas. Bring one task that takes too long.
In a free 15-minute call we will tell you whether AI can help, and what a sensible first step would cost.