Cybersecurity by Design in AI Systems

A cross-cutting case, built from several projects

This is not the account of a single installation. It is a cross-cutting technical case gathering patterns BigLearn applies in solutions for insurance, healthcare, public administration, hospitality and retail: knowledge-based assistants, complaint handling, channels carrying sensitive data, order flows and agents connected to internal processes.

Clients are anonymised and we publish no incident metrics we cannot demonstrate. The verifiable result is the architecture: where language ends, where authorisation begins, which data may circulate, and at which point a person has to decide.

The dangerous mistake: confusing intelligence with authority

A language model is useful because it accepts ambiguous input: an email, a complaint, a commercial question, a document. That same flexibility creates the attack surface. Incoming content can carry hostile instructions; a retrieved document can be out of date or poisoned; and a plausible answer can still be factually wrong.

The architectural decision was to treat input, retrieved context and model output as untrusted data. The model interprets and proposes. Deterministic components verify identity, authorisation, schema, limits and process state before anything happens.

Threat model before the first prompt

For each flow we began by identifying assets, trust boundaries and possible effects. The assets were not only databases: they included system instructions, internal documents, user identity, integration credentials, conversation history, and the very ability to send, alter or classify information.

The adversarial scenario included direct and indirect prompt injection, context leakage, cross-tenant access, tool abuse, manipulated parameters, unsourced answers, request replay, excess data in logs and privilege escalation. That changes the question from «does the model answer well?» to «what happens when it answers badly?»

The technical controls applied

untrusted language → structured intention → policy validation → authorisation → minimal execution → logging → response

Prompt injection: the filter is not the answer

An instruction like «ignore the previous rules» is the simplest example. The real problem is indirect: the instruction can sit inside a PDF, on a page the agent consults, or in an email that looks like a legitimate complaint. Trying to recognise every dangerous phrase is a useful defence, but an incomplete one.

The effective defence reduces the possible impact: the document does not redefine policy; the model holds no secrets; tools are limited; arguments are validated; a sensitive operation requires independent authorisation. So even when an attempt gets through the language layer, it meets a technical boundary that does not speak prompt.

Secure RAG: finding a document is not permission to reveal it

In knowledge-based solutions, retrieval is conditioned by the identity and context of the request. Filtering happens before excerpts are sent to the model. That prevents the model receiving information the user had no access to and then trying to «forget» it in the answer — a protection that would already be too late.

Company knowledge stays separate from the model. It can be updated, versioned and withdrawn without retraining anything. When there is not enough source material, the correct answer may be not to answer, to ask for more context, or to hand the case to a person.

The pattern applied across sectors

Insurance: the assistant consults approved knowledge, gathers data progressively and hands parameters to a deterministic process. The calculation belongs to the business engine; the model invents no premiums and takes no underwriting decision.

Healthcare and sensitive channels: AI can organise an account and extract useful information, but it does not decide about people. Access, retention and routing are defined by the process, and the final decision stays human.

Public administration: emails and complaints are untrusted input. The solution separates the original text from the extracted fields, classifies and geolocates, keeps traceability, and proposes the responsible department without turning the model's suggestion into an administrative decision.

Hospitality and retail: opening hours and public information can be answered with low friction; orders, status changes and guest data pass through validation, access control and logging. Conversational freedom does not become operational freedom.

What became reusable

The main result was a consistent way of building: an inventory of assets and data, classification of actions by impact, explicit trust boundaries, authorised retrieval, small tools, validated output, human intervention, and enough operational evidence to investigate an error. These patterns reduce the security work in new projects without assuming two sectors carry the same risk.

Security by design does not mean promising zero risk. It means knowing what the system is able to do, reducing the blast radius when it fails, and being able to reconstruct what happened. In an AI agent, that architectural honesty is worth more than a prompt that promises to obey.