Prompt injection and data leaks: a defence guide for staff

AI tools are useful, and they have two weaknesses that ordinary software does not. They can be tricked by the content they read, and what you give them may end up somewhere you did not intend. This guide explains both in plain English, then gives you short rules for staff and a checklist for the people who build or run AI systems.

Not legal advice. This is general security guidance. Follow your own organisation's policy where it differs.

The two risks in one minute

Prompt injection is when text that the AI reads contains instructions, and the AI follows them as if you had given them. The text might be in a web page, an email, a PDF, a spreadsheet, a code file, a calendar invite or a support ticket. You do not have to type anything malicious yourself for it to happen.

The NCSC says the root cause is that language models do not enforce a boundary between instructions and data. It describes the model as an "inherently confusable deputy", and says prompt injection may never be fully mitigated in the way SQL injection can be. That is why the NCSC's advice is to design systems so that a confused model cannot do serious harm, not to rely on filters.

Data leaks are when sensitive information ends up where it should not be. Usually this is a person pasting something into a public chat tool. It can also happen when an AI agent is tricked into sending data out, or when stored prompts and outputs are exposed.

On the first: the NCSC has long advised not to include sensitive information in queries to public AI tools, and to avoid submitting anything that would cause problems if it became public. The provider can see your queries, and they are likely to be stored and used in developing the service.

What a real attack looks like

Here are three patterns. The first is an illustration of how indirect prompt injection works. The other two are from published research.

1. The hidden instruction. You ask an AI assistant to summarise a web page or an email. The page contains hidden text telling the assistant to look up your recent documents and put their contents in a link it then shows you. If you click, the information goes to the attacker.
2. The poisoned project folder. A developer opens a coding agent in a folder that contains a file written for the agent, such as an AGENTS.md or CLAUDE.md. Security firm Forever Security reported in October 2026 that a malicious AGENTS.md left in a Downloads folder made one coding agent install a package it chose, and that a CLAUDE.md variant could redirect where the agent sends its requests. The firm's success rates come from its own lab tests and are not independent, but the pattern is real: agents obey instructions found in files.
3. The helpful troubleshooting message. CrowdStrike's 29 October 2026 update to its prompt injection taxonomy describes a hypothetical in which a tampered test tool prints a "fix" telling the agent to send local configuration files, including credentials, to an outside address. Each step looks routine on its own. CrowdStrike's point is that agent security has to cover the plan, not just the prompt.

Rules for everyone

1. Never paste secrets. Passwords, API keys, tokens, private keys and connection strings never go into an AI chat tool. Not "just to debug".
2. Use only approved tools, with your work account. Personal accounts and unknown browser extensions are where data goes missing.
3. Strip it down. Remove names, account numbers and anything you do not need before you paste. Give the AI the minimum.
4. Treat what AI reads as untrusted. Be careful asking an AI to act on content from outside your organisation: web pages, inbound email, attachments, shared documents, open-source code.
5. Read before you approve. When an agent asks to run a command, send something, install something or change a setting, read what it will do. If you do not understand it, say no and ask.
6. Keep approvals switched on. Do not turn off permission prompts to save time.
7. Be suspicious of unexpected behaviour. If an AI suddenly asks you to visit a link, enter credentials, install software, or send information it has not needed before, stop and report it.
8. Open agents only in places you trust. Do not start a coding or desktop agent in your Downloads folder or in a repository you have not reviewed.
9. Check what it gives you. AI makes things up and can be steered. Verify facts, and review code and commands before you run them.
10. Report quickly. If you pasted something you should not have, or an agent behaved oddly, tell your security contact straight away. Early reports are far cheaper than late ones.

What not to put into an AI tool that is not approved for it

The OWASP Top 10 for LLM Applications in plain English

The OWASP Top 10 for LLM Applications (2025 edition) is the most widely used list of risks for systems built on language models. Prompt injection is number one.

For IT and builders

The NCSC's published advice, plus a few practical additions of our own, can be summarised as follows.

• Assume the model can be fooled, and put hard limits around what it is able to do. Safeguards should be deterministic and enforced outside the model, not instructions in a prompt.
• Least privilege. Give the model and each tool only the access that the task needs. If a tool can delete or send, a tricked model can delete or send.
• Separate trust levels. Do not let content from an untrusted source steer a tool with access to sensitive data without a control in between.
• Do not rely on deny-lists. The NCSC specifically warns against trying to block phrases like "ignore previous instructions". Attackers can rephrase.
• Log everything that matters: inputs, outputs and tool calls, in a form that cannot be quietly altered. The NCSC's agentic AI guidance also asks for each agent to have its own identity, short-lived credentials, and for agent activity to be attributable to an owner.
• Set the oversight level on purpose: whether a person approves each action, supervises, or is out of the loop, depending on impact. Include a way to stop the agent immediately.
• Sandbox agents. Limit their network and compute, and restrict outbound connections to what they need.
• Check agent configuration. Look for unexpected instruction files (such as AGENTS.md or CLAUDE.md), added tool servers or changed base URLs. Keep approvals and sandboxes on, and rotate tokens if you find something unexpected.
• Test for abuse. The MITRE ATLAS knowledge base and CrowdStrike's public prompt injection taxonomy list techniques that you can use to plan tests.

Questions people ask

Can we just filter out malicious prompts? Filters help at the edges, but the NCSC warns that treating prompt injection like SQL injection is a mistake, because a model does not separate instructions from data. Design for limited damage rather than perfect detection.

If the AI company says it does not train on our data, are we safe? Not automatically. Training is one issue. You also need to know who can read the data, how long it is kept, where it is processed, and what happens if the provider is breached or sold. See our vendor questionnaire.

Is a private or self-hosted model safe to feed sensitive data? It can be, after a proper security assessment, according to the NCSC, but it is not free of risk and is likely to be expensive. The terms of use and who can see queries still matter for cloud-provided models.

Does this only matter for developers? No. Anyone who asks an AI tool to read an email, a document or a web page is exposed to indirect prompt injection. The more the tool can do for you, the more that matters.