↓ Skip to main content
fr1zz0

Prompt injection: telling intents apart from techniques

·10 mins·
AI Prompt-Injection Llm Ai-Security Jailbreak
Table of Contents
AI Security - This article is part of a series.
Part 1: This Article

For the first two weeks I spent studying prompt injection I did the wrong thing: I collected prompts. I would read a thread, find a phrasing that had worked against some model, and paste it into my notes. I ended up with a sixty-line file that explained nothing. Sixty one-off cases.

Most of the material online is organised the same way: copy-pasted prompts that work, and that stop being useful at the next release. On top of that, easy as those prompts are to pick apart, they don’t really help you build a defence that holds.

So I threw my prompt collection away and started thinking about what made those prompts work in the first place, and what the core points these attacks are built on actually are.

Five things I was all calling “technique”
#

What unstuck me was realising that an injection attempt contains things of different kinds, and I was keeping them all in one list:

  1. the intent, meaning what I want to get
  2. the technique, meaning how I restructure the conversation to get it
  3. the evasion, meaning how I slip the text past the filters
  4. the amplifiers, which don’t change the technique but make it reliable
  5. the utilities: automation, variant generation, measuring results

Written out it looks obvious. It wasn’t in my notes, where leetspeak and narrative injection sat side by side as if they belonged in the same category. The first one disguises the text, the second one restructures the context: two operations of a different nature, and as long as I kept them together I couldn’t reason about either. In practice they almost always combine, but that doesn’t make them the same thing, and calling them by one name only costs you clarity.

What I want to get
#

Intent is the part that matters for a threat model. Take the system prompt leak as our example: getting the model to hand back the instructions it was configured with.

The risk isn’t the text itself. Those instructions describe what the application permits and forbids, which tools it can call, which data it treats as its own. Reading them means having the authorisation logic in front of you in plain text, and knowing exactly where to push on the next attempt. A leak is rarely the damage. It’s the reconnaissance.

There are two ways to get there. In a single turn, with one message built to obtain the instructions straight away. Or across several turns, building the context up bit by bit: you open with innocent requests, establish a convention, and only then ask for the part you actually want.

The second interests me more than the first because it moves the target. There’s no malicious message to intercept: taken one at a time, none of those turns looks suspicious. The trajectory is the attack, and a defence that looks at one message at a time has no way of seeing it.

How to talk a model round
#

Nine classes from my notes, covering some of the techniques in use:

  • Narrative injection: wrapping the request in fiction, a story, a character. It moves the request from “this is an order” to “this is what happens in a story”.
  • Token smuggling: the model doesn’t read words, it reads tokens. Splitting a term, or having it reassembled at runtime, changes what a filter sees without changing what the model understands.
  • End sequence: dropping into the text the markers that the conversation format uses to say “the user’s turn ends here”.
  • Russian doll: nesting the payload in several layers of encoding, then asking the model to unwrap them.
  • Polarity inversion: rephrasing by inverting, say asking what not to do so that the list of things not to do is the answer.
  • Priming: dictating the opening of the response, because a sentence that starts with a yes is much harder to break off with a refusal.
  • Mistake and reorientation: claiming something went wrong and that the earlier instructions were mistaken, so the model follows you into the fix.
  • Fake system tags: writing markers that look like they come from the layer above, and adding rules as if they had arrived from higher up.
  • Reiteration: restating an instruction as if it were a reminder you had already agreed on, rather than putting it as a fresh request.

Three of these are not the way to go, however often you still find them treated as the heart of the subject. Token smuggling, end sequence and russian doll all work on the shape of the string, and a model like Opus 5 recognises them for what they are. Most of the material online leans on exactly these, which is one reason it ages badly. What holds up is elsewhere: in the techniques that act on the context and on the apparent authority of whoever is speaking, and above all in the multi-turn case, where there is no single message to recognise.

The boundary is a convention
#

The model doesn’t receive “a system instruction” and then “user data” as two separate objects. It receives one flat sequence of tokens. The separation exists because the conversation format draws it with markers, and because training taught the model to treat those markers as authoritative.

That’s a convention, not a mechanism. A sturdy one, but if you can write something that looks like a boundary, you are playing on exactly that difference.

For a defender the consequence is uncomfortable: no amount of input sanitisation fixes this. You aren’t filtering code against a grammar, you’re filtering natural language, where the same meaning has endless forms.

I thought that was just my hunch, and it turns out somebody measured it. In October 2025 a group of fourteen researchers from OpenAI, Anthropic and Google DeepMind took twelve published defences against jailbreaks and prompt injection and attacked them adaptively, changing their strategy to suit each defence’s design instead of throwing fixed strings at it. They got past nearly all of them with a success rate above 90%, and human red teaming hit 100% on every single one. These are defences whose original papers reported failure rates close to zero. If you ever need an argument for not trusting the numbers of a defence evaluated against static examples, that’s the one.

The mitigations left standing aren’t in the text, they’re in the architecture. Simon Willison named the combination to avoid, the lethal trifecta: access to private data, exposure to untrusted content, and the ability to communicate externally. Give an agent all three and one poisoned document is enough to walk your data out of the building. Meta turned that into an operating rule, the Rule of Two: until research can reliably spot injections, an agent should hold no more than two of those three properties in the same session, and if it needs all three then it doesn’t run on its own, a human has to approve.

Amplifiers, the category I was missing
#

This category was the last one I added, and I added it because I had nowhere to put two things I kept running into that are neither techniques nor evasions. They don’t change what you ask. They change how often it works.

Many-shot fills the context with many examples of the behaviour you want, until the pattern sitting in the conversation outweighs the behaviour learned in training. This isn’t folklore, it’s published research, and what struck me is that effectiveness grows with the number of examples along a power law, the same one you see in in-context learning on harmless tasks. So it isn’t a flaw of its own: it’s the flip side of how models learn from the examples you put in front of them. Two uncomfortable consequences follow. The lever scales with the context window, and the models that are better at learning in context are also the more exposed.

Prefill means putting words in the model’s mouth by filling in the start of its response. Several APIs expose it as a parameter because it’s very handy for controlling output format, but it’s also a strong lever: a refusal is much harder to produce once the response has already opened with a yes.

Of the two, prefill is the more instructive case, because it has already been shut. From Claude 4.6 onwards, a request carrying a prefilled assistant message on the last turn returns a 400 error, and the migration guide lists steer around unnecessary refusals among prefill’s historical uses. So it’s the vendor itself putting in writing that the parameter was also used to get past refusals.

What matters isn’t the decision, it’s where it was taken. Nobody made the model harder to talk round: they took the parameter away. The lever was never in the system prompt, it was in the shape of the API, and it was a product choice that could be revoked.

Hence the rule I keep. If your product exposes prefill, or accepts very long contexts, you have already handed reliability levers to anyone, no matter which technique gets used against you. And when you want those levers back, the place to act is the API surface, not the prompt.

Where I go from here
#

I started this study by collecting tricks, and along the way I realised there was something far more interesting going on: an attack surface that isn’t a bug to patch but a property of how these systems work. The boundary between instruction and data is drawn by the format and respected through training, not enforced by the machine. I’ve been turning it over for weeks and it still strikes me as the best part of the whole thing.

And it shifts under your hands while you study it. Context windows keep growing and the many-shot lever grows with them. New models ship and you have no idea how much of what you read six months ago still holds. APIs expose wonderfully convenient parameters that double as attack levers. I don’t know many fields that move like this, and for someone learning it’s lucky: nobody has finished figuring it out, so there’s room to go and measure something.

If you have ideas to share or want to comment on any of this, write to me, I’d be glad to talk it through.

Sources
#

In order of how useful they were to me.

Originally written in Italian, translated with AI assistance and reviewed by the author.

AI Security - This article is part of a series.
Part 1: This Article