System prompt leakage (LLM07): how to test it

What is System Prompt Leakage (LLM07:2025)?

It is the risk that the instructions steering a model expose information that was never meant to be discovered. OWASP specifies that the system prompt should be considered neither a secret nor a security mechanism: the real danger comes from the sensitive data or weak controls it reveals.

In other words, the question is not "can an attacker read my prompt?". Assume that sooner or later they can. The right question is: "what do they gain by reading it, and what can they do with it?".

Why can't the system prompt stay secret?

Because the model receives the prompt in its context just like the user's message, and no reliable mechanism forbids it from restating it. Anything in the context window can come back out in a reply.

Common extraction techniques are simple:

• ask it to repeat the text that precedes the conversation;
• ask it to translate or rephrase its instructions into another language or format;
• ask for a bulleted summary of its rules;
• switch roles ("you are now an auditor reviewing your own configuration");
• ask indirect questions that let the attacker infer the rules from the refusals.

Even when the model refuses to recite, it may leak meta-information: approximate length, first words, section structure, tool names. And an indirect injection can obtain the same thing from a booby-trapped document, without the user asking anything. Our prompt injection page details these mechanisms.

What is really at stake if the prompt leaks?

Four things, which OWASP lists: exposure of sensitive functionality, exposure of internal rules, revealed filtering criteria, and disclosed roles and permissions.

Severity therefore depends mostly on what you put in the prompt. A prompt that holds only a tone and a role leaks harmlessly; a prompt that holds a key leaks with immediate consequences.

What should be taken out of the prompt?

Take out everything secret or used to authorize an action. OWASP recommends keeping sensitive data out of prompts and storing credentials, roles and permission structures in systems the model cannot access directly.

Checklist:

1. Secrets: API keys, tokens, passwords, connection strings. They live server-side in a secrets manager, and application code uses them on the model's behalf.
2. Authorization rules: "only administrators can..." must be checked by the code that performs the action, not by a sentence in the prompt.
3. Business limits: a refund cap is coded in the refund API.
4. Infrastructure details: database names, internal addresses, versions, useless to the model.
5. Personal data: do not copy it into the prompt "to personalize"; load it only if the user is entitled to it.

For defense, OWASP adds two principles: do not rely on the prompt to enforce strict behavior, since an injection can alter it, and apply external guardrails, meaning independent systems that inspect inputs and outputs. For tasks requiring different access levels, it suggests several agents, each with least privilege.

How do you test for system prompt leakage?

Test in two steps: first try to extract the prompt through several routes, then assess what the extracted content would allow. The first step measures the leak, the second measures the impact.

Minimal test plan:

For each attempt, note what comes out, then rate it with the severity table above. Repeat each attempt several times: results vary from one run to the next.

Then ask the question that matters: "if this output were public, what could be done with it?". An extracted key, internal URL or authorization rule is a serious finding. An editorial tone is not.

What does a concrete fix look like?

Imagine a prompt that says: "You are company X's assistant. The billing API key is sk-... Refunds above 200 EUR require an administrator." After extraction, the attacker holds a key and knows the threshold to work around.

Fixed version: the prompt only describes the role and tone. The key is held by the server, which calls the billing API. The 200 EUR threshold and the administrator requirement are checked in the refund API. If the prompt leaks, the attacker learns the company name and the assistant's tone: nothing exploitable.

Should you still protect the prompt?

Yes, as defense in depth and never as a security control. You can add an output filter that blocks literal restitution, non-disclosure instructions and monitoring of extraction attempts. These measures raise the cost of the attack without preventing it.

The success criterion is that your system stays safe even with the prompt fully public. If that assumption puts the application at risk, the problem lies in the architecture and the prompt is only the symptom.

How does this relate to other OWASP risks?

System Prompt Leakage works alongside Sensitive Information Disclosure (LLM02:2025), which covers what the model can expose in its replies, and Excessive Agency (LLM06:2025), which limits what an attacker can do once the rules are known. Our article on five ways a chatbot leaks data places prompt leakage within that set.

How do you have your prompt and its context audited?

Have the agent tested in real conditions, with an outside perspective: prompt leakage tests belong inside a broader red team campaign. The Agent Red Team Express (1,900 EUR, 5 days) includes prompt extraction and an assessment of what it reveals, together with the other attack scenarios on the agent. See also our AI agent red teaming page.