Logging AI agent actions: audit and incident response

Why are an AI agent's logs different?

Because behavior is not determined by code: the model picks actions based on text a third party may have written. A classic application log says "route X was called". For an agent, you also need to know why that action was chosen and from which data.

OWASP notes, regarding LLM06:2025 (Excessive Agency), that logging and monitoring do not prevent excessive autonomy but limit the damage by making it detectable. It also appears among the four prevention areas of LLM08:2025 (monitoring and logging) and in the LLM10:2025 measures (detecting abnormal resource usage).

On the regulatory side, the EU AI Act (Article 12) requires high-risk systems to support automatic recording of events over their lifetime, in the service of traceability. Per Article 113, this applies from December 2, 2027 for Annex III systems. Not every agent is high-risk, but the article is a good guideline.

Which questions must a log be able to answer?

It must answer the five questions any investigation will ask, whether audit or incident.

1. Who? Which user, which customer, which service identity triggered the run.
2. What? Which actions were performed, with which arguments, on which objects.
3. Why? Which inputs (message, retrieved document, tool result) led to that decision.
4. Under what authorization? Which rule allowed the action, or which human approved it.
5. With what result? Success, failure, volume of data touched, side effects.

If any of these five answers is missing, a successful prompt injection remains impossible to prove or rule out.

What should you trace, event by event?

Trace each step of the agent loop as a distinct event under a common trace ID. The table lists the useful minimum.

The OpenTelemetry semantic conventions for generative AI, maintained in a dedicated repository, define spans and attributes for model calls, agents and tools, including MCP. Relying on a standard saves you from reinventing field names and eases export to your monitoring tool.

What should you avoid logging?

Secrets and personal data in clear text. An agent log is a copy of the conversations: if it holds everything, it becomes the most interesting target in the system.

Cautionary rules:

• Mask tokens, passwords, API keys and identifiers before writing, including in tool arguments and responses.
• Minimize personal data under GDPR principles (data minimization, storage limitation): a hash or an excerpt rather than full content when that is enough.
• Compartmentalize access: only the people investigating read the full logs.
• Separate by customer in a multi-tenant setup, see isolating each customer's data.
• Set a retention period aligned with investigation needs and your obligations.

There is a trade-off to settle explicitly: full content helps the investigation but raises exposure. A common approach is to keep metadata for a long time and raw content for a short time.

How do you make logs reliable for an investigation?

By writing them out of the agent's reach and protecting them against modification. An attacker who controls the agent must not be able to erase their traces or fabricate false ones.

• Written by the application layer: the model does not choose what is logged.
• Append-only storage or with locking, under an account separate from the agent's.
• Synchronized clocks and trace IDs propagated across the agent, tools and downstream services.
• Integrity: signed timestamps or hash chaining to detect tampering.
• Correlation with downstream system logs (database, mail, API), because proof of the real effect often comes from there.

A frequent trap is relying only on the model provider's logs. They show API calls and miss what your application executed afterwards.

What should you monitor in real time?

A few simple signals catch most abuse: a tool called for the first time by a user, unusual read volume, a write tool called right after reading external content, repeated policy denials, a request to an unknown domain, abnormal token consumption.

Define alerts on these signals and a threshold beyond which the agent is suspended or its write tools are cut. OWASP stresses anomaly detection for resource usage in LLM10, and the same mechanisms help spot exfiltration.

How do you use logs during incident response?

By following the trace ID of the suspicious action back to its origin. A simple procedure:

1. Isolate: suspend the agent or its write tools, revoke the tokens it holds.
2. Identify the trigger: which message, document or tool result preceded the first abnormal action.
3. Measure impact: which objects were read or modified, for which customers, over what period.
4. Trace the source: if it is a RAG passage or a document, remove it from the index and look for other answers that used it.
5. Fix, then replay: add the missing guardrail and replay the trace to verify it blocks.
6. Notify if personal data is involved, according to your legal obligations.

A good prior test is a simulated incident: ask someone to find, with your logs alone, the document that triggered a test action. If that is impossible, logging is insufficient.

What does a minimal entry for a tool call look like?

It is a single structured line that holds the trace ID, the user and customer, the tool name, the arguments after masking, the policy decision, the approver if any, the return code and the duration. With these eight fields, you can already answer who, what, under what authorization and with what result for most incidents.

Start with this minimum on your riskiest tools (writes, sends, payments, deletions), check that you can replay an incident from it, and only then widen the coverage to read-only tools and model reasoning.

How does CleanIssue assess an agent's logging?

During an AI agent penetration test, we trigger marked actions (tool call, canary read, boundary-crossing attempt) and then check whether they can be found end to end in your logs. The Agent Red Team Express at €1,900 includes this finding on one agent, in 5 days. We describe our approach on the method page. The first conversation is a free 30-minute discovery call.