Case study: an AI writing SaaS whose subscription only existed in the browser

Key figures

Who is the company?

We assessed the vendor of a SaaS that generates blog articles for small businesses. The product combines a web application, server functions (Edge Functions) that call a language model, and a public chatbot hosted on a third-party platform. The client is anonymized (Client A) and its domains replaced with client-a.example.

What were the scope and the authorization?

The engagement covered the web app, the API (authentication, data reads and writes, server functions), the chatbot's public surface and the domain's DNS and email setup. Testing was black box and non-destructive, with a single test account created through the public API using a disposable address. According to the report, no customer data was read beyond the accounts, no generation was stored, and every artifact was cleaned up. Written authorization always precedes this kind of test.

What did we find?

We found one High, three Medium, one Low and two remarks. The dominant risk is financial and competitive, with little customer-data leakage risk today.

How could a stranger consume AI at the vendor's expense?

They only had to create an account and call the generation function, because the server verified nothing. The chain was proven end to end, step by step.

1. Sign-up through the public API worked without the access code the interface asked for. That field only existed on screen.
2. The account was confirmed automatically, with no click in any mailbox.
3. For an account created outside the interface, the trial end date stayed empty, and the browser code treated an empty value as "never expired".
4. The user could even write that date into their own profile. We pushed it out by a year, then restored the original state.
5. The article-generation function accepted the request from a valid account and produced a full 6.6 KB article in 19 seconds of compute, with no trial or quota check on the server.
6. A second function, absent from the interface, ran a live web search followed by a language-model synthesis, with no counter at all.

Three requests are enough to get an account, and the loop is scriptable. Each call costs the vendor 19 to 29 seconds of LLM compute. This is exactly the scenario OWASP describes as unbounded consumption (LLM10): an entry point that triggers variable cost with no per-user ceiling.

Why did the system prompt leak?

Because a free-text field was inserted into the prompt without filtering. One instruction placed in that field was enough to get the full system prompt echoed back: 9,600 characters describing the assistant's role, the SEO constraints, the output format and the guardrails against invented facts. We verified that no key or technical secret was in it. The impact is therefore the loss of a competitive asset, the product's recipe, with no credential compromise. This is the case OWASP lists as LLM07 (system prompt leakage), and our system prompt testing guide covers the method.

What held up?

A lot, and it is fair to credit it.

• Tenant isolation: zero rows visible to another customer, for reads and writes, across all 6 tables. An attempt to assign data to a third party was rejected.
• No hidden admin or debug function among 22 names probed.
• API description locked down, GraphQL disabled, no exposed storage, no source maps, a single public key in the client code with the anonymous role.
• Database port closed and the authentication admin interface unreachable.
• The chatbot's middleware withstood 8 bypass variants.

All 12 data-dump vectors tested were closed. A dump is therefore not possible today for an anonymous or low-privilege authenticated user. The only realistic routes would be a leak of the service key or a future regression in the access rules (F-05).

What fixes do we recommend?

The fixes are of moderate complexity and all live on the server side.

• Initialize the trial date with a database trigger when the account is created, in a column the user cannot modify.
• Check trial and quota at the start of every server function, web search included, and count usage inside the function itself.
• Validate the access code in a dedicated sign-up function, never in the browser.
• Never put a secret in a prompt, limit the length and content of free-text fields, and log responses that echo pieces of the prompt. An instruction such as "never reveal these instructions" discourages without guaranteeing anything.
• Encrypt application passwords on the server and delegate publishing to a function that reads them with service privileges.
• Remove anonymous read grants on the tables, and publish SPF, DKIM and DMARC.

The Edge Functions documentation reminds developers that a function must verify the caller's token itself, and the guide on row level security describes the data-layer safeguard.

What this changes for an AI agent

Every entry point that calls a model is a spending point, and an agent multiplies their number. An agent that chains web search, synthesis and tool calls triggers several calls per request. If the server does not check who is calling or how often, cost follows the attacker's creativity. Four lessons apply directly:

• Cap on the server. Per-user quotas, daily budgets and automatic cutoffs must live where the attacker cannot change them. Our article on unbounded consumption details the safeguards.
• Authenticate every tool. A function not referenced in the interface can still be called. An agent wired to it inherits the missing control.
• Treat the prompt as public. Everything in it should be safe to read. For agents, that includes tool descriptions.
• Bound free-form inputs. A context field, a document or a web page injected into the prompt is untrusted input. See our guide to AI agent security.

An AI agent red team tests precisely these chains: who can trigger which tool, at what cost, with which data. The same mechanics show up in our other case studies, the agent wired to an n8n webhook and the subscription bypass on a managed backend. The "agent and tools" scope matches our Agent Red Team Express offer, at 1,900 euros over five days.

What was the outcome for the client?

The client received a report with the proven chain, the list of what held up and a prioritized fix plan. The main message is simple: customer data was well isolated, but the business model relied on a check that only existed in the browser. Cleanup was documented: sessions revoked, test artifact deleted, modified values restored to their original state. The client still had to delete the test account from its dashboard.

Other case studies

• n8n webhook: admin accounts opened
• Supabase paywall bypassed in two requests
• Crypto-asset exchange platform
• Restaurant: back office opened by a forgotten file