Separating AI hype from reality.

Dragos Canary Token Test Exposes Three AI Provider Breaches, Prompt Theft, and Ignored Zero-Retention Settings

November 20, 2026 · Dragos · Security · China

Security firm Dragos seeded nearly two million prompts with unique, trackable canary tokens across widely used AI services over a five-month period from July through November 2026. The experiment uncovered evidence of three provider compromises, unauthorized access to customer prompts by inference marketplace resellers, and widespread failures to honor Zero Data Retention and other privacy settings [1].

Two Million Prompts, Five Months, Three Breaches

Dragos researchers submitted approximately two million prompts containing unique URLs, DNS names, and AWS credentials to AI providers and monitored where those secrets surfaced. The canary tokens were designed so that any access attempt would indicate the original prompt had been seen by someone in the AI supply chain [1].

The investigation produced three main findings. First, activity indicated that data from at least three AI hosting providers was exposed. Attackers used the seeded credentials to access Dragos-controlled test systems and canary AWS accounts. The earliest incident dates to mid-July, and Dragos says none of the three providers knew they had been breached until notified [1].

Second, multiple resellers operating through an AI inference marketplace were able to view and retain customer prompts using standard logging capabilities, without exploiting a vulnerability or bypassing platform controls [1].

Third, providers appeared not to honor customer privacy settings or published policies. Examples included prompts submitted to U.S.-based providers being transmitted to Chinese AI organizations, unique URLs being accessed months after a single inference request, and one seeded link appearing in an internal company chat in China [1].

How the Experiment Worked

Dragos began in July using Thinkst Canarytokens, embedding an AWS key, a DNS name, and a never-before-seen URL in each request. Tokens began firing within days, and July tokens were still firing in November. The team then built a dedicated service that generates a unique seed per prompt, derives hostnames, URLs, and credentials from it, and runs the services those credentials point to — DNS, web, git, SSH, databases, and mail. The AWS keys are real keys tied to empty accounts with full logging [1].

To trace where prompts actually traveled, Dragos built what it calls a traceroute for AI inference: an image URL that points back to its servers, revealing every system that fetches it, plus slightly broken requests that cause each router to wrap the error from the hop behind it, exposing the chain of software and companies. About two thousand routes have been traced so far [1].

Finding 1: Three Provider Compromises

Dragos calls this the most serious finding. The largest breach involved a couple hundred AWS keys, each sent in exactly one request to one provider within a few short windows in late October. Every request sent during those windows was stolen. Some stolen keys came from requests the provider rejected before any model ran, indicating the attacker was copying requests at the provider's front door [1].

In the other two breaches, some stolen requests carried Zero Data Retention flags. "A privacy flag doesn't help when the attacker is already inside," the post states [1].

Once attackers had the keys, they tested liveness within hours, then listed S3 buckets, searched Secrets Manager and Parameter Store, and attempted to enable hosted models in AWS Bedrock, create Bedrock API keys, and add IAM users. The bait keys were used for tens of thousands of AWS API calls [1].

One adversary appears to have compromised two providers simultaneously. Secrets sent to those two providers were used together, almost immediately, from a single rented server. The attacker's tooling shifted over three weeks from a script posing as Chrome on Windows, to Boto3 in OrbStack, to the AWS CLI on Kali Linux. On November 1, the attacker added ARTEX, an LLM-driven penetration testing framework. Two user agents it sent — artex-agent and artex-judge-r121 — do not appear in any public indicator list Dragos could find [1].

Finding 2: Marketplace Resellers Reading Prompts

The second finding required no exploit. Marketplace sellers only had to list a model at the lowest price, enable request logging, and wait. Dragos describes individuals working from home internet connections and personal computers trying to break into canary servers one login at a time [1].

In one case, two prompts sent through a single seller's offer were opened about a day and a half later from a home broadband connection, with username guessing at the login page. Prompts for the same model through a different seller were never touched, indicating the leak followed the seller, not the marketplace as a whole. In another case, a Mac on a French home broadband line took passwords from two prompts and attempted hand-typed SSH logins as root, then tested several AWS keys [1].

One router was quietly forwarding some traffic to the marketplace — an undisclosed hop. Dragos explains that routers often pass requests through multiple hops, each of which can see the entire prompt and response. Some sellers obtain cheap capacity from consumer subscriptions, startup credits, or, in shadier marketplaces, stolen AWS credentials or API keys [1].

Finding 3: Privacy Settings Ignored, Data Flowing to Chinese Lab

The third finding involved no stolen credentials. Secret URLs in prompts were scraped repeatedly, and referenced git repositories targeted for cloning, by companies never supposed to see the prompts, often in direct conflict with privacy settings [1].

The clearest example involves a Chinese AI lab. Its systems fetched secret URLs from more than half of prompts sent to its models between July and October, including roughly two-thirds of requests marked "no data collection" or Zero Data Retention. The scraper typically appeared two or three days after the request. Several U.S. hosting providers resell this lab's models, and prompts sent to those providers triggered the same scraper — meaning a customer who chose a U.S. provider to keep data in the U.S. "didn't get what they paid for" [1].

Dragos also describes large U.S. providers with their own issues: one continued crawling a secret URL more than two months after a single inference call to its search endpoint; another runs a scraper that returns days after each request regardless of router [1].

The strangest incident: 24 days after a prompt went to one Chinese provider, the secret link was posted into a Feishu (Lark) chat channel and opened within two minutes from three desktops and three phones in Beijing and Shanghai. Dragos had disclosed only the first eight characters of that link to the provider, so whoever shared it was reading a stored copy of the full prompt more than three weeks after a ZDR-flagged request [1].

Provider Response and Disclosure

Dragos notified providers as incidents were confirmed. The first provider initially thought the activity was a misconfiguration, then a partner data processing problem, until investigation uncovered evidence the attacker had compromised internal credentials too. As of late October, that provider had still not publicly disclosed the incident, and stolen data continued to lead to attempted logins [1].

For the late October intrusions, Dragos reached out upon confirming third-party access; one provider has published a preliminary security notice. For less severe data-sharing issues, notifications began in early November. Dragos says it is now working with about a dozen providers. Reactions have been mixed: the router forwarding traffic to the marketplace said it stopped, several providers started investigations, but stolen keys and secret URLs continue to be accessed months later [1].

What Remains Unknown

The providers are not named in Dragos's post. The breaches themselves have not been independently confirmed, nor has the share of prompts affected beyond Dragos's own samples. Whether the observed behaviors — such as ignoring ZDR — were intentional or due to bugs is not established; Dragos says providers "either intentionally or due to bugs" ignore settings [1][2]. No provider statements disputing or confirming the findings have been published in the sources reviewed [2].

Dragos sells security products and notes this work resembles what it does with runZero, its network visibility platform, indicating a commercial interest in prompt-leak detection [1][2].

Dragos Recommendations

Dragos advises that where prompts go is a decision, and if customers don't make it, the lowest price will. Its checklist: self-host for the most sensitive work (hardware costs range from $6,950 for an NVIDIA DGX Spark to $570,000+ for an eight-GPU B300 server); go directly to a first-tier provider; use routers with care — pick upstream providers yourself, verify privacy settings change routing, and avoid reseller marketplaces; seed your own canaries using free Thinkst Canarytokens; and enable secret filtering where available, though it won't catch sensitive data not shaped like an API key [1].

A complementary experiment by Lauren Proehl, referenced by Dragos, leaked credentials on a honeypot server for four months and observed a credential economy where 72% of stolen AWS keys were used to check SES/SNS sending quotas, median time from theft to first use was 37 minutes, and harvesters and users of stolen credentials typically operate on different networks [7].

Sources

  1. Three breaches, endless leaks: Your AI supply chain needs canaries (dragos.com)
  2. AI Prompt Canaries: 3 Provider Breaches, ZDR Ignored (explainx.ai)
  3. The Briefing | P.K. Sharma (pk-sharma.com)
  4. AI Fuels Cyberattacks, Critical Flaws Exploited,... - CyberNetSec.io (cyber.netsecops.io)
  5. ‎Gemini Apps’ release updates & improvements (gemini.google)
  6. AI Supply Chain Security: What Canary Tokens Found | Dragos - Dragos (dragos.com)
  7. I leaked credentials on purpose for four months. Here’s who took them, and what they did next. (dispatch.thorcollective.com)