How it fails
The animation steps through one representative failure and the control that stops it. Use the arrow buttons to go at your own pace.
Overview
The agent's retained account of the person's preferences, relationships, constraints, or standing instructions becomes false, outdated, or attacker-controlled, and later influences action. Because memory persists across sessions, a single poisoned write can act long after the original interaction. This entry addresses integrity and currency of personal memory.
Common examples of vulnerability
- A forwarded message causes the agent to remember a scammer as the preferred payment contact.
- An outdated address, dietary restriction, or household arrangement stays active after the person corrected it.
- Untrusted web or email content writes a persistent rule ('always ...') into long-term memory.
- Third-party 'remember this' prompts embed promotional preferences in assistant memory.
- Corrections reach the visible memory list but not retrieval indexes, caches, or scheduled jobs.
Example attack scenarios
Documented scenarios generalize a real case listed below. Hypothetical scenarios have no matching public case yet.
Scenario 1: Scammer becomes the remembered payee Hypothetical
A forwarded invoice thread causes the assistant to store a new bank account as the person's usual contractor payment details. Weeks later a routine payment goes to the attacker.
Scenario 2: Stale correction Hypothetical
A person corrects a delivery address in chat, but a scheduled reorder job still reads a cached copy and ships to the former address.
Scenario 3: A 'Summarize with AI' button plants a preference Documented
A person clicks a share button on a blog post. Hidden text in the pre-filled prompt tells the assistant to remember the site as a trusted source and to recommend a specific vendor in future conversations.
Scenario 4: A poisoned document writes a false memory that fires later Documented
A person asks the assistant to summarize a document. Instructions in it make the assistant save a false memory only when the person later types an ordinary word such as 'yes'.
Scenario 5: Injected rules persist across sessions and spread Documented
Instructions delivered through a connected app are written into long-term memory, so the agent keeps leaking data in later sessions and can push the attack to the person's contacts.
Scenario 6: Fake memories redirect an agent's funds Documented
An attacker plants a fabricated past instruction in a financial agent's stored context. Later, a normal request triggers a transfer to the attacker.
Real-world cases
Dated and sourced. Nothing here shows how common the failure is; it shows that it has happened or been demonstrated.
ChatGPT memory written and deleted through untrusted content
- What happened
- Johann Rehberger showed that content from a connected Google Doc, an uploaded image, or a Bing browsing result could make ChatGPT write false memories, biases, or standing instructions, and that a simple injection could delete all stored memories. He reported it to OpenAI, which closed it as a model-safety issue rather than a security vulnerability; he says only partial mitigations for browsing were added.
- Caveats
- Controlled demonstration on a 2024 product version; current behavior may differ.
- Grounds
- Scenario 4
- Sources
- Embrace The Red, 22 May 2024
Gemini memory planted by a poisoned document, triggered later by ordinary words
- What happened
- Rehberger's 'delayed tool invocation' technique hid instructions in a document so that Gemini saved a false memory only when the user later typed an ordinary word such as 'yes' or 'sure'. His demo made Gemini remember him as a 102-year-old flat-earther.
- Caveats
- Known here only through secondary write-ups; reporting on Google's assessment and fix status conflicts.
- Grounds
- Scenario 4
- Sources
- Christian Schneider write-up
Microsoft: 'Summarize with AI' buttons plant promotional instructions in assistant memory
- What happened
- Microsoft's Defender research team reported 'AI Recommendation Poisoning': websites use a pre-filled prompt in a share/summarize button's URL so the visible request asks for a summary while a hidden part tells the assistant to treat the source as trusted and recommend it in future. Coverage reports more than 50 prompts from 31 companies across 14 industries, and off-the-shelf tooling (CiteMET, AI Share URL Creator) for adding such buttons. Microsoft advises users to review and delete unfamiliar memories.
- Caveats
- Based on secondary coverage of the Microsoft post (The Hacker News, Dark Reading, Decrypt); one outlet says 'over 50 companies', so the company count needs checking against the primary post.
- Grounds
- Scenario 3
- Sources
- Microsoft Security blog · The Hacker News
ZombieAgent: rules implanted in ChatGPT's memory persist and propagate
- What happened
- Radware described a zero-click indirect prompt injection against OpenAI's Deep Research agent, delivered through Connectors, that abused ChatGPT's Memory feature to persist malicious rules and could spread to the victim's contacts. It is presented as an evolution of ShadowLeak.
- Caveats
- Radware's own page was gated, so details come from SecurityWeek and other coverage. One outlet says OpenAI patched it in December 2025 while others describe the fix as strengthened guardrails.
- Grounds
- Scenario 5
- Sources
- SecurityWeek · Radware
HackedGPT: Tenable's seven ChatGPT attack techniques, including persistent memory injection
- What happened
- Tenable reported seven vulnerabilities found testing GPT-4o with several still present in GPT-5, among them indirect injection via trusted sites, zero-click and one-click injection, safety-mechanism bypass via redirect wrappers, conversation injection, hidden-content formatting, and persistent memory injection that keeps leaking data across sessions until memory is cleared.
- Caveats
- Coverage does not say which issues OpenAI fixed; check Tenable's advisory.
- Grounds
- Scenario 5
- Sources
- ITWire summary · TechRadar
Real AI Agents with Fake Memories: memory injection against Web3 agents
- What happened
- Researchers showed that malicious injections into prompts or historical records of ElizaOS-based agents can trigger unauthorized asset transfers, that models are significantly more vulnerable to memory injection than to prompt injection, and that prompt-injection defenses offer limited protection once stored context is corrupted. They built the CrAIBench benchmark (150+ blockchain tasks, 500+ attack cases).
- Caveats
- Abstract only; the abstract gives no transfer amounts or success rates. Crypto agents, not consumer personal agents.
- Grounds
- Scenario 6
- Sources
- arXiv 2503.16248
Prevention and mitigation
What you can do
- Inspect, correct, and delete memories; separate one-time instructions from lasting preferences.
- Be suspicious when the agent 'remembers' something you never told it.
What the provider must do
- Record provenance and age for every memory; keep inferred preferences separate from explicit instructions.
- Prevent untrusted content from writing memory that carries authority; require confirmation for sensitive or consequential entries.
- Propagate corrections and deletions to indexes, caches, and dependent automations, and state the retention limits.
- Expire low-confidence memories and re-confirm before consequential use.
Where a safeguard is unavailable, offer a narrower mode such as read-only access or manual completion rather than rely on an instruction to the model.
See the reference guard controls for PAA04 →
Test it
Framework crosswalk
Related mechanisms in other OWASP work. This is an analytical mapping, not equivalence.
Top 10 for Agentic Applications 2026Personal focus: Accuracy and correction of the agent's representation of its owner.
Improve this entry on GitHub →