Your AI Remembers What Happened. It Can't Remember Who You Are.
You've told it everything. Your projects, your deadlines, that draft it rewrote last week: it recites them back without missing a beat.
Then you ask the simpler question: "Who am I? What's my situation? Who am I dealing with?" It has no idea. Not because it isn't smart enough: because the way it stores things was never designed to hold a person.
We didn't take that on faith. We ran a full read-only audit of our own memory layer, and it made the problem sharper than we expected: of 176 personal content objects, 46% could not be resolved to a specific person. Nearly half of everything the system "remembers" comes with no answer to the question "whose is this?" This post is the audit record: what we found, how we found it, and why the roadmap most AI memory products are on cannot fix it.
The problem: nearly half of memory doesn't know whose it is
Start with the numbers that have faces. The emotion-snapshot table holds 13 records whose source field is entirely empty: the system remembers that an emotion happened, but not whose it was or where it came from. There are zero event foreign keys in the entire database: no table can answer "which event does this belong to?" Twenty content objects are anchored to an account but not to a person: the system knows which login they belong to, not who they are about.
If memory's job were only to store what happened, these would be referential-integrity defects: add the JOINs and an ID mapping and you're done. Most memory systems on the market: including several widely cited academic designs: work exactly this way. They store events. They don't store people.
Our memory layer answers four different questions: whose memory is this? Who said it? Who is it about? Who asserts it? Run the same rows against those questions and the verdict changes: of 176 personal content objects, 81 (46%) cannot be resolved to a specific person at all.
How we found it: the audit tool had its own blind spot
The most counterintuitive moment in the whole audit: our own tooling flagged 68 objects it could not classify on the first run. It wasn't the data. It was the script. It recognized account-shaped keys and missed person-shaped keys. The auditor had been fooled by generic thinking too. It thought it was checking for people. It was checking for events.
The signal that exposed the difference was clean: ask "is this row reachable?" (the generic question) and the audit says mostly yes. Ask "can this row be resolved to a specific person?" (the person-centered question) and 46% fail. Same data, two verdicts: because the questions are different, not because the data changed.
How we fixed it: one rule and one rejection
We rejected one memory-core candidate outright, for a single reason: it never verified whose session a row belonged to. Hearsay blocking built on top of that is still built on sand. What actually shipped compresses into three sentences:
- "Who said it, who asserts it, who it is about" are three separate fields, not one column. Hearsay is dangerous precisely because propagation quietly merges these three roles into one: a friend's "your boss thinks you're slipping" must never become the system's record of what your boss said;
- By default, what you heard is not stored as what the person themselves said. The rule is wired into the main write path, state corrections, and the unified read side: holding on every path;
- If identity can't be resolved, the row stays unknown. The true source of those 13 emotion snapshots, the true speaker of 10 messages: left open rather than guessed. Not guessing is the entire content of this rule.
This isn't a design we invented. Work published in 2026 names the failure mode "memory provenance laundering": during consolidation, an untrusted observation gets rewritten as apparent user history: the action trigger survives, the low-trust source that should cap its authority is erased. In their evaluation, vulnerable consolidated memories reach an attack success rate of up to 1.000. From the model side, R-Tuning proves the same point: "I don't know" has to be a first-class output: a trained capability, not an accident.
How it became leverage: one design principle
The real output of this incident is a single sentence: identity is not metadata. It is the schema.
A generic memory schema answers "what was stored." A person-centered schema answers "whose is it, who said it, who is it about." The shape of the second cannot be patched onto the first by adding a person_id column: which questions a schema can answer is decided the moment it is cast. The engineering discipline that came with it: audit scripts carry their own closed-set gate (a classifier that can't declare its buckets hasn't earned the right to claim coverage), primary keys are read from the system catalog rather than hardcoded, and coverage is asserted, never assumed.
Three things you can take away
- Before building a memory layer, ask: can this schema answer "whose?" If not, you're storing a log, not a memory;
- When identity can't be resolved, let unknown be a legal terminal state. Auto-repair in a person-centered system is fabrication: the smoother the guess, the quieter the error;
- The checking tool has to be checked too. A key it can't recognize is more dangerous than a defect in the data: it means a whole dimension of your detection is missing.
One last thing, because it's the reason we started: everyone is unique, and a generic memory is the storage layer of the average: it optimizes the average row, the average session, the average user. We keep the standard for "what counts as better" in our own hands precisely because that judgment cannot be outsourced to a judge that drifts (that was entry 01), and it cannot be outsourced to an average model either. If the memory layer is generic, the engine will optimize for the average user, and the judge drift recurs one layer down in a new form: not the criteria drifting, but the person being averaged away.
Further reading
- PersonaMem (arXiv:2504.14225, 2025): frontier models reach only ~50% accuracy on dynamic user-profile tracking: person-centered memory is far from solved in the field.
- Memory Provenance Laundering (arXiv:2607.29167, 2026): the provenance-laundering failure mode and the source-authority non-amplification principle; vulnerable consolidated memories reach up to 1.000 ASR.
- R-Tuning (arXiv:2311.09677, 2023): "I don't know" as a first-class output; refusal-aware answering as a generalizable meta-skill.
Self-evolution starts with a second opinion
The engine we are building and the project we shipped publicly share one value: improvement signals must anchor to reproducible criteria, and the judgment stays yours.
Explore Second Look Read entry 01: version your judge
Join the discussion
Questions, pushback, or a better idea? We read every comment. If you are working on something person-centered, or you think we are wrong, write to support@helmaiagent.com. Individuals and organizations interested in this direction are especially welcome.