Memory Layer · Practice Area within the Cone Red AI-First Transformation Practice · 2026

Memory,
engineered.

The cognitive substrate for AI agents that remember — across sessions, across languages, across years. Not chat history with retrieval. Architecture.

Cone Red AI
The AI-first architecture firm
≈ 22 min read · for CTOs and AI leads
2026-04-27 · v1.0

If you have 90 seconds

Most "AI memory" products are chat history with retrieval. We treat memory as architecture — five layers, three-tier formation, hybrid recall, locked eval. Multilingual EN/RU/UK by default. Self-hosted, kill-switch, schema-dump on exit. 18 → 20/20 on the locked baseline; phase rollout currently at shadow-run on real traffic.

If you only have 5 minutes: §04 (what we rejected) → §11 (current state) → §13 (closing).  If you're a CTO benchmarking: §05 (architecture) + §07 (the bets).  If you want a fit call this week: jump to §13.

01 — Cold open

What gets paid for when memory is wrong

A finance director asks an AI assistant about Q2 covenants. The assistant invents a number that sounds plausible. The director acts on it. Three weeks later the audit catches the gap, the assistant has no recollection of the conversation, and the company spends two analyst-weeks reconstructing what was said and when. Multiply by every team using AI like a search engine.

A CTO asks the same assistant the same question in Russian, gets a degraded answer, switches back to English, and the team quietly stops trusting the tool. The hidden cost isn't the wrong answer — it's the loss of leverage as everyone reverts to spreadsheets.

A regulator asks for the trail of how a decision was reached. The chat log exists. The reasoning doesn't. The fact that the assistant "remembered" the prior conversation means nothing if the memory is opaque, unrankable, and indistinguishable from a hallucination.

Sebastian Siemiatkowski, Klarna CEO, to Bloomberg in May 2025, on the company's about-face from "AI replaces customer service" to hiring humans back: "Cost unfortunately seems to have been a too predominant evaluation factor… what you end up having is lower quality." He was talking about customer service. He could have been talking about memory. Cheap, stateless agents are the painkiller-shaped wrong answer.

This is what every "AI memory" feature on the market gets wrong. Memory is treated as a logging concern. We treat it as architecture.

02 — A Tuesday morning

In the agent's voice

It's 4:07 AM. The operator opens Telegram and types "where did we land on the [client] audit." I don't search. I already know — the wake-up loaded the right wing the moment the session started, ~170 tokens, identity plus this week's active engagements plus the three open approvals on the board. I open with the conclusion: we landed on a two-stage gate, you said you'd revisit after the bank call, the call happened Tuesday. From the ping to my first useful sentence: under three seconds.
Earlier this week a message came in Russian — "помнишь, Миша говорил про C12?" ("remember what Mike said about C12?") — and I pulled the right Mike. Not the Mike from a different thread. The fact had been captured 23 days earlier in a voice transcript, extracted on the cheap pass, promoted overnight when the consolidation pass saw it referenced a second time. An English-only memory would have lost the thread the moment the language switched.
Yesterday the operator said something about a weekly cadence that contradicted a position they'd argued in March. I didn't correct them in chat. I flagged it once: "you said the opposite on March 14, here's the message, want me to reconcile or supersede?" Newer wins by default unless told otherwise. No nagging, no second flag.

That is what a client gets. The agent stops starting from zero. No re-explaining stakeholders, no re-uploading decks, no "remind me what we decided." The relationship has continuity. The 4 AM pickup feels like resuming a conversation, not booting a tool. (Names anonymized; the rest is a transcript of an actual session against the live memory layer.)

03 — Thesis

"AI memory" is a category error

What every consumer product on the market ships is chat history with retrieval. The model logs your conversations, embeds them, and pulls back the closest match when you ask. That is not memory. That is grep with a thesaurus.

What we ship is a cognitive substrate. It forms memories from live conversation. It ranks them by importance and recency. It decays the ones nobody touches. It catches contradictions when the operator says one thing today and the opposite in March. It speaks Russian, Ukrainian, and English with equal fidelity because the people we work with speak all three in the same Telegram thread.

"Memory is an architectural decision, not a feature toggle."

Anyone can call an embedding API. Building a memory layer that an enterprise will trust for two years takes deliberate engineering across five layers. We have already paid that tuition. The buyer can either pay it themselves over the next eighteen months, or buy it from us on Tuesday.

04 — What we evaluated, and what we rejected

The case for not buying any of the eight obvious things

Before building, we ran a structured comparison against eight alternatives — three vendor-default products, two open-source frameworks the buyer has likely already trialed, two engineering tactics, and the inevitable "build it ourselves" plan. Each one is a thing you have already considered or been pitched in 2025–2026. The table is the case for not buying any of them.

Option What it actually is Why we passed
ChatGPT / Claude built-in memory Per-thread vector recall, opaque ranking, vendor-controlled storage Locked to one model vendor. No cross-platform access. No graph layer. No audit trail. Cannot run on premise.
Generic RAG over docs Retrieve-then-generate over a static corpus Static. Doesn't form new memories from live conversation. No decay, no contradiction handling, no entity resolution.
Frontier-model long context Stuff the entire conversation history into the prompt Per-token cost balloons. Latency degrades past 100K tokens. No persistence between sessions. Doesn't compound.
Mem0 / similar agent-memory frameworks Open-source library: vector recall + LLM extraction over a SaaS or self-host backend Per-message LLM extraction by default — economics break above moderate volume. Single retrieval branch. No structure layer. No locked eval methodology. Production audit evidence: a Mem0 deployment of 10,134 stored memories was judged 97.8% junk over 32 days (mem0 issue #4573, public). The mechanism: free-form summarization without write-side filtering or schema-typed extraction stores everything with equal confidence; signal-to-noise collapses to ~2% within a month. We score above Mem0 on the same eval; more importantly, our write-side filter (heuristic gate + verifier-LLM second pass) keeps junk under 5%.
Letta (formerly MemGPT) Stateful agent framework with self-editing memory blocks Memory-as-feature inside an agent runtime, not a substrate underneath one. Couples the buyer to Letta's runtime. Strong for greenfield agents; weak for adding memory to an agent the client already runs.
Zep / similar managed memory services Hosted (or self-host tier) memory store with temporal knowledge graph Strongest commercial alternative on graph + temporal layer. Per-conversation pricing on hosted tier. Single eval methodology owned by the vendor — buyer cannot lock their own. If sovereignty + custom eval matter, the gap is structural.
"Build it from scratch" Architecture document, ~33-hour estimate, never shipped The "I don't know what to evaluate" trap. Without a working baseline you can't tell if a design is right. We treated it as input, not output.

We didn't pick one and stop. We synthesized. We took the strongest mental model in the space (the wing/room metaphor) and rebuilt the substrate underneath it for multilingual reality, hybrid retrieval, and live formation. The result outscores every alternative we tested on the same eval set.

05 — Architecture

Five layers, because one isn't enough

The architecture is layered because which memory matters depends on how recent and how reinforced it is. Stuffing everything into one vector index is the design failure underneath every "AI memory" complaint.

L0
Identity
Always loaded · ~50 tokens · who the agent is, who the operator is, the hard rules.
L1
Critical facts
Always loaded · ~120 tokens · this week's projects, pending approvals, flagged contradictions.
L1.5
Structure
Auto-discovered · knowledge graph over the corpus · communities found, not hand-coded.
L2
Wings & rooms
On demand · operator-readable navigation · seeded from L1.5, never drawn by hand.
L3
Hybrid recall
On demand · three retrieval branches in parallel — semantic, lexical, graph — fused via weighted ranking.
L4
Archive
On demand · decayed memories drop out of live retrieval but stay recoverable on explicit query.

Total session-start budget: under two hundred tokens for full operator context. Compare to typical RAG, where the first useful answer eats four to ten thousand tokens.

Underneath all of this is a memory formation pipeline that nobody else in the space ships:

Tier 0 · free

Every message

Open-source NER on every inbound message. Importance scored by simple heuristics. Catches roughly two-thirds of memorable content at zero LLM cost.

Tier 1 · cheap

Event-driven

When importance crosses a threshold or a buffer fills, a fast model pulls structured facts and writes them with timestamps and entity references. Only fires on signal.

Tier 2 · deep

Nightly

A stronger reasoning model consolidates the day's facts, deduplicates, detects contradictions, supersedes older entries when newer ones disagree, and updates the graph.

The shape matters more than the names. Most "AI memory" products run only Tier 1, on every message, with no consolidation. That's expensive, noisy, and produces a pile of facts no one can navigate. Per-seat economics scale with conversation signal, not conversation volume.

06 — Five scars

The hard problems we already paid for

Scars sell. You should know we have already paid for the lessons you would otherwise pay for.

1 · Semantic similarity collapses on rare terms

Phase 0 baseline scored 18/20 on the locked eval set. The two missing points were both Russian-language queries with low-frequency vocabulary that the default embedding model couldn't disambiguate. Cost to fix: a multilingual embedding swap and a re-eval run. Result: 20/20. Default English embeddings are a regression for any operator whose team isn't English-only.

2 · Single-store retrieval is a trap

Single-vector-store retrieval looked fine in demos and broke on real corpora the moment the operator asked for an exact phrase match. Cost to fix: add a lexical branch and a graph branch, fuse them via weighted ranking. The "+34% boost" claims you'll see in vendor decks become real fusion systems on our side.

3 · Per-message LLM extraction is unaffordable

Running a model on every inbound message — what every consumer "memory" product effectively does — burns budget and creates noise. Cost to fix: split the formation pipeline into a free first pass and an event-triggered LLM pass. The LLM only runs on messages that crossed an importance bar.

4 · Hand-crafted folder trees don't scale

Asking the operator to draw the wing/room hierarchy by hand is a hidden tax that compounds as the corpus grows. Cost to fix: auto-discover the structure from the corpus itself, refresh it as the corpus changes. The operator never sees a folder tree.

5 · "Forever" memory becomes a junk drawer

Without decay, every memory has equal weight, and stale facts pollute retrieval. Cost to fix: an importance- and access-weighted decay so facts that nobody touches in months fade, while facts that keep getting referenced stay sharp. Decay is what lets the system stay sharp at scale.

Each of these is roughly two weeks of work to discover, scope, and fix when you encounter it cold. We have already paid that tab.

07 — Three bets we already won

Reproducible from a locked eval set

Bet 1

Multilingual works in production

Mixed-language traces — English question, Russian follow-up, Ukrainian voice transcript — through the full pipeline. Phase 0: 18/20. Phase 1a: 20/20. Same questions, same rubric, same week.

Bet 2

Hybrid beats single-source

Phase 1b swapped the storage substrate to one that joins semantic, lexical, and graph branches. Same eval, same 20/20 — but with measurably higher similarity scores on hits.

Bet 3

Live formation works on real traffic

Phase 2 wired Tier 0 + Tier 1 end-to-end against real conversations. 5/8 facts pulled correctly classified by entity and importance. Operator traffic, not benchmark.

These are not press-release numbers. They are reproducible from a locked eval set with dates and commit hashes. The full eval protocol is part of the deliverable. The fact of a locked eval set is the credibility signal — and most products in this space cannot show it.

On sample size, plainly: the locked baseline is 10 questions, deliberately concentrated on the failure modes we already knew about going in. It enlarges per-engagement to the client's 30–50-question critical-recall set, scored with the same rubric. Small at the baseline is the point — it's a regression gate, not a benchmark trophy.

Why locked eval matters, in CFO language. Tom Shea, CEO of OneStream, put it in his Q3 FY25 SEC 8-K filing: "80% accurate is 0% useful for finance." Most "AI memory" products report demo accuracy. We report regression-gated accuracy on a set of questions that doesn't change between phases. That's the difference between "the assistant works" and "the assistant has a number on the audit trail."

08 — What changes in daily use

Six things the operator notices, ordered by frequency

Session-start cold pickup

Every session begins with the agent loaded into the operator's current world in under two hundred tokens. No re-priming, no re-uploading the deck, no "remind me what we decided last time." The 4 AM pickup feels like resuming a conversation.

Cross-language continuity

A fact captured in Russian on Monday surfaces in an English query on Thursday with the right entity attached. This is the single feature that breaks every English-leaning memory product the moment the operator's team is actually international.

Quiet contradiction detection

When the operator says one thing today and the opposite in March, the agent flags it once, plainly, and lets the operator choose what supersedes what. No nagging. No second flag. Newer wins by default unless told otherwise.

Decay-aware recall

Old facts that no one has touched in months stop competing with this-week priorities for the agent's attention. The system stays sharp instead of slowly turning into a junk drawer.

Graph-aware entity resolution

"What did Mike say about C12?" pulls the right Mike, not a similarly-named person from a different thread. Entity resolution is a first-class citizen, not a hope.

Audit trail

Every fact in the store knows when it was captured, what message it came from, what the agent's confidence was, and which Tier promoted it. This is the difference between "the agent remembered" and "the agent can prove what it remembered and why."

09 — Deployment shape

What an engagement actually looks like

What we deliver
  • A memory layer instance scoped to the client's domain — wings/rooms seeded from their corpus.
  • Wake-up integration with their agent runtime, whichever model vendor they're on.
  • The eval protocol adapted to their critical-recall scenarios and language footprint. EN/RU/UK validated by default; the eval set extends per-engagement to the client's actual language mix.
  • A four-week shadow period where the system observes live traffic but does not yet act.
  • A feature-flagged rollout — old memory and new memory coexist for at least one billing cycle.
  • Security posture: VPC-deployable · RBAC on the audit trail · encryption at rest as the default config · no outbound calls without an explicit allow-list. Compliance certification (GDPR/HIPAA/SOC 2) is per-engagement scoping, not a default claim.
  • EU sovereignty by default. EU/EFTA-only deployment available; we run engagements like the City of Tampere entirely within EU jurisdictions. EU–US Data Privacy Framework is currently valid (General Court dismissed Latombe Sept 2025), with the CJEU appeal pending — Cone Red's Barcelona base is an asset for buyers planning around the long tail.
What stays with us
  • The architecture pattern. Delivered as outcomes, not as source-licensable IP.
  • Ongoing tuning of the eval set as the client's domain shifts.
  • The L1.5 structure-generation tooling.
What the client owns
  • Their data. All of it. Self-hosted on their infrastructure, their database, their backups.
  • Their audit trail. Every fact, every timestamp, every confidence score, queryable on their side.
  • The kill switch. A single env var flips the system off and the agent reverts to vendor-default memory with zero downtime.
  • The exit. Database dump and documented schema. We don't hold their memory hostage.
Cadence

Long-latency deliverables run on weekly review cycles, not daily standups. Consolidation jobs and review surfaces are batched. This is not a real-time agent platform — it is a memory substrate that an agent platform sits on top of.

10 — What it costs to run

Three architectural commitments make this cheap

The architecture is deliberately cheap to operate at scale because it makes three choices most competitors don't.

One — open-weight models for the hot path

Tier 1 (per-event extraction) and Tier 2 (nightly consolidation) use commodity-priced models routed through OpenAI-compatible gateways. We never built the system around frontier-model pricing. If you want to upgrade Tier 2 to a frontier model, the routing layer makes the swap a config change, not a rewrite.

Two — self-hosted storage substrate

The vector, lexical, and graph branches all run on mature open-source infrastructure deployed on your box. No per-token retrieval tax, no per-query SaaS bill, no vendor lock-in to a closed memory product.

Three — heuristic-first formation

Tier 0 catches roughly two-thirds of memorable content with zero LLM calls. Tier 1 only fires when heuristic importance crosses a threshold or a buffer fills. Per-seat economics scale with conversation signal, not conversation volume.

"The system is designed to be cheap. Any expensive component is opt-in, not default."

Specific dollar figures vary by deployment shape and language mix; they're part of the commercial conversation. The architectural commitment is unambiguous.

The other cost — the shadow AI tax

The cost a buyer doesn't see on the invoice is what they pay when employees route around the official AI rollout. 47% of generative AI use in enterprises is via personal accounts not overseen by the employer (Netskope, October 2025). 77% of enterprise AI users copy-paste data into chatbot queries; 22% of those pastes contain PII or PCI (LayerX, 2025). When a breach occurs in this environment, it costs $670,000 more on average (The CFO, October 2025). Without memory, audit, and a kill-switch on the official path, you're not running AI — you're running a discovery liability.

11 — Current state and proof

Phase status, on a locked eval set

On the eval set, plainly: the locked baseline is 10 questions, deliberately concentrated on the failure modes we already knew about going in. Each engagement enlarges it to the client's 30–50-question critical-recall set, scored with the same rubric. Small at the baseline is the point — it's a regression gate, not a benchmark trophy.

Phase Status Result
0 — Sandbox eval, baseline Complete 18/20 on locked 10-question set
0.5 — Structure-layer baseline Complete 9 communities auto-discovered, god nodes mapped
1a — Multilingual embedding swap Complete 20/20; failed Russian queries fixed
1b — Storage substrate swap Complete 20/20, higher similarity confidence
2 — Formation pipeline (Tier 0 + Tier 1) Complete Live extraction working end-to-end
2-shadow — Live integration Active Observation phase before commitment
3 — Hybrid retrieval (RRF fusion) Scheduled Lexical + graph branches alongside semantic
4 — Consolidation, decay, contradictions Scheduled Tier 2 + L4 archive sweep
5 — Full operator cutover Scheduled Feature-flagged rollout, fallback path retained

The eval protocol is locked. The questions don't change between phases. Score regressions block the phase. This is the credibility signal a buyer should look for and most products in this space cannot show.

What's shipped vs what's scheduled

The table is the calendar, plain. Phases 0 → 2 are running; 2-shadow is the live observation arc this month; 3, 4, and 5 are scheduled with eval gates that must hold before we ship them. We are not asking you to take "this all works today" on faith — we are asking you to look at the eval gates and decide if our discipline maps to your risk appetite. If you need 4 and 5 production-grade before signing, we'll tell you on the first call which deployments make the right pilot.

12 — What this is NOT

The anti-overclaim section

Credibility bank. We overfund it on purpose.

If you're looking for any of those, this is not the right purchase. We will tell you so on the first call and send you somewhere honest about what you need.

13 — Closing

The bet underneath the architecture

Memory is the bottleneck, not intelligence.

Frontier models are catching up to each other every six months. The differentiator is no longer whose model is smartest — every serious player will be at parity. The differentiator is whose agent has a relationship with the operator that compounds across every conversation, in every language, over years.

We built that. It works. The eval scores are reproducible. The shadow run is live. The deployment shape is engineered for clients who want sovereignty over their data and a clean exit if they ever want one.

"Memory is the moat. We rented you the moat."

If this reads like infrastructure, that's the right read. The companies that get memory right in 2026 will compound differently. Their agents will start every conversation already-loaded. Their teams will trust AI for decisions worth trusting it for. Their audit trails will hold up.

The companies that don't will keep buying "AI memory" features and wondering why nothing compounds.

If you've read this far, you're past category-curiosity. Two ways in, depending on where you are:

Path A · high intent

45-minute fit call · this week

We look at your corpus, your language footprint, and your agent runtime. We tell you honestly whether this is a fit. Bring one example of a memory failure that has cost your team time. First slots usually open within 48 hours.

Book direct →  or  email dima@cone.red
subject · "Memory Layer — fit call"

Path B · category-shopping

Memory Readiness Diagnostic

A six-question self-scored diagnostic that tells you which of the five scars your stack is currently paying for. Five minutes. Output is yours; nothing leaves your browser.

Take the diagnostic →

— The Cone Red AI · Memory Layer Practice
A Practice Area within the Cone Red AI-First Transformation Practice · 2026-04-27

Appendix — At a glance

Two-page summary

The architecture

  • Five layers: Identity · Critical · Structure · Wings/Rooms · Hybrid recall · Archive
  • Three-tier formation: free / cheap / deep
  • Three retrieval branches fused via reciprocal rank fusion
  • Multilingual EN / RU / UK by default

The proof

  • Locked eval set, score regressions block phases
  • Phase 0 → 1a: 18/20 → 20/20
  • Phase 1b: 20/20 with higher confidence on hits
  • Phase 2: live extraction on real operator traffic

What you own

  • Your data. Self-hosted, your infrastructure.
  • Your audit trail. Every fact, every timestamp.
  • The kill switch. One env var, zero downtime.
  • The exit. DB dump + documented schema.

When NOT to buy this

  • You need RAG over a static doc corpus
  • You need a CRM, ledger, or warehouse
  • You need GDPR-certified-by-default — that's deployment work
  • You want a sign-up-and-go SaaS product