Social Memory in Multi-Agent Systems

For the past two years I have been building conversational memory systems — the kind of memory that supports a one-to-one dialogue between a human and an AI agent. The goal is straightforward on the surface: accumulate facts about the user, track personal events, maintain context over long conversations, and make the agent progressively more useful. In practice, building a personal memory without a concrete enterprise domain turned out to be extraordinarily difficult. The ontology is wide open, the relevance signals are noisy, and temporal reasoning adds a dimension of complexity that most graph-based approaches handle poorly.

But the landscape of AI is shifting. We are moving — some would say horribly fast — from single-agent, one-on-one conversations to multi-agent systems where agents talk not only to the humans who own them but to other agents. This shift demands a fundamentally different kind of memory. Task-oriented enterprise agents need context graphs enriched with decision traces, promise graphs that capture intent and commitment, and audit trails that allow formal evaluation of outcomes. I have written about these architectural layers elsewhere.

What I want to explore here is a further layer — a derivative built on top of promise graphs and the assessment of kept and broken promises. I call it social memory: the memory an agent maintains about its interactions with other agents, the reliability of those agents, and the evolving web of trust and reputation that enables — or constrains — collaboration in a multi-agent ecosystem.

Author’s note: This article builds on two prior concepts. The context graph captures decision traces — the record of what an agent did and what information it used. The promise graph extends the context graph by recording not just actions but commitments: what an agent promised to do, whether it fulfilled those promises, and the gap between intention and execution. Social memory is the third layer — the relational memory that agents build about each other over time, informed by the promise and context graphs beneath it.

From Context Graphs to Promise Graphs

Context graphs emerged as a response to a real problem: enterprise agents need more than conversation logs. They need structured records of the decisions they made, the data they consumed, and the outcomes that followed. A context graph turns an agent’s operational history into a queryable, auditable structure — useful for compliance, debugging, and rudimentary learning.

Causal Graphs as the Missing Layer: Bridging Context Graphs, Decision Traces, and Semantic…

The convergence of three architectural patterns — causal knowledge graphs (prioritizing cause-effect relationships), context graphs…

But context graphs capture actions, not intentions. They record what happened, not what was supposed to happen. This is where promise graphs enter the picture, drawing on Mark Burgess’s Promise Theory from distributed systems. A promise graph tracks the full lifecycle from signal to commitment to execution: an agent receives a request, evaluates feasibility and priority, makes a promise (or declines), attempts to fulfill it, and either succeeds or fails. The gap between promise and action is where the richest learning happens.

Promise graphs reveal things that action logs cannot: intention-execution gaps that expose capability limits, dependency structures that show how failures propagate, conditional reasoning that captures contingent plans, and the temporal arcs from commitment to resolution. In a multi-agent system, agents coordinate through promises. The promise graph of the system reveals its coordination topology — who depends on whom, where coupling is tight, and where autonomy exists.

Promise Graphs for Agent AI: A New Architecture for Multi-Agent Coordination

When we design AI agent systems, we often jump straight to actions and decisions. But there’s a fundamental layer we’re missing — one that…

Social memory is what sits on top of this foundation. If the promise graph tells you what happened between agents, social memory tells you what to expect next time.

What Is Social Memory?

Social memory is the accumulated record of an agent’s interactions with other agents (and humans), distilled into assessments of reliability, capability, and trustworthiness. It is not a single data structure but a collection of interrelated concerns that together determine how an agent navigates its social environment.

Trust and Permission Boundaries

The first and most critical dimension is understanding what can be shared with whom. Some memories, tools, and data belong to an agent’s close circle — the organization, the team, the immediate collaborators. It is safe to share certain information within these boundaries. But the boundaries must be explicit and enforced, because in a multi-agent enterprise system the risk is real: your agents must not whisper your secrets to your competitors’ agents. Social memory must encode not just relationships but the permissions and information-sharing policies attached to each relationship.

Capability Discovery and Assessment

Beyond trust, social memory must record what other agents can do. This goes further than a static capability registry. It captures observed performance: Agent X claims to be good at summarizing legal documents, but in three past interactions its summaries missed critical clauses. Agent Y handled a complex data pipeline flawlessly under time pressure. These observations — the track record of promises kept and broken — feed directly from the promise graph into the social memory layer, creating a living assessment of each agent’s real capabilities versus its advertised ones.

Dynamic Hierarchy Formation

One of the most interesting questions in multi-agent coordination is how agents form temporary hierarchies to achieve goals. Can we use this agent? Can we delegate a task to it? Can we assemble a team of agents with complementary skills for a complex objective, assign roles, and dissolve the structure when the job is done? Social memory supports this by maintaining the information needed to make delegation and collaboration decisions: who is available, who is capable, who has been reliable, and who works well together. This is the holacratic dimension of agent systems — fluid, purpose-driven structures that emerge from the social knowledge each agent carries.

Outcome Verification

Perhaps the most important function of social memory is answering the question: did the other agent actually deliver what we expected? Verifying outcomes is non-trivial. It requires comparing the original promise (from the promise graph) against the delivered result, applying domain-specific quality criteria, and recording the verdict. Over time, these assessments accumulate into a reputation score — not a simple number, but a contextual, multi-dimensional profile that reflects performance across different task types, time pressures, and collaboration patterns.

The Architecture: Three Layers of Agent Memory

Putting it all together, we can see three distinct memory layers that build on each other in a multi-agent system.

The context graph forms the foundation. It records decisions, data consumed, and actions taken. It is the audit log — retrospective, factual, and primarily useful for compliance and debugging.

The promise graph adds intention and commitment. It tracks signals received, promises made, feasibility assessments, dependency chains, and the gap between what was promised and what was delivered. It is prospective as well as retrospective — an agent can reason about its outstanding commitments and their dependencies.

Social memory builds on both layers to maintain relational knowledge. It synthesizes the history of interactions into trust assessments, capability profiles, permission boundaries, and collaboration patterns. It is the layer that enables an agent to make intelligent decisions about who to work with, what to share, and how much to rely on another agent’s promises.

Each layer feeds the one above it. Context graphs provide the raw facts. Promise graphs add structure and meaning. Social memory distills it all into actionable relational intelligence.

The Human Dimension

There is a further dimension that makes social memory genuinely complex: agents interact not only with other agents but with humans. And the interface with humans is fundamentally different from the interface with agents. Agents communicate through structured protocols, APIs, and formal message schemas. Humans communicate through natural language, emotion, implication, and context that is often unstated.

Should an agent differentiate between its social memory of other agents and its social memory of humans? Should it maintain something akin to a working relationship with a human collaborator — learning their communication style, their risk tolerance, their decision-making patterns? And does this relationship genuinely matter for the quality of collaboration, or is it an unnecessary anthropomorphization?

I believe it matters, though not because agents need to simulate human emotions. It matters because the interface is different, and effective collaboration requires adapting to the interface. An agent that remembers a human prefers concise summaries over detailed reports is more effective than one that doesn’t. An agent that knows this particular human tends to change requirements mid-project can plan accordingly. This is not emotional intelligence — it is operational intelligence about the characteristics of a collaboration partner, and it belongs squarely within social memory.

Why Social Memory Matters for Enterprise Multi-Agent Systems

The stakes are high. Enterprise multi-agent systems handle sensitive data, make consequential decisions, and operate in environments where trust failures have real costs. Without social memory, every interaction between agents is a cold start. There is no institutional knowledge about which agents are reliable, which partnerships have worked well, or which information-sharing boundaries have been established through experience.

Social memory transforms a collection of independent agents into something closer to an organization — a system with accumulated relational knowledge, earned trust, and learned collaboration patterns. It is the difference between a group of strangers thrown together for a project and a team that has worked together, knows each other’s strengths and weaknesses, and can coordinate efficiently based on shared history.

This is not about humanizing agents. It is about recognizing that any system of autonomous entities that must collaborate needs a mechanism for building, maintaining, and leveraging relational knowledge over time. Humans evolved social memory because it was essential for survival and cooperation. Multi-agent systems need it for the same fundamental reason: coordination without trust and accumulated social knowledge is brittle, inefficient, and dangerous.

Conclusion

We have moved from one-on-one conversational memory, through task-oriented context and promise graphs, to the social memory layer that enables genuine multi-agent collaboration. Each layer addresses a different question: context graphs answer what happened, promise graphs answer what was intended, and social memory answers who can I trust and why.

The journey is far from complete. Social memory opens questions about reputation algorithms, privacy-preserving trust propagation, decentralized identity for agents, and the formal verification of social claims. These are problems I am actively exploring in my ongoing work on sovereign AI agents and the next volumes of the Age AI book series. But the core insight is simple: agents that remember their relationships, not just their tasks, will be the ones that work effectively in the complex, multi-agent future we are building.