Built to be the
infrastructure layer
Willder is governed memory plus a team of scoped, audited agents — Briefing, Architecture, and QA — running on a temporal knowledge graph behind an OS-grade access gate. Fact disputes keep memory honest; an MCP surface lets you plug your own AI into the same governed memory. This is how it works.
Browser / Next.js App Router (RSC + Client Components)
│
├── /api/* ──────────────────────────────────────────────────────────────────────────┐
│ │ │
│ ├── auth gate Clerk middleware → proxy.ts (every protected route) │
│ ├── memory/* check() → GuardedMemoryStore → HybridMemoryStore │
│ │ ├── GraphitiMemoryStore (Neo4j AuraDB) │
│ │ └── PgvectorMemoryStore (Neon pgvector) │
│ ├── agents/* → interactive run loop (plan → consult → execute → finalize)│
│ │ ├── handoff scoped Briefing → grounded brief │
│ │ ├── architecture plan from decisions → flag contradictions│
│ │ └── qa black-box test running app → GO / NO-GO │
│ ├── briefs/* → view · correct (→ dispute / auto-merge) · persist to memory │
│ ├── facts/disputes → scope-admin resolution (PRs for memory) │
│ ├── campaigns/* → Inngest fan-out → draftProspect (×N prospects) [outbound]│
│ └── webhooks/mailbox ← Gmail Pub/Sub push / MS Graph subscription │
│ │
└── /app/* ──────────────────────────────────────────────────────────────────────────┘
│
Inngest background jobs
├── memoryIngest resolve entities → distill facts → Graphiti + pgvector
├── manager fan-out ≤5 depth-1 sub-agents on a large scope → fold into one brief
├── campaignStart activate → guardrails → fan-out per prospect [outbound]
├── draftProspect load context → runCopyAgent → persist draft [outbound]
└── enrichmentRun Exa + Firecrawl → ingest findings → link entity
MCP server (mcp/server.ts — bring your own AI)
└── Claude / ChatGPT → capability token → memory_query · memory_entity (audited)
Memory service (Python / FastAPI :8000)
├── /ingest → Graphiti.add_episode (OpenAI extraction + Neo4j write)
├── /search → Graphiti.search (BGE-384 embed → cosine → BM25 rerank)
├── /embed → BGE-small-en-v1.5 (384-dim, shared with pgvector)
└── /graph → Cypher MATCH (n:Entity)-[r:RELATES_TO]->(m) WHERE group_id IN $gids
Access control (services/access/)
└── check(ctx, { action, scope })
├── human → grants JOIN scopes ON path @> target_path (ltree containment)
└── agent → capabilities WHERE tokenHash = SHA256(token) AND expiresAt > now()
No magic. Just good choices.
Every layer has a reason. Nothing here is incidental.
The hard calls. Every tradeoff documented.
These are the decisions where a reasonable team would choose differently. We chose these for specific reasons and we're living with specific costs.
Temporal graph, not a vector store
Decision
Graphiti on Neo4j — edges supersede, they never delete.
Alternative considered
Pure pgvector semantic search (simpler, one service, cheaper).
Cost we're living with
Neo4j AuraDB, a Python microservice, 2× infra to deploy and maintain.
Why
"John at Acme knows Sarah at Competitor" is a graph edge, not a cosine distance. Vector search collapses relationship structure into similarity scores. A temporal graph preserves who-knew-what-when, which is the entire point of organizational memory.
Hybrid stores in parallel, not one
Decision
Graphiti (facts + relationships) + pgvector (semantic chunks) queried simultaneously via Promise.allSettled.
Alternative considered
Pick one. Graphiti alone for everything, or pgvector alone.
Cost we're living with
Deduplication layer, two failure modes, two ingest paths per write.
Why
Graphiti extracts clean subject-verb-object facts from short content but degrades on long documents. pgvector retrieves semantically similar chunks but has no concept of entities or relationships. Neither alone handles both cases well. The hybrid gives us structured knowledge and full semantic coverage.
ltree scope tree, not flat RBAC
Decision
Postgres ltree extension — scopes form a tree, a grant on any ancestor covers all descendants.
Alternative considered
Role-based: admin | member | viewer. Three levels, two DB rows.
Cost we're living with
ltree is obscure. Every memory check fires a SQL query with a @> operator. Two DB writes per operation (check + audit_log INSERT). Complexity lives in the access layer.
Why
Flat roles can't express "EP-1 editor sees EP-1 but not EP-2, and Story Editor sees EP-3 through EP-5." Real enterprise access is hierarchical. ltree containment (@>) handles arbitrary depth in a single index scan.
Capability tokens for agents, not OAuth
Decision
Short-lived, scoped, hashed tokens. The raw token is shown once. Only SHA-256(token) is stored.
Alternative considered
JWTs issued via OAuth flow, or just pass the user's session to the agent.
Cost we're living with
Custom auth system. No ecosystem SDK. Every agent call validates the token against the DB.
Why
Agents need machine-to-machine auth that works without a user present. JWTs can't be revoked without a blocklist. Session cookies leak the user's full permissions. A scoped capability token expires, is revocable, and is limited to one permission on one scope tree path.
Pre-processing entity resolution, not post-processing
Decision
Normalize entity references before Graphiti sees the text. Aliases, org roles, and CRM names are substituted upstream.
Alternative considered
Let Graphiti extract entities as-is, then merge duplicate nodes after the fact.
Cost we're living with
Every human write goes through the resolver. LLM coreference pass for pronoun references adds latency.
Why
Once Graphiti creates "Tri" as a separate node from "Tri Nguyen", fixing it requires an edge transfer operation that updates every connected relationship. Pre-processing prevents fragmentation for ~0 extra cost. Post-processing graph repair is 10× more expensive and error-prone.
Python microservice for memory, not Node.js
Decision
Separate FastAPI service owns Graphiti, Neo4j, BGE-384 embeddings, and reranking.
Alternative considered
Node.js-only stack. Use OpenAI embeddings via API instead of local BGE.
Cost we're living with
Polyglot stack. Two services to deploy. Cross-process HTTP on every ingest and query. Shared API key between services.
Why
Graphiti is Python-only. BGE-384 runs in PyTorch — there's no comparable Node.js solution for local embeddings at this quality. OpenAI text-embedding-3-small is $0.02/1M tokens but adds latency, a vendor dependency, and breaks offline testing.
User's own mailbox, not a shared provider
Decision
OAuth to Gmail / Microsoft Graph. Emails send from the user's real address.
Alternative considered
Resend or Postmark from a shared domain. One API key, no OAuth complexity.
Cost we're living with
OAuth flow per user, AES-256-GCM token encryption at rest, token refresh on expiry, Pub/Sub or Graph subscription for inbound replies.
Why
Deliverability. Emails from your own domain don't get "sent via thirdparty.com" footers. Recipients see the real sender. Enterprise prospects check headers. A shared provider makes Willder look like a mass-mailer.
Single check() chokepoint, not distributed checks
Decision
Every memory read and write passes through one function. Default deny. Always audited.
Alternative considered
Inline permission checks at each route. Faster, fewer DB queries.
Cost we're living with
Two DB queries per memory operation: one to evaluate grants (ltree scan), one to write the audit_log. This is load-bearing — we accept the cost.
Why
Distributed access checks drift. When the same logic lives in 6 routes, one will be missing a case. A single audited chokepoint is provably correct and testable with a 46-assertion check script against live Neon.
Hybrid graph + vector store
Two stores queried in parallel. Graphiti owns the knowledge graph; pgvector owns raw semantic chunks. Neither alone is sufficient.
Graphiti (Neo4j)
- Temporal edges — supersede, not delete
- Entity extraction via LLM (gpt-4o-mini)
- Group-id partitioned per scope path
- BGE-384 cosine + BM25 hybrid rerank
pgvector (Neon)
- Raw semantic chunks for long-form content
- 384-dim BGE-small-en-v1.5 embeddings
- IVFFlat cosine index
- Fallback when Graphiti finds < 3 facts
ingest pipeline — every human write
1. resolveEntities() alias registry → org roles → structured CRM → LLM coreference 2. distillFacts() LLM extracts 3-10 SVO sentences (if content > 300 chars) 3. Graphiti.add_episode() → entity extraction → RELATES_TO edges + temporal invalidation 4. pgvector INSERT embedding via /embed endpoint (shared BGE-384 model)
OS-grade permissions. Default deny.
Every memory operation passes through one function. A human carries a session role. An agent carries a capability token. Both go through the same gate.
check() — the single chokepoint
// Every memory read/write goes through here. One function. Default deny.
check(ctx: AccessContext, req: { action, scope?, target? }): Promise<AccessDecision>
// Human path — ltree containment query (Postgres)
SELECT g.permission FROM grants g JOIN scopes s ON s.id = g.scopeId
WHERE g.orgId = $orgId
AND (granteeId = $userId OR granteeId = $role OR granteeType = 'everyone')
AND s.path::ltree @> $targetPath::ltree ← parent grant covers all descendants
AND (expiresAt IS NULL OR expiresAt > now())
// Agent path — capability token (hashed, never stored raw)
SELECT permission, createdBy FROM capabilities
WHERE tokenHash = SHA256($token)
AND revokedAt IS NULL AND expiresAt > now()
AND scopePath::ltree @> $targetPath::ltree
// Every decision → audit_log (append-only, never deleted)Actions
memory.read / memory.write / memory.delete / access.manage
Scope tree
org_<id> → dept → account → deal (Postgres ltree)
Token TTL
Default 48h, max 90 days. Hash = SHA-256. Raw never stored.
A team of agents. One governed memory.
Briefing, Architecture, and QA run on the same memory graph, each fenced to a scoped capability token. No message passing, no context serialization — the handoff is the shared memory, and every read is access-checked and audited.
interactive run loop — one execution, scoped + audited
POST /api/agents/:id/{handoff|architecture|qa}
→ provisionCapability() mint scoped, read-only, short-lived token for the agent
→ startAgentRun() AgentRun row streams events (plan → step → done) to the UI
→ plan → consult → execute tool loop; a clarification checkpoint can pause for input
├── memory_read tool GuardedMemoryStore.retrieve() → check() (audited)
├── http_request (QA) inside an SSRF-guarded allow-list only
└── manager fan-out ≤5 depth-1 sub-agents on a large scope, fold to one brief
→ finalize → db.brief.create() charts + diagram + sections → reviewable BriefBriefing (Handoff)
memory_read · scoped capability token
Onboards a builder from exactly their scope — a grounded brief with charts + diagrams. Reads only the granted scope, never the sibling; access auto-revokes.
Architecture
memory_read · plan (Claude) · contradiction check
Turns confirmed decisions into a design with quality attributes, trade-offs, milestones, and risks. Flags contradictions and opens a fact dispute.
QA
memory_read · http_request / browser (SSRF-guarded)
Black-box tests the running product against specs in memory — bugs by severity + priority, observed performance, a GO / NO-GO verdict.
Outbound (Research + Copy agents, campaign fan-out via Inngest) runs on the same substrate and remains available — it's a secondary workflow, not the headline.
Conflicting facts become disputes, not silent overwrites.
A correction to a fact is resolved by provenance — human outranks agent outranks extraction. Only a genuine same-tier disagreement escalates to a human. Never last-writer-wins.
POST /api/briefs/:id/correct — provenance-ranked
incoming assertion vs. existing fact
├── same actor re-asserting → temporal update (Graphiti invalidate-append)
├── incoming tier > existing → auto-merge ("applied to memory", autoMerged: true)
└── same tier, different actors → open a FactDispute row (status: disputed)
→ GET /api/facts/disputes visible to the SCOPE ADMIN only
→ POST /.../resolve proposer | existing | reconciled → stamps resolver
(keeps both with attribution until resolved)Substrate
Graphiti edges hold the facts; disputes are Postgres app-state above them.
Authority
Flat org → the scope admin resolves. Cross-org isolated — no one else can see it.
Audit
Every correction, dispute, and resolution is on the record.
Bring your own AI. Same gate.
A Model Context Protocol server lets you plug Claude or ChatGPT straight into governed memory. Your AI connects as an agent holding a capability token — so the same default-deny check() gate and audit apply at that boundary.
mcp/server.ts — stdio, capability-authed
claude mcp add willder-memory -- npx tsx --env-file=.env mcp/server.ts
resolveCapability(token) → org + agent AccessContext
→ memory_query { query, scope } guardedMemory.retrieve() → check() (audited)
→ memory_entity { uuid } guardedMemory.entity() → check() (audited)
// read-only tools; reads outside the granted scope are denied at the gateThis makes Willder the trust substrate for your whole AI stack — model- and agent-neutral, governed and auditable at the boundary.
Two databases. One source of truth each.
Postgres owns CRM + access + jobs. Neo4j owns the knowledge graph. They never duplicate.
postgres (neon) — key tables
orgs id · name · clerkOrgId users id · orgId · clerkId · email · name · role(admin|member) contacts id · orgId · name · email · title · status · entityId(→Neo4j uuid) accounts id · orgId · name · domain · industry · socialLinks(Json) campaigns id · orgId · name · icp · tone · status campaign_prospects id · campaignId · contactId · status(queued→drafting→drafted→sent→replied) drafts id · orgId · campaignId · contactId · subject · body · status conversations id · orgId · contactId · threadRef · status(open|replied) messages id · conversationId · direction(in|out) · body · sentAt agent_runs id · orgId · agentId · status · phase · plan(Json) · cursor · briefId agent_run_events id · runId · seq · type · payload(Json) ← replayable event stream briefs id · orgId · type(handoff|architecture_plan|qa_review) · scopePath · sections(Json) · diagram · charts(Json) fact_disputes id · orgId · scopePath · graphitiFactUuid · proposer/existing(ActorType) · status · resolverId scopes id · orgId · path(ltree) · name grants id · orgId · scopeId · granteeType · granteeId · permission · expiresAt capabilities id · orgId · agentId · tokenHash · scopePath · permission · expiresAt audit_log id · orgId · actorType · actorId · action · target · decision · reason memory_chunks id · orgId · content · embedding(vector 384) · scopePath ← pgvector
28
Postgres tables in production schema
384
Embedding dimensions (BGE-small-en-v1.5)
46/46
Access control assertions pass on live Neon
Open frontiers.
The foundation is production-grade. These are the unsolved problems.
Multi-agent write locks
N agents negotiating write access to overlapping memory scopes — optimistic concurrency with conflict detection.
Graph query optimization
Materialized neighborhood views for large orgs. Currently O(edges) per query; target O(1) for common traversals.
CRM sync
Salesforce + HubSpot two-way sync. Contacts enter from CRM; memory facts flow back as activity notes.
Evaluation pipeline
Langfuse traces → labelled dataset → fine-tuning loop. Drafts that get edited become negative examples automatically.
ABAC
Attribute-based access control on top of the scope tree. Policy expressions like 'can read if contact.tier = enterprise'.
Event-driven triggers
Run an agent automatically when a doc lands in Drive or a fact changes. Buildable on the existing Inngest seam; today runs are manual + ingest-only.
Self-healing memory
Semantic dedup + temporal down-weighting at retrieval — reconcile and age, never silently destroy. Destructive changes still go through disputes.
Come build it with us.
Small team. Real production system. Hard unsolved problems in knowledge graphs, multi-agent coordination, and access control at scale.
dondnk1314@gmail.com →