How it’s put together

Document Substrate is a small number of pieces with very strict manners. Four pictures: the pieces, the three shelves of memory, a question’s journey, and the house rules.

1

Five pieces, each with one job

The app talks to four helpers: a filing room that remembers everything, a patient worker for slow jobs, a page reader for PDFs and scans, and the AI models — which are only ever reached through one doorway.

The app in the middle connects to four helpers: the filing room (Supabase), the patient worker (Inngest), the page reader (LlamaCloud), and the AI models behind one gateway doorway.
Under the hood

Next.js App Router; Supabase for auth, Postgres + pgvector, and the private source-documents bucket; Inngest for durable background functions (every pipeline step is a retryable step.run); LlamaParse / LlamaExtract on LlamaCloud; models via the Vercel AI Gateway through slots in lib/ai/models.ts (fast = Haiku, smart = Sonnet, embedding = text-embedding-3-small) — never a provider SDK directly. No Python services, no extra vendors.

2

Three shelves of memory

This is the architecture’s one big idea. Originals stay locked. Everything the system searches and reasons over lives on a working-copy shelf written entirely in code names. What the code names really mean sits on a third shelf, encrypted, opened only at the very last step.

Three stacked shelves: the originals (personal details, locked), the working copy (code names only — all any reasoning AI reads), and the vault (encrypted real values, opened only at the last step).
Under the hood

Layer 1: Storage bucket (raw PII). Layer 2: document_pages, document_chunks (page-aware, 800 / overlap 100, vector(1536), HNSW), document_summaries, document_extracts — placeholders only, and embeddings index only this layer. Layer 3: document_entities, AES-256-GCM per value, one placeholder per document ([A1B2C3D4_PERSON_1] — the prefix stops two documents minting the same token). audit_log records every reveal.

3

A question’s journey

Six steps, and the AI only exists in the middle four. Your real words go in at step 1 and come back at step 6 — everything between happens in code names.

Six stations: you ask, names become codes, the right pieces are found, the AI writes in code names, the answer is checked, and names are swapped back — scoped to your documents and logged.
Under the hood

redactQuery() resolves names to tokens via a blind index (no decryption at query time); retrieval runs over the redacted chunks; the output guard runs before hydration; hydrate() is scoped to the documents the answer actually used, throws on an empty scope, and writes an audit_log row when it reveals. Citations deep-link by page number — there are no bounding boxes.

4

Three house rules

They come up everywhere, so they get their own picture: the AI suggests and a person confirms; side jobs run on their own lane so they can’t block the road; and every answer and every copied value points back to a page.

Three cards: it suggests and you decide, side jobs can't block the road, and everything points to a page.
Under the hood

Classify writes documents.classification, never documents.type — bespoke extraction only runs against a human-confirmed type. Extraction lives on its own extract_status lane and never touches documents.status. Grounding re-locates every scalar leaf into document_extract_fields with page + character span, or marks it unlocated so an invented figure is visible.