CortexDB 0.10.4 — questions across the customers under a parent scope (experimental, off by default), an honest note on parent-scope packs, and your request id in every error
0.10.4 is a feature release on top of 0.10.3, the last published release. Everything in
RELEASE_NOTES_0.10.3.md still applies; this page lists only what changed since. It adds no column
family, directory, WAL row kind, index field or on-disk format (one key prefix, erasure_job/, for
the erasure receipts in an existing column family), and nothing is migrated: a 0.10.3 server opens
and serves a data dir 0.10.4 has used (see "Rollback").
Summary
- Questions across the customers under a parent scope (experimental, off by default). Asked at
a parent scope such as
org:acmeabout the customers under it (org:acme/user:*), a question used to be answered from whichever customers' events came first in storage order: in practice the oldest events of one or two customers, whatever the question said. 0.10.4 adds two opt-in lanes beside that path. Each needs an operator flag and an explicit request field or route, so nothing changes until both are in place:- the exact lane (
CORTEX_COHORT=on) answers lookups, counts, top-N lists, funnels, segments, sequences, shares and "latest" questions over the typed facets of every customer the caller may read (event intent, labels, modality). It is computed from the WAL with every forget and erasure applied, throughPOST /v1/cohort/query(no model call) and the/v1/answerfieldcohort(one narration call whose numbers the server checks); - the ranked lane (
CORTEX_SUBTREE_RECALL=request) runs one relevance-ranked recall across the authorized child scopes (/v1/recallfieldsubtree). On/v1/answerit serves content lookups ("which customers asked about refunds in a chat") and syntheses over conversations ("what do customers ask about financing"). By default such a question now READS EVERY CONVERSATION of its cohort (lane: read_all): bounded parallel model calls read each conversation once, the server counts the matching customers and the customers behind each theme, the answer model re-reads each listed theme's example conversations before any is cited, and one narration states them, its numbers checked; coverage says "read N of N conversations".CORTEX_COHORT_SYNTH_MAX_CALLS=0(orcohort.read: "sample") keeps the cheaper ranked sample of excerpts, stated as a sample.
- the exact lane (
- Every customer is named by a stable alias to the model and by its real scope to you. Answers carry the plan, the window, the time anchor, the coverage, per-customer counts and the evidence event ids, each with its own scope.
- Measured on a 30-question cross-customer set (31 items with an anchored variant, frozen
ground truth, a 300-customer test store): through
/v1/answerthe exact lane answered 19 of 19 exact-lane questions exactly, and the 6 attribute questions answered from labels added at ingest 6 of 6; 0.10.3 answered none of the 31. Of the 6 questions about what customers said, now answered by reading every conversation of their cohort, the content lookup passes its bar (F1 0.80 against 0.8) and the five syntheses read all of their cohort and state their coverage but stay below their bar (3, 2, 3, 1 and 4 of the ground truth's top-5 themes; 27 of 42 cited examples carry their theme). Treat the conversation lanes as previews. Details under "Measurements". - The cohort lanes never reach a scope with no children, with the flags off or on. The English
regression gate holds under every flag combination, and the 115-question customer-journey set
produces the same answer contexts as 0.10.3 (112 to 114 of 115 byte-identical per run; every
difference is one of three questions whose ranked cut-off or order also varies between boots
and builds of 0.10.3 itself). One single-customer change
is opt-in: Hindi and Hinglish "latest" / "last" words for the inferred newest-first order
(
CORTEX_RECENCY_CUES_MULTILINGUAL, off by default). - Parent-scope packs say they are storage-order samples (on by default). A recall or answer
at a parent scope that was served from the storage-order walk now carries one
warnings[]note, and the answer prompt one more line.CORTEX_PARENT_SAMPLE_WARNING=0turns it off. - Enrichment can be confined to chosen scopes (
CORTEX_ENRICHMENT_SCOPE_PREFIXES, unset by default), facts name the model that extracted them (extractor), events list their facts (derives), and the admin surfaces report the extraction model actually running. Details under "Enrichment on chosen scopes, and where each fact came from". - Erasure with proof.
POST /v1/erasuresnow writes a gdpr audit row (its id is in the 202), a durable receipt (ids, counts, timestamps and hashes, never content) that survives a restart, and anerasure_completelifecycle row itslifecycle_streamlink delivers. Two new routes:GET /v1/erasures/{id}/verify(trusted operator) re-reads every store for what the erasure removed and answers one check per store plusall_ok;GET /v1/erasures/{id}/receiptserves the receipt with its SHA-256. The redacted copies of an erased event no longer keep its labels and intent,GET /v1/auditlists newest first, and the ranked lane no longer counts a customer whose events were all erased. Details under "Erasure: receipts and proof". - Cross-customer answers say where their time went and what they cost. A cohort answer
carries
diagnostics.cost_usd(its model calls at the price book's rates), a finerdiagnostics.time_mssplit and aServer-Timingheader that separates CortexDB's own time (cortex) from the model's (llm);CORTEX_COHORT_WARM_ROOTSbuilds named parent scopes' projections at boot. Details under "Timing and cost". - The admin console uses all of it. Console › Ask across customers splits the wait into CortexDB, model and network, shows the cost and the read-all coverage, can answer without the model and re-run a plan without it; Console › Recall can search across customers; a presentation mode shows customers by alias only. Memories shows which model extracted each fact and from which event, a new Re-derive page re-extracts one customer's facts with another model and shows before and after, a new Erasure page erases one customer and shows the proof and the receipt, and Audit lists newest first. Details under "In the admin console".
- One time budget per cross-customer answer. Planner, reads, model calls and narration all
finish inside
CORTEX_COHORT_ANSWER_BUDGET_MS(85 s by default, at most 95 s), under a 100 s proxy origin limit; measured answers took 2.5-7.0 s (exact answers) and 13-48 s (reading 52-626 conversations). - Every error envelope carries the request's own id, in the body and in
X-Request-Id. /v1/recall422s say which field was wrong (details.part,details.reason,details.field_path), as/v1/answerdoes.scripts/cold_backup.pyrestores ownership as1000:999, the image'scortexuser and group.- No client changes. Every client released with 0.10.3 works against 0.10.4 unchanged; none of them sends the new fields yet.
Upgrade from 0.10.3
Before you upgrade: what clients and operators will notice
With every new flag at its default (the cohort lanes off), these are the only changes:
- Parent-scope recalls and answers (
CORTEX_PARENT_SAMPLE_WARNING, on by default). Only for requests at a scope whose child scopes hold data, and only when the pack's child events came from the storage-order walk (never at a leaf, never on a rankedsubtreepack):- the
/v1/recallpack (and the pack/v1/answer,/v1/composeanduse_pack_idreuse) gainswarnings: ["parent_pack_unranked_sample: events from K of M authorized descendant scopes, selected in storage order, not by relevance"](orevents from K authorized descendant scopes, ...when M is not known). K counts the descendant scopes the final pack holds; - a
/v1/answerbuilt from such a pack adds one line to its prompt: "The memories below come from K scopes under this one (customers, teams, sessions or similar) and were selected in storage order, not by relevance: they are a sample, so do not generalise them to every scope under this one." K is counted over the context the model reads. The answer text can change accordingly: it should stop presenting one child's history as "the customers'"; - nothing else changes. The note is not a
diagnostics.degraded_sourcesentry, it does not setdiagnostics.partial, and such answers stay cacheable underCORTEX_ANSWER_CACHE. A client that treats anywarnings[]entry as an error should treat this one as informational. - Kill switch:
CORTEX_PARENT_SAMPLE_WARNING=0(oroff,false,no) restores 0.10.3's packs and prompts byte for byte.
- the
- Error envelopes echo your request id. The errors a route's handler returns (experience,
recall, layer, lifecycle, event page, auth and others) used to mint a fresh
req_<uuidv7>for the body and put it inX-Request-Id, replacing the id the client sent. Now the body'srequest_id,X-Request-IdandX-Cortex-Request-ID(when present) all carry the request's own id. Consequences:- when the client sends
X-Request-Id,request_idis that value: do not treat it as a server-unique id, and do not expect thereq_prefix (docs/schemas/error_envelope.schema.jsonno longer has thereq_pattern); - the audit row, the request log line, the header and the body still join on one id.
- when the client sends
/v1/recall422 INVALID_BODYnow carriesdetails.part,details.reasonanddetails.field_path, like/v1/answer. Status, message and success responses are unchanged.- Fact provenance on the layer listings. A
GET /v1/eventsitem that live facts cite (an enriched capture, or a typed triple's own event) gainsderiveswith their ids, and aGET /v1/factsitem extracted from now on gainsextractor; an event or fact without them keeps its old shape.GET /v1/admin/layers/statsreports the extraction model the running router was built with (it used to report the model the Settings page had just saved). Recall and answer packs do not change. - Erasures (see "Erasure: receipts and proof"): the
POST /v1/erasures202 gainsaudit_id,receipt_urlandverify_url;GET /v1/erasures/{id}gainsaudit_id,created_atandcompleted_at, and answers after a restart; the preview'sestimated_duration_msis 20 to 35 times smaller (1 s plus 2 ms per event, was 250 ms plus 80 ms); the returnedlifecycle_streamnow carries only that erasure's row; a redacted event loses its labels and intent along with its content (the WAL twin of every erased event, and the v1 row of a descendant of a whole-scope erasure or of a/v1/forgetwithcascade: redact_events, which also loses its location).GET /v1/auditlists newest first: a client that read the first page as the oldest rows must sort or page differently. scripts/cold_backup.pydocuments and restores the image's owner as1000:999(usercortex, groupcortex), not1000:1000. Usechown -R 1000:999on a restored dir for the image.- Enrichment-on deployments only (nothing changes with enrichment off, the default):
- with
CORTEX_ENRICHMENT_LONG_TEXT_WINDOWINGon andCORTEX_ENRICHMENT_MAX_LLM_CALLS_PER_EVENTunset, the per-event attempt budget is now 4 per window, at least 16 (it was 8), which doubles the most a failing long event can spend on retries. To keep 8, setCORTEX_ENRICHMENT_MAX_LLM_CALLS_PER_EVENT=8(see Fixes); - a small batch of enrichment jobs now runs its jobs concurrently on the job permits free at that
moment (it ran them one after another), so a burst of a few long events reaches the provider in
parallel.
CORTEX_ENRICHMENT_CONCURRENCYstill bounds the jobs in flight: set it lower to spread them out.
- with
- Non-English "latest" questions on one customer: unchanged unless you opt in. The Hindi and
Hinglish recency words are read only with
CORTEX_RECENCY_CUES_MULTILINGUAL=1. With it on, such a question can serve the newest-events tail instead of ranked retrieval (about 3,600 more input tokens, about $0.018 at the opus rate, per affected answer);CORTEX_RECENCY_INFERENCE=0still turns all inference off. - The served contract grows, additively.
docs/schemas/openapi.jsondocuments the six new routes (the four cohort routes, and the erasure/verifyand/receipt), the new fields and theServer-Timingheader of the cohort responses;answer_response.schema.jsondocumentscontext_block, the optionalcohortblock,diagnostics.cost_usdand the citation fieldsscopeandsubject;stratified_pack.schema.jsondocuments the two new warning notes and the optionalsubtreesummary. None of them appears on a response unless the request opted in, apart from the parent note above.
Upgrade steps
Nothing is migrated and there is nothing to run afterwards. Replace the binary or image and restart as for any release, giving the server 120 s to stop. Then check:
GET /v1/admin/version:crate_version0.10.4 and thegit_shayou deployed. The capability list is the same 11 names as 0.10.3 (the two cohort names are known but not advertised; see "Detecting support").GET /v1/admin/ready:ready: true.- If a client of yours treats any
warnings[]entry as an error and calls parent scopes, either let it ignoreparent_pack_unranked_sampleor setCORTEX_PARENT_SAMPLE_WARNING=0. - To try the cohort lanes, see "Turning the lanes on".
Upgrading from 0.10.2 or earlier
0.10.4 contains everything in 0.10.3. Read RELEASE_NOTES_0.10.3.md, "Upgrade from 0.10.2" (and
from there the earlier notes) first; this page applies on top.
Questions across customers (experimental)
API reference: §8.3.1 (subtree on /v1/recall), §8.4.2 (cohort on /v1/answer), §8.4.3
(POST /v1/cohort/query, GET /v1/cohort/vocabulary, the plan grammar and the cursor), §8.15
(POST /v1/admin/cohort/warm, GET /v1/admin/cohort/status), §9 (pack warnings and subtree).
Operator settings: docs/generated/env-vars.md §9.2.
Why
A recall at a parent scope reads the child scopes' events in storage order, takes the first
10,000, and the token budget then keeps a few dozen of them. The query plays no part, so every
question at the parent got the same pack, usually from one customer, and /v1/answer either
abstained or described that one customer as "the customers". No existing setting (filters, views,
recency order, the answer's supplemental passes) answers "which customers did X", "how many" or
"what do customers say about Y" correctly at a parent. The new lanes route each kind of question
to a mechanism that can answer it, and refuse what they cannot answer.
Turning the lanes on
| Step | What |
|---|---|
| Exact lane | CORTEX_COHORT=on in the server's environment. Needs the unified WAL (the default); a legacy-WAL server answers 503 COHORT_REQUIRES_UNIFIED_WAL. |
| Ranked lane | CORTEX_SUBTREE_RECALL=request as well. The ranked and synthesis questions on /v1/answer need both flags. |
| Read all | On with the ranked lane: a synthesis or a "which customers said X" question reads every conversation of its cohort (CORTEX_COHORT_SYNTH_MAX_CALLS, default 32 map calls; 0 reads the ranked sample instead). It spends map-model tokens per question, about the cohort's conversation text; CORTEX_COHORT_SYNTH_MAP_MODEL picks a faster, cheaper model for it. |
| Caller | The scope.read.descend capability on the parent scope (the static operator key has it). Every count and list covers only the customer scopes the caller may read. |
| Request | /v1/answer with "cohort": {} (or with options), /v1/recall with "subtree": {}, or the /v1/cohort/* routes. view must be holistic, descend or absent. |
| Warm-up (optional) | POST /v1/admin/cohort/warm {"scope": "org:acme"} builds the scope's projection in the background ahead of the first question, and CORTEX_COHORT_WARM_ROOTS=org:acme does the same at boot; until it is ready a question answers 503 COHORT_WARMING with Retry-After. |
| Glossary (optional) | <data_dir>/cohort_glossary.json maps your business phrases to facets ("paid for an order" -> intent shop.order.payment.done). Configuration, never customer data; read on each cohort request. |
| Check | GET /v1/admin/cohort/status shows the switches, the knobs, the resident projections and their counters, whatever the flags say. |
The flags are read from the process environment only: the admin console's settings and feature overrides do not reach them. Both switches are read on every request; the numeric knobs once, at start.
Detecting support. cohort_query_v1 and subtree_recall_v1 are known names but are not listed
in GET /v1/admin/version capabilities[] in 0.10.4, so do not gate on them yet. Probe with
POST /v1/cohort/query instead: 404 means a server without the lanes (0.10.3 or earlier), 409 COHORT_DISABLED means the flag is off, anything else means the lane is served. An older server
ignores an unknown cohort field on /v1/answer and answers from the storage-order sample, so treat
a 2xx answer without a cohort block as "no cohort lanes here" and never show its text as a
cohort answer.
In the admin console. Console › Ask has an Across customers switch next to the scope. When it
is on, the question is asked at a parent scope (by default the root of the working scope, so
org:acme for org:acme/user:u1; the field is editable) with cohort, and the page shows the
narration above the structured block: the lane, the plan in words with the planner's notes, the
coverage ("read N of N"), the customers table (alias, real scope, counts, cited), the top-N,
funnel and segment tables, and citations that open each event at its own scope. It detects
support as described above, words every refusal (409, 422, 503), waits out
503 COHORT_WARMING, and has no client timeout, so a question that reads conversations can take
its time (8-48 s measured on 52-626 conversations, inside the 85 s answer budget). When the switch is off, Ask sends exactly the request it sent in 0.10.3. What the
console shows of the time, the cost and a read-all answer is under "In the admin console".
What each lane answers
| Question | Lane | Exact? | Example (synthetic shop) |
|---|---|---|---|
| Which customers did X | Exact list | Yes, complete over the authorized customers | "Which customers cancelled an order?" |
| None did X | Exact list | Yes: an empty set with its coverage ("0 of 240") | "Which customers viewed a Zephyr?" |
| How many | Exact count | Yes | "How many customers confirmed an order last week?" |
| Top N | Exact top (by customers or events; by intent, label family, day or week) | Yes; ties at the cut-off are listed | "Top 5 titles by viewers" |
| Funnel | Exact funnel (2-8 steps, optionally ordered) | Yes | "Order funnel: placed, paid, delivered" |
| Segments | Exact all_of / none_of | Yes | "Viewed both X and Y"; "confirmed but never had a conversation" |
| Sequence | Exact sequence | Yes | "Cancelled, then later confirmed" |
| Share | Exact share | Yes | "What share of views were hardcover?" |
| Latest | Exact latest | Yes | "The 10 newest orders, one per customer" |
| Attribute joined through an identifier | Exact, with a via join the planner infers from the vocabulary | Yes, when the attribute is a label | "Viewed paperbacks but ordered a hardcover" |
| What customers said | Read all: every conversation of the exact cohort the question names is read, and the customers whose conversations show it are listed (read: "sample" or read-all off: ranked, one or two excerpts per customer) | Read all: complete over every conversation read, as the model reads each one; sample: no, coverage stated | "Which customers discussed refunds in a chat?" |
| Themes across conversations | Read all: every conversation read, the themes grouped and counted by customer by the server (read: "sample" or read-all off: synthesis over one excerpt per customer) | Read all: counts complete over what was read, themes as the model reads each conversation; sample: no, coverage stated | "What do customers ask about financing?" |
| One customer | Focus: the alias from an earlier answer, answered by the single-customer path | As today | "What did C-4F2AQ7M do?" |
| Anything the planner cannot map | None | Refused with a reason, no model call | unknown_term, needs_text_attribute, needs_conversation_reading |
The exact lane only knows typed facets: an event's context.intent, its labels and its modality. An
attribute that lives only in event text (a colour, a size, a city written in a sentence) cannot be
counted exactly; label it at ingest. On the measured store, six attribute questions that 0.10.3
could not answer became exact once those attributes were written as labels.
The planner. /v1/answer maps the question to a plan with deterministic rules
(CORTEX_COHORT_PLANNER=rules, the default): it matches the words against the caller's authorized
vocabulary (intents, label values, label families, modalities, the glossary), reads time windows
(in June 2026, between ... and ..., last week, past 30 days, resolved against the anchor),
clauses, negation and sequence words, and refuses with unknown_term rather than drop a word that
could narrow the cohort. CORTEX_COHORT_PLANNER=llm adds one JSON-plan call (the model sees the
question and the vocabulary, never scopes, event ids or event text).
CORTEX_COHORT_PLANNER=auto, recommended when users ask in more than one language, runs the rules
first and makes that call only when they cannot plan the question (unsupported, unknown_term
above all) or the question is not English (Latin-script Hinglish, Devanagari, any non-Latin
script): an English question the rules can plan gets exactly the rules plan and answer, with no
model call, and cohort.notes says why the model planned when it did. An explicit cohort.plan
always wins. Every answer echoes the plan and how each phrase was mapped (cohort.notes), so a
reader can see that "cancelled" meant intent shop.order.cancel. GET /v1/cohort/vocabulary lists
what can be matched.
The time anchor. Relative windows resolve against cohort.anchor: now, data_end (the
newest authorized event under the scope) or an RFC 3339 instant. A store whose data ended weeks
ago answers "last week" with data_end correctly and with now as an empty week, and both say
which anchor and window they used.
Aliases and attribution
- The model sees customers only as aliases,
C-followed by 7 characters (for exampleC-4F2AQ7M), derived per parent scope from a key in<data_dir>/cohort_alias.key. The same customer keeps the same alias across requests, so a follow-up can name it (cohort.focus).naming: "alias_ephemeral"uses per-request aliases instead, andnaming: "scope"has the server replace each alias in the answer text with the customer's scope path after the model call. - The model never sees a scope or an event id. The caller always gets the real scopes:
cohort.result.subjects[]lists each customer's alias, scope, matching event count, first and last date and evidence event ids, withevidence_scopesgiving each evidence event's own scope (an event can sit below its customer, in a session for example). - Citations carry
scope(the cited event's own scope) andsubject(the alias the model saw). Every id reads back withGET /v1/events/{id}?scope=<that scope>. cohort.resultis the structured answer and the thing to display or grade. On the ranked lanes it lists the customers whose excerpts were read, and each row'scitedflag says whether the narration cites that customer: the model's reading of a sample, never a complete list.
Coverage and the number check
cohort.statusiscomplete(exact over every authorized customer),partial(a ranked or sampled read) orunsupported(withcohort.unsupported.reason).cohort.coveragestates the customers and events considered, the data span, the window and the anchor; on the ranked lanes also the customers read out of the customers in the cohort and, for a synthesis, the conversations read out of the conversations in the cohort.- After the narration call the server checks every number in the answer (digits in any script,
percentages, dates, times, number words, counts written
12.4kor3 lakh) against the numbers it computed, in the role the answer uses it in: a count against totals, a group's value only next to its own key, a customer's count only in a sentence naming that customer's alias, a share with rounding, a date only as a date. A number from the question counts only when the plan used it, and text in an excerpt never counts. On the exact lane any number that fails replaces the narration with the server's own summary; on the ranked lanes the sentences stating it are removed (fail closed when more than half would go). Either way the numbers are listed incohort.narration_unverified_numbersanddiagnostics.degraded_sourcesgainscohort_narration_unverified. - The check has known limits: inside one clause a key and a value are paired by position, so "Quill and Harbor had 179 and 184" without "respectively" pairs them in order; and a count that happens to equal another count of the block verifies.
Refusals
The lanes refuse rather than answer from a sample presented as complete:
| Condition | Response |
|---|---|
| A flag is off | 409 COHORT_DISABLED / SUBTREE_RECALL_DISABLED |
The caller lacks scope.read.descend | 403 POLICY_DENIED (a /v1/recall subtree instead falls back to today's pack with the warning below) |
view other than holistic / descend, cohort with stream or filters, an option out of range | 422 INVALID_BODY (a /v1/recall subtree with another view falls back with the warning below) |
| No authorized customer scope under the scope holds data (a leaf, for example) | 422 COHORT_NO_DESCENDANTS |
More than CORTEX_COHORT_MAX_EVENTS events under the scope, or a projection larger than CORTEX_COHORT_MAX_BYTES | 413 COHORT_SUBTREE_TOO_LARGE (nothing cached) |
| The projection is being built or repaired after a forget, or the build queue is full | 503 COHORT_WARMING with Retry-After; stale data is never served |
| A question the planner cannot map | 200 with cohort.status: "unsupported", the reason and a pointer to the vocabulary; no model call |
The ranked lane cannot run (the scope-partition map still building after a restart, more than CORTEX_SUBTREE_EXACT_VECTOR_MAX vectors in the subtree, an unsupported store) | /v1/recall: today's pack plus warnings: ["subtree_recall_unavailable: <reason>"]; /v1/answer: unsupported with degraded_sources: ["subtree_recall_unavailable"], never narrated from the storage-order sample |
| The ranked lane is at its in-flight cap or deadline | 503 SUBTREE_BUSY with Retry-After |
Safety
- Tenancy and access. Access is decided per customer scope (per exact event scope on the exact lane), from the same revision-checked authorization snapshot the rest of the read path uses, re-checked before the response. Counts, lists, the vocabulary and the planner's view cover the readable customers only, and nothing reports how many were left out. The ranked lane keeps the three read-path fences: authorized partitions only, the tenant restriction inside the full-text query, and the stored-tenant check on every hit.
- Forgets and erasures. Every row passes the same visibility checks as a recall (forgets, redactions, cancels, exclusions) plus the per-event erasure markers. Before every answer the projection is caught up with new writes and repaired for exactly the customers a forget or erasure touched; an untracked change forces a full rebuild in the background.
- Nothing is stored. The projection lives in memory only and is rebuilt from the WAL, so no
derived copy can survive an erasure (
docs/ERASURE_COVERAGE.md). The answer cache never stores or serves a cohort answer. - The model provider sees aliases and, on the exact lane, counts and dates; on the ranked lanes
the excerpts it narrates; on the read-all lane every conversation of the cohort (map calls, with
no alias, scope or event id: only the question, its topic words and the text), the map's short
labels (theme call) and the computed block (narration). With
CORTEX_COHORT_PLANNER=llmthe planner call also sends the vocabulary: intent names, label family names and the values of families with at most 64 values. Do not enable it if a label family holds customer identifiers.
Performance
Measured on the 300-customer test store (about 129,000 events and 139,000 vectors) in a 3-vCPU, 3 GB container:
- Exact lane. A cold projection builds in the background in 2.4-2.5 s and takes about 33 MB
(the process grew by 45 MB). Warm,
POST /v1/cohort/queryanswers in 9 ms at p50 (23 ms p95) with a plan and 55 ms (71 ms) with a question;GET /v1/cohort/vocabularyin 45 ms. A 1,000,000-event, 10,000-customer projection measures 146 MB; with 8 concurrent readers and repeated refreshes the process grew by 86 MB, less than one projection, because refreshes are applied in place and never copied. - Ranked lane.
/v1/recallwithsubtreeat the 300-customer parent: 82 ms p50 and 92 ms p95 client-side, of which the search is 60 ms; about 13 customers per pack. The first ranked recall after a boot pays for a cold embedding call (1-3 s). The scan runs on its own small worker pool, so leaf recalls keep theirs: under two concurrent ranked parent recalls, leaf answer contexts were served 3-5 % slower at p50 and within run-to-run noise at p99. /v1/answerwithcohort. One model call per answer (two with the LLM planner). With the answer model used for the measurement (claude-opus-4-8): exact-lane answers took 4.5 s at p50, with 1,200-3,200 input and 160-700 output tokens ($0.010-0.029 each); ranked lookups and syntheses took 10-17 s, with 9,200-13,500 input and 580-860 output tokens ($0.065-0.085 each). Planning and execution take under 100 ms; the rest is the model call.- Read all. With
claude-haiku-4-5as the map model (8 map calls in flight, 40,000-token / 20-conversation batches) and the check and narration calls on claude-opus-4-8, the final pass read 52 conversations in 13-16 s end to end, 165 conversations in 22 s and 574-626 conversations in 28-48 s (33 map calls, one theme call, six check calls in parallel, one narration of 7-10 s); the development passes of the same logic ranged 8-45 s. Cost per question: $0.18 (a lookup) and $0.89-2.11 (a synthesis): the map's input tokens are about the cohort's conversation text (about 2,100-3,000 tokens per conversation on that store), and the check calls add $0.3-0.6 at the answer model's rate. Withclaude-sonnet-5or the answer model as the map model the answers took about as long on 52-165 conversations and cost 1.4x and 2.6x as much. - Writes are unaffected. New-scope
wait=indexedcaptures under concurrent forgets: p99 282 ms on 0.10.3 and on 0.10.4 with the lanes off or on (an earlier candidate of this release; the final fixes do not touch the write path).
Timing and cost
API reference §8.4.2 ("Timing and cost") and §8.4.3.
diagnostics.cost_usdon a cohort answer: every model call of the answer (the planner, the narration, read-all map and theme calls) at the price book's rates, the ones the usage ledger records the same calls at.0when no model was called, absent when a model has no rate (declare it inCORTEX_MODEL_PRICES). Onread_allit equalscohort.read.cost_usd. A leaf answer never carries it.diagnostics.time_mssplitscohort.exec:cohort.exec.cohort_filter(the exact cohort) andcohort.exec.ranked_recallon the ranked lanes, followed by the ranked search's own stage times (subtree.wait,.candidates,.searchand.search.*,.visible_subjects,.hydrate; milliseconds only, never the counts);cohort.read_all.fetch,.map,.themes,.checkand the per-call.map.call_p50/.map.call_maxon a read-all answer.llm_msstill counts only the planner and the narration: the model's whole share is the header'sllm.Server-Timingon the200s ofPOST /v1/cohort/query,GET /v1/cohort/vocabularyand cohort answers:cortex;dur=...;desc="CortexDB", plan, exec, llm;desc="model", total, in ms with one decimal,cortex=total-llm. Every other response, leaf answers and recalls included, has no such header.- One budget for the whole answer.
CORTEX_COHORT_ANSWER_BUDGET_MS(default 85,000, clamped 40,000-95,000) runs from the request's start and bounds every model and store call of a cohort answer: the LLM planner call gets at most 15 s, the read-all map and theme calls end byCORTEX_COHORT_SYNTH_DEADLINE_MS(now at most 60,000) and at least 35 s before the budget ends, the check calls at least 25 s before it, and the narration by it. A narration that misses it answers with the server summary (cohort_narration_failed), never a 5xx, so a cohort answer stays inside a 100 s proxy origin limit (Cloudflare's) even under a slow provider. An abandoned map call is still billed by the provider and recorded in the usage ledger. CORTEX_COHORT_WARM_ROOTS(comma-separated scopes, unset by default) queues each named scope's projection build when the server starts, asPOST /v1/admin/cohort/warmwould, so the first question about it does not wait (about 2.5 s for the 300-customer test root). Each root is logged; a root the server refuses (COHORT_DISABLED,COHORT_WARMING, orINVALID_BODYfor an entry that is not a scope) is logged with its code and nothing else happens.benchmarks/latency/cohort_latency_bench.pymeasures plan, question, answer and ranked recall latency: p50 and p95 per question type of the server time (recall_total,Server-Timing), the round trip and the model's share, and the cost. It defaults to a built-in synthetic spec, takes--specfor your own questions, and refuses a server that is not on loopback unless--live.
Enrichment on chosen scopes, and where each fact came from
- Enrichment scope allowlist (
CORTEX_ENRICHMENT_SCOPE_PREFIXES, unset by default). A comma-separated list of scopes: while it is set, LLM enrichment runs only for captures in those scope subtrees, so enrichment can be turned on for a few customers of a store that must not spend on the others. It gates the deferred scanner, the retry set, the write-time dispatch, memory consolidation and the re-derivation route (POST /v1/admin/derived/rederiveanswers422 ENRICHMENT_SCOPE_NOT_ALLOWED, at the dry run too, when a job would re-enrich an event outside it). With it set, the compaction pass's whole-store LLM stages no longer switch on with enrichment. A capture outside it is passed over for good (widening the list does not backfill). Unset, nothing changes. - Facts name their extraction model.
GET /v1/factsitems carryextractor, the model the extraction call reported (for examplegpt-4.1-2025-04-14), on facts extracted from 0.10.4 on. - Events list their facts.
GET /v1/eventsitems carryderives: the live facts that cite the event. An event without facts has noderiveskey, as before. - The extraction model shown is the one running.
GET /v1/admin/layers/statsreported the model the Settings page had just saved, before the restart that loads it. It now reports the model the running router was built with, and the saved one aspending_restartuntil the restart; the re-derivation cost estimate prices the running model too.
Erasure: receipts and proof
POST /v1/erasures hard-deletes a scope's events (or the named ones) and everything derived from
them; docs/ERASURE_COVERAGE.md lists every store it clears. 0.10.4 makes the result provable:
- The gdpr audit row. The execute writes one row with
capability: forget.gdpr, the targetscope,event_id= the erasure id,gdpr: trueand the request'srequest_id(theX-Request-Idthe response echoes);decision: allowwith status 202, orfailedwith 502. Its id is the 202'saudit_id(anddetails.audit_idon a 502); a replay of the sameidempotency_keyanswers the sameaudit_id.GET /v1/auditnow lists newest first, so the row tops the console's Audit page. - The receipt. Every job that ends is stored under
erasure_job/<erasure_id>in thev1_layersstore's default column family (no new column family), soGET /v1/erasures/{id},/verifyand/receiptanswer after a restart. The receipt holds the erased scope path, the erased event ids (deleted and redacted apart, with a digest over all of them), the counts, the selection cutoff, timestamps, the requester, the audit row and request ids, the operator'saudit_note(verbatim, at most 1,000 characters: put a ticket number in it, not personal data), digests of the canonical request and of the idempotency key, the authorization footprint, the residual clear time and the list of what the erasure keeps by design. Never content.GET /v1/erasures/{id}/receiptserves{"receipt": {...}, "sha256": "...", "durable": true},sha256over exactly the servedreceiptbytes. - The proof.
GET /v1/erasures/{id}/verify(trusted operator: the static operator key, or an auth-disabled server) re-reads every store for what the receipt lists and answers one check per store (store,what,remaining,ok): the v1 log rows, idempotency records, content rows, vectors and the deferred-vector queue, BM25 documents, the WAL twins (no payload, no labels, no intent), chunk and session fragments, facts, beliefs, episodes and concepts citing the events, the no-resurrection markers, the scope tombstone, and, withCORTEX_COHORT=on, the cohort projections of the scope's ancestors and the erased customer's alias (a profile answers 404).checks_okis every check passing;all_okadds thatlog_residual_cleared_atis set. A check that cannot run answersok: falsewith a note, and so does a check that did not reach every row (the whole-scopecaptured_walscan stops after 1,000,000 rows). Two checks appear only when the proof cannot pass:entity_names, when the erased events had derived rows (they were enriched) andCORTEX_ENTITY_GRAPHis off, because the entity names and aliases enrichment extracted from them stay inresolver/andentity_vector/(docs/ERASURE_COVERAGE.md: "graph off: not covered"); andreceipt_ids, when the receipt's id lists were cut at 250,000 ids of a kind. A customer that was never enriched, or a server with the graph on, proves clean as before. On a test copy of the 300-customer store (graph off): a never-enriched customer (321 events, one of them a byte canary) proved clean, 13 of 13 checks,all_ok, its text and label in no file; a customer enriched just before (229 events, 20 facts) answeredall_ok: falseon theentity_namescheck alone, and its receipt listed the extracted names among what it keeps. Measured on a synthetic three-customer store (crates/cortex-api/tests/erasure_proof.rs): every check passes after a whole-scope erasure, a label canary is in nounified_wal/file once the residual cleared, and a planted content row and idempotency record are both reported. - No labels or intent left behind. The redacted WAL twin of every erased event used to keep the capture's labels and intent, and a redacted descendant row its labels, intent and location; both redactions now remove them. A copy an older binary redacted is redacted again when the erasure is re-run.
- Counts drop at once. The exact lane and the read-all synthesis already counted from the
projection; the ranked lane's
subjects_authorizednow skips a customer with no visible event. - The lifecycle link works. The 202's
lifecycle_streamdelivers that erasure'serasure_completerow and nothing else. - A realistic estimate. The preview's
estimated_duration_msis 1 s plus 2 ms per in-scope event (measured since DB-4: 0.56 s for 510 events, 2.8 s for 2,531).
What stays by design, and is listed on every receipt: the receipt itself, the audit rows, a
whole-scope tombstone (the scope path and cutoff), the no-resurrection markers (event ids), the
redacted copies' identities (ids, scope, timestamps), the scope registry entry until
DELETE /v1/scopes, the deleted idempotency records' bytes until RocksDB compacts
v1_idempotency/ (each write's idempotency key and scope path), copies outside the server (backups, the source system: a re-sync from it
re-creates the customer, model provider logs of earlier answers), and, on a receipt whose events
had been enriched while CORTEX_ENTITY_GRAPH was off, the entity names and aliases extracted from
them.
In the admin console
The embedded console uses the new fields. Every page and switch that existed in 0.10.3 sends the
same requests and renders the same way while the new switches are off (pinned by the web_ui
tests: the leaf Ask, Compose and Recall code is hashed against the previous candidate, and a
Recall record renders without the new provenance links).
- Console › Ask, across customers. A "Where the time went" bar splits the round trip into
CortexDB (plan and query), the model and the network (round trip minus the server's total),
from
Server-Timing; the phase chart is open and labelled, a wait for a warming projection is shown apart, and the cost is shown. A read-all answer shows its read block (map model, conversations read, unread, cut and dropped, calls, tokens, time) and its themes with their customer counts and example events. No model answers withskip_answer_llm; Re-run this plan without the model replays the answer's own plan and shows CortexDB's milliseconds. - Console › Recall, across customers. One ranked recall across the customers under a parent
(
subtree,diagnostics: "full"): the customers it reached, the events, and the search's stage times (counts are listed apart, never charted as times). - Presentation mode (Console page, per browser, off by default): views across customers and
the Re-derive and Erasure pages show a customer by alias or as
org:x/user:•••, and so do the scope rows of Memories, Observability › Usage, Data & scopes and Audit; the top-bar scope field is masked. Single-customer answers and event text still show what is stored. - Memories shows each fact's extraction model (
extractor) and source events, and each event's derived facts; a click opens the event or the fact with its links. - Re-derive (Memory section): the running and pending extraction model and the enrichment scope allowlist; one scope's events, a dry run (plan and cost estimate), apply, the job's progress, and before and after facts per source event with the log proof (captured head unchanged, derived rows appended). The "before" facts are kept in this browser only (the JSON download is the durable copy). A model-only Save and restart control swaps the extraction model.
- Erasure (Operate section): preview, type the exact scope to confirm, execute, wait for the
residual clear, run
/verifyand show every check, download the receipt with its SHA-256 recomputed in the browser, and exact cross-customer counts before and after. - Audit lists newest first, with a scope column, a gdpr filter and a link from an erasure to its row.
- Event cards (Memories, Recall, Ask evidence) show a small language badge when the event
carries a non-English language or a second script:
hi-Latnfor Hinglish,Devanagariforund-Deva. The detail rows addlanguage. An implicit-English event stores no language and renders exactly as before.
Fixes
- Handler errors replaced the client's request id with a freshly minted one in the body and in
X-Request-Id, so a client could not find its own request in the server's log. Every error envelope now carries the request's own id. /v1/recall422s did not say which field was wrong. They carrydetails.part,details.reasonanddetails.field_path.scripts/cold_backup.pynamed the wrong group (1000:1000) for the image'scortexuser. It is1000:999.- Parent-scope packs presented a storage-order sample as if it were the answer. They now say so (the note above), and the cohort lanes answer the questions the sample could not.
GET /v1/auditshowed the oldest rows. It truncated an oldest-first list, so on a busy server (the console polls every few seconds) a recent row, an erasure's included, never appeared. It now lists newest first andlimitkeeps the newest rows.POST /v1/erasureswrote no gdpr audit row (only the authentication layer's generic row, with no scope) and droppedaudit_note. It writes one, returns its id, and keeps the note on the receipt.- An erasure job vanished at a restart (
GET /v1/erasures/{id}answered 404). The job is kept as a durable receipt. - The redacted copies of an erased event kept its labels and intent, so a byte grep for a
booking or vehicle id still found it in
unified_wal/afterlog_residual_cleared_atwas set. Both redactions now remove them; re-running an erasure re-redacts copies an older binary left. - The erasure's
lifecycle_streamlink carried every lifecycle event (theerasure_idfilter was ignored, and the erasure emitted none). It now delivers that erasure'serasure_completerow only, also when opened after the 202. - The erasure preview's duration estimate was about 30 times too high (the pre-DB-4 cost of 80 ms per event). It is now 1 s plus 2 ms per event.
- The ranked lane's
subjects_authorizedcounted customers whose events were all erased or forgotten, and could tag and alias them. It now probes each authorized customer for a visible event (up to 5,000 customers). - A small burst of enrichment jobs ran one after another. The view builder groups jobs into
a batch and ran a batch's jobs in turn, so re-deriving four long conversations took the sum of
their extraction times (about 2.5 minutes for four 12-18 KB transcripts). A batch now also runs its
jobs on job permits that are free at that moment;
CORTEX_ENRICHMENT_CONCURRENCYstill bounds the jobs in flight, and under a backlog that holds every permit nothing changes. - Console: Settings › Save cleared a stored API key left blank. The key field never shows the stored key, and the server reads an empty value as "remove it", so saving a connection card to change its model also removed its key at the next restart. A blank key field is no longer sent.
- Console: Re-derive's link to the facts opened the wrong scope (the top-bar working scope,
not the customer just re-derived). It now makes that customer the working scope and opens
Memories on its Facts tab (
#/memories?layer=facts). - Console: Re-derive's before and after counted live facts only while the plan and the purge count the superseded versions too (a plan of 124 facts against "90 facts before"). Both sides now count every fact the events support, as live and superseded.
- Console: the re-derivation estimate was called "a planning upper bound". It is a planning figure from fixed per-event constants: four long non-English transcripts on gpt-4.1 cost $0.24 against an estimate of $0.05. The page says so and points at Observability › Usage.
- The cohort rules planner split Devanagari words at the virama and nukta (the refusal read
'ग' matches no intentforग्राहकों), and read a word of a caseless script as an all-capitals proper noun. Words keep their combining marks, a Devanagari glossary phrase matches, and a Devanagari question the rules cannot map is refusedunknown_termwith its whole words (or planned by the model underCORTEX_COHORT_PLANNER=auto). Latin-script questions tokenize as before. - "Latest" and "recent" questions in Hindi and Hinglish did not infer
order: "recency"(the cue words were English only), so on a typed event stream they were answered from relevance-ranked events and could name an older item as the newest. WithCORTEX_RECENCY_CUES_MULTILINGUAL=1(off by default) the answer route also readsहाल ही में,हालिया,नवीनतम,ताज़ा,सबसे नया,आखिरी,aakhri,haal hi mein, andlastin a question detected as Hinglish. A "last" word is not a cue before a time unit (आखिरी बार,last week, as Englishlastis not), as "in the end" (last mein,आखिरी में), or before a date or a price noun (aakhri tareekh,last date,आखिरी कीमत: a deadline or a final price is a stated fact). Every added word is one an English question does not use, so English inference is unchanged with the switch on or off (pinned against a frozen copy of the previous gates over the repo's English question files), and with it off a non-English leaf answer is byte-identical to 0.10.3's (pinned in the English regression gate). An inferred order still serves the newest-events tail outsideholisticonly on a typed event stream. - A non-English cross-customer question could drop a term the deployment had settled. Under
CORTEX_COHORT_PLANNER=auto(orllm) the JSON-plan call saw the vocabulary but neither the operator glossary nor the words the rules had already mapped, because the rules return no notes for a question they cannot plan. A negation asked in Hinglish was planned with one step of an event kind where the same English question excludes every event of that kind (the rules map the word to its label family), so it listed more customers than the English question. The planner prompt now carries the operator glossary (phrase and facet) and, for a question the rules could not plan, the term mappings they resolved before they stopped, and tells the model to keep those meanings in any language. The same prompt now says that the ranked lane reads the conversations, in any language: with the extra sections the model had started to refuse "which customers discussed X on a call" asneeds_conversation_reading. A reply with a sentence around its JSON object is read as the object (it used to be unusable), and when an unusable reply leaves a question refused,cohort.notessays so (LLM planner unusable (...)). A question the rules plan never makes the call, so English plans are unchanged. - A fully read long event could be dead-lettered as
input_coverage_limit. WithCORTEX_ENRICHMENT_LONG_TEXT_WINDOWINGon, knowledge and entity extraction run once per window, before any retry or verifier polarity re-check, so a two-window non-English call re-derived with gpt-4.1 could spend the default budget of 8 attempts, skip its re-checks and count as failed (3 of 4 events in one rehearsal run). Unset,CORTEX_ENRICHMENT_MAX_LLM_CALLS_PER_EVENTnow defaults to four attempts per window, at least 16, when windowing is on; without windowing it stays 8, and an explicit value wins. - Console: pasting the API key reset the Re-derive and Erasure forms. Any redraw of the page (pasting the key after a restart, a top-bar change) refilled the scope field with the last listed or previewed scope and set Re-derive's Modality back to "any", so a List events right after it picked every event of the customer (158 instead of its 4 conversations in one rehearsal run) or another customer; in Presentation mode the scope field is masked, so the swap could not be seen. What is typed or picked there now survives a redraw until it is listed or previewed.
- The console missed a fast restart. A server that came back within one 1.5 s poll was never noticed: no reload, and a false "didn't come back" message after 2 minutes. The console now also compares the server's uptime with the time since it asked for the restart.
Measurements
The figures come from this release's code on a test copy of a 300-customer event-stream store, with the cohort flags on and the static operator key, graded against ground truth computed from the store's source data and frozen before the runs (version 2). The exact-lane and attribute figures are from the last full pass of the logic as released (2026-10-02); the read-all figures were measured again after the last change to its prompts (below). No question, customer or theme of that store is named here.
A cross-customer question set
30 questions (31 items with an anchored variant of one), graded on the structured result
(cohort.result), never on the prose:
| Questions | /v1/cohort/query with a plan | /v1/cohort/query with the question | /v1/answer with cohort |
|---|---|---|---|
| 19 exact-lane questions: lookups, counts, top-N, a funnel, segments, a sequence, an empty set, a relative window | 19/19 | 19/19 | 19/19 |
| 6 attribute questions answered from labels added at ingest | 6/6 | 6/6 | 6/6 |
| 6 questions about what customers said: 1 content lookup, 5 conversation syntheses, each reading every conversation of its cohort | n/a | n/a (this route is the exact lane; it answers them unsupported) | 1/6 |
0.10.3 answered none of the 31 (its parent answers came from the storage-order sample).
- One business term, settled in the glossary. "booked" had two readings in that store: any
booking event, or only an initiated or confirmed one. Its owner ruled that it means any booking
event, the store's
cohort_glossary.jsonmaps it so, and with that mapping the attribute questions pass 6 of 6 (5 of 6 under the other reading). Three questions (two exact-lane, one attribute) depend on a glossary entry for a business phrase: without it they answerunknown_term. - Narration check. On 23 of the 25 exact-lane and attribute answers every narrated number verified. On the other 2 the check could not tie one number to the block (a count written as the word "two", and the event count of a group tied at the cut-off) and replaced the narration with the server's own summary, as designed; the structured result was right on all 25 (an earlier candidate verified 25 of 25).
- Time and cost. The 25 exact-lane and attribute answers took 2.5-7.0 s each (p50 3.6 s) at
$0.010-0.028 each, of which CortexDB's own time is about 50-75 ms; with the six read-all items a full
/v1/answerpass costs about $8.4 in model calls ($7.93 of it the read-all items). - The excerpt-sample lanes (
read: "sample", or read-all off), measured on an earlier candidate's pass of those lanes: the lookup states that it is a sample and, over the customers its narration cites, reaches F1 0.59 (precision 0.89, recall 0.44; over every customer whose excerpt it read, F1 0.73, recall 1.0; bar 0.8; 0.54-0.67 across runs). The syntheses state their coverage on all five and find 1-4 of the ground truth's top-5 themes (bar: 4 of 5 on 4 of 5 questions), and each cites some excerpts that do not carry the theme they are cited for; the limit is how much of each conversation fits in an 8,000-token context. That pass cost $0.87-0.90 for all 31 answers.
Reading every conversation (lane: read_all)
Map model claude-haiku-4-5, check and narration calls on claude-opus-4-8,
CORTEX_COHORT_SYNTH_PARALLELISM=8, _BATCH_TOKENS=40000, _BATCH_MAX=20, _MAX_CALLS=64, the
same strict grader:
| Content lookup | Syntheses (5) | |
|---|---|---|
| Conversations read | 52 of 52 | 52 to 626 of 52 to 626, every cohort equal to the ground truth's |
| Customers listed | 12: precision 1.00, recall 0.67, F1 0.80 (bar 0.8: met) | n/a |
| Ground-truth top-5 themes found | n/a | 3, 2, 3, 1, 4 (bar: 4 on 4 of 5 questions: not met) |
| Every narrated theme backed by 2 or more customers | n/a | yes, all five |
| Cited example conversations that carry the theme they are cited for | n/a | 27 of 42 (64 %; per question 12/12, 8/10, 0/6, 0/2, 7/12; bar: 100 %: not met) |
| End to end | 12.7 s | 16-48 s |
| Model calls | 24 map, 1 narration | 8-33 map, 1 theme, 6 check, 1 narration |
| Cost per question | $0.18 | $0.89-2.11 (0.30-1.49 M input tokens) |
1 of the 6 items passes the strict grade (the lookup).
Measured again after a blind-rule fix. The worked examples in the read-all map, theme and
check prompts had been written while iterating against this question set, and some of them
paraphrased its questions and its list of themes. They were replaced with examples from unrelated
domains (a bookshop, a phone app, a book club) or removed, and the six items were measured again:
the figures above. With the earlier prompts the same items scored F1 0.81, top-5 themes 4, 2, 3, 1
and 4, and 33 of 42 valid examples (79 %); those were development-set figures for prompts tuned on
the same questions, and are not claimed. What is left is agreement between two readers: the ground
truth was labelled by a different model against a fixed list of themes, and on some questions the
two name different reasons or aspects for the same conversation; which themes reach the top five
also varies from run to run. Map models compared on the lookup and two syntheses (earlier
prompts): claude-haiku-4-5 found the most of the lookup's customers (F1 0.77, against 0.62 for
claude-sonnet-5 and the answer model, which read "discussed X" more narrowly) at the lowest cost
($2.14 for the three questions, against $2.92 and $5.55), with the same pass rate on the
syntheses; it is the setting measured above. The development passes cost $44.83 in model calls on
the test store, and the measurement after the fix $7.93.
Single-customer scopes do not change
- The English regression gate (Golden A and B packs, and a leaf
/v1/answer) is byte-identical under all 64 combinations ofCORTEX_COHORT,CORTEX_SUBTREE_RECALL,CORTEX_PARENT_SAMPLE_WARNING,CORTEX_ENRICHMENT_SCOPE_PREFIXES,CORTEX_COHORT_WARM_ROOTSandCORTEX_COHORT_PLANNER=auto(the last three applied at boot, as the binary does), and no leaf response carries aServer-Timingheader. - The 115-question customer-journey regression set, replayed without the answer model
(
skip_answer_llm) against three 0.10.3 replays, in three configurations (every new flag off; the cohort lanes, read-all andCORTEX_COHORT_PLANNER=autoon; those plusCORTEX_RECENCY_CUES_MULTILINGUAL=1), on the release candidate image and on a measurement build of the same code (nine runs): 112 to 114 of 115 answer contexts are byte-identical per run. Every difference is one of three questions that also vary between boots or builds of 0.10.3 itself: one whose ranked cut-off changes on every boot (different in all nine runs), one that swaps one event at a ranked cut-off (four runs) and one whose two events come in the other order with the same bytes (one run). Every run had 0 errors, 0 degraded sources and the newest-events tail on the same 83 questions, and no response carries acohortblock. - A non-English leaf
/v1/answerwhose only recency cue is a Hindi or Hinglish word is byte-identical, withCORTEX_RECENCY_CUES_MULTILINGUALunset, to the same answer with order inference switched off, with and withoutview: "holistic"(the regression gate's own pin); with the switch on the same questions take the inferred order. - The multilingual mock evaluation: no top-5 change over 1,214 baseline queries.
- LongMemEval was not re-run for this release: its scopes have no children, so no request reaches the new code.
Questions in Hindi and Hinglish
Measured on a synthetic store shaped like the customer-journey data (12 customers under one
parent, typed event streams of about 35 events each, four of them bulked to about 435, one
conversation each in English, Hinglish or Hindi script), answer model claude-opus-4-8, read-all
map model claude-haiku-4-5, the same store and questions before and after. "After" is the
multilingual recency words on (CORTEX_RECENCY_CUES_MULTILINGUAL=1, measured before the switch
existed, when they were always read; off by default in this release) and
CORTEX_COHORT_PLANNER=auto:
| Question set | Before | After |
|---|---|---|
| "Latest" / "recent" questions on one customer in Hindi and Hinglish without an English cue word (5) | 0 of 5 named the newest item | 5 of 5 |
| "Latest call / chat" questions in Hindi and Hinglish (4) | 3 of 4 | 4 of 4 |
| The same shapes asked with an English cue word (4) | 4 of 4 | 4 of 4, the same inference |
Across customers, CORTEX_COHORT_PLANNER=auto: a count, a list, a read-all lookup and themes in Hinglish and in Hindi (8) | 7 refused unknown_term, 1 planned as a lookup of "karte hain" | 8 of 8, the same customers as the English question |
The 4 English cross-customer questions under auto | the same plan, count and customers as rules, no planner call |
A newest-slice answer now reads the newest-events tail, about 10,900 input tokens instead of about 7,300 (about $0.018 more per answer at the opus rate); the median answer time stayed at about 2.5 s. The planner call adds about 1-5 s and about $0.01 to a non-English cross-customer question ($0.018-0.024 for an exact one, $0.033-0.086 with read-all).
Other checks
On the release commit's code: the workspace test suite (8,374 tests passed, 0 failed, 209 test
binaries), the English regression gate with its multilingual-cue pin, the erasure proof suite, the
script gates, and the multilingual mock evaluation (no top-5 change over 1,214 baseline queries;
it measures /v1/recall, which CORTEX_RECENCY_CUES_MULTILINGUAL does not touch). On the
previous candidate, whose code differs only in the erasure proof's checks, the cohort answer's
deadlines, the read-all prompts and that default-off switch: the live Docker suite (43 of 43 and
every scenario), the upgrade and rollback probe (85 of 85, "Rollback"), the server,
tenant-isolation, backup and restore, retention, WAL-migration and commitment smokes, and the
Python SDK, CLI, MCP and TypeScript client suites.
Environment variables
All new, all read from the process environment only (docs/generated/env-vars.md §9.2 has the
full rows):
| Variable | Default | What it does |
|---|---|---|
CORTEX_COHORT | off | on: the exact lane, /v1/cohort/* and /v1/answer cohort. |
CORTEX_SUBTREE_RECALL | off | request: /v1/recall subtree and the ranked and synthesis lanes of /v1/answer cohort. |
CORTEX_COHORT_PLANNER | rules | llm adds one JSON-plan call per cohort answer; auto (recommended for multilingual users) makes it only for a question the rules cannot plan or that is not English. |
CORTEX_COHORT_MAX_EVENTS | 2,000,000 | Events under one scope; past it 413. |
CORTEX_COHORT_MAX_BYTES | 1 GiB | Memory of every resident projection together; least recently used evicted, 413 for one alone past it. |
CORTEX_COHORT_CACHE_ROOTS | 4 | Resident projections, and the depth of the build queue. |
CORTEX_COHORT_WAIT_MS | 3,000 | Wait for a build or repair before 503 COHORT_WARMING. |
CORTEX_COHORT_WARM_ROOTS | unset (nothing warmed) | Comma-separated scopes whose projections are queued for a background build at boot. |
CORTEX_COHORT_DEADLINE_MS | 5,000 | Reserved for a large-scope tier that is not built. |
CORTEX_COHORT_SYNTH_MAX_CALLS | 32 | Read-all synthesis: the most map calls one cohort answer makes to read every conversation of its cohort (at most 256); past it the newest are read and the answer is partial. 0: the ranked sample instead. |
CORTEX_COHORT_SYNTH_BATCH_TOKENS | 60,000 | Read-all: estimated conversation tokens per map call. |
CORTEX_COHORT_SYNTH_BATCH_MAX | 40 | Read-all: conversations per map call. |
CORTEX_COHORT_SYNTH_PARALLELISM | 6 | Read-all: map calls in flight per answer (at most 8); each also takes a permit of the shared provider lane. |
CORTEX_COHORT_SYNTH_DEADLINE_MS | 45,000 | Read-all: budget of the map and theme calls (at most 60,000), ending at least 35 s before the answer budget; unfinished calls leave their conversations unread (partial). |
CORTEX_COHORT_ANSWER_BUDGET_MS | 85,000 | The whole cohort /v1/answer from the request's start (40,000-95,000): planner call (at most 15 s), reads, map, theme and check calls and narration all finish inside it; a narration that misses it answers with the server summary. |
CORTEX_COHORT_SYNTH_CONVERSATION_MAX_BYTES | 48,000 | Read-all: bytes of one conversation a map call reads; a longer one keeps its head and tail and is counted. |
CORTEX_COHORT_SYNTH_MAP_MODEL | the answer model | Read-all: the model of the map and theme calls, on the answer provider and key. |
CORTEX_SUBTREE_MAX_INFLIGHT | 2 | Concurrent ranked passes; the excess waits to the deadline, then 503 SUBTREE_BUSY. |
CORTEX_SUBTREE_DEADLINE_MS | 2,000 | Deadline of one ranked pass, checked inside its scan. |
CORTEX_SUBTREE_EXACT_VECTOR_MAX | 1,000,000 | Vectors in the authorized subtree, counted before any scan; past it the ranked lane is refused (subtree_too_large). |
CORTEX_PARENT_SAMPLE_WARNING | on | The parent-scope note; 0 / off / false / no turns it off. |
CORTEX_ENRICHMENT_SCOPE_PREFIXES | unset (every scope) | Enrichment only for captures under these comma-separated scopes; set but naming no valid scope = nothing is enriched (§4.2). |
CORTEX_ENRICHMENT_MAX_LLM_CALLS_PER_EVENT | 8; with long-text windowing on, 4 per window, at least 16 | Not new: only its default with CORTEX_ENRICHMENT_LONG_TEXT_WINDOWING on changed (Fixes). |
CORTEX_RECENCY_CUES_MULTILINGUAL | off | 1: /v1/answer's inferred newest-first order also reads Hindi and Hinglish "latest" / "last" words (Fixes). Off: non-English answers as in 0.10.3. |
Error codes
| HTTP | Code | Where | Retriable |
|---|---|---|---|
| 409 | COHORT_DISABLED | /v1/cohort/*, /v1/admin/cohort/warm, /v1/answer with cohort, while CORTEX_COHORT is off | no |
| 409 | SUBTREE_RECALL_DISABLED | /v1/recall with subtree, a ranked plan, while CORTEX_SUBTREE_RECALL is off | no |
| 404 | COHORT_ALIAS_NOT_FOUND | A focus alias that names no customer the caller may read | no |
| 413 | COHORT_SUBTREE_TOO_LARGE | Past CORTEX_COHORT_MAX_EVENTS or CORTEX_COHORT_MAX_BYTES | no |
| 422 | COHORT_NO_DESCENDANTS | No authorized customer scope under the scope holds data | no |
| 503 | COHORT_WARMING | The projection is building or repairing, or the build queue is full (Retry-After) | yes |
| 503 | COHORT_STATE_CHANGED | POST /v1/cohort/query with a cursor whose last customer is no longer listed with the same count: restart the listing | yes |
| 503 | COHORT_UNAVAILABLE | A read the cohort lane needs failed | yes |
| 503 | SUBTREE_BUSY | The ranked lane's in-flight cap or deadline (Retry-After) | yes |
| 503 | COHORT_REQUIRES_UNIFIED_WAL | A cohort request on a legacy-WAL server | no |
| 422 | ENRICHMENT_SCOPE_NOT_ALLOWED | POST /v1/admin/derived/rederive re-enriching an event outside CORTEX_ENRICHMENT_SCOPE_PREFIXES (dry run too; nothing changed) | no |
| 409 | ERASURE_NOT_COMPLETED | GET /v1/erasures/{id}/verify on a running or failed job | yes, once it completes |
| 409 | ERASURE_RECEIPT_UNAVAILABLE | /verify or /receipt on a job with no receipt (still running, or run by an older binary) | no |
New strings, not error codes: the pack warnings parent_pack_unranked_sample: ... and
subtree_recall_unavailable: <reason>, and the /v1/answer diagnostics.degraded_sources codes
subtree_recall_unavailable, cohort_narration_unverified, cohort_narration_failed and
cohort_synthesis_partial (a read-all answer that did not read every conversation of its cohort).
Files in the data dir
Created only when a cohort lane is used; 0.10.3 ignores both:
cohort_alias.key: 32 random bytes, mode 0600, created on the first cohort request; the key of the stable aliases. Back it up with the data dir: deleting or replacing it changes every alias a client may have stored.cohort_glossary.json(optional, written by you): business phrases mapped to facets.
Written by every POST /v1/erasures job that ends (not a file: keys in an existing store):
erasure_job/<erasure_id>in thev1_layersstore's default column family: the job's receipt (see "Erasure: receipts and proof"). 0.10.3 never reads the prefix.
Startup warnings
The startup configuration lint lists each new variable with its effective value. With
CORTEX_ENRICHMENT_SCOPE_PREFIXES set it also names what the allowlist does not confine:
ENRICHMENT_SCOPE_ALLOWLIST_LAYERS_AUTO (CORTEX_V1_LAYERS_AUTO on),
ENRICHMENT_SCOPE_ALLOWLIST_COMPACTION (CORTEX_CONSOLIDATION_ENABLED opted in),
ENRICHMENT_SCOPE_ALLOWLIST_INERT (enrichment off) and ENRICHMENT_SCOPE_PREFIX_INVALID.
Rollback
A 0.10.3 image opens a data dir 0.10.4 has used, with the cohort flags on or off: no column
family, directory, WAL row kind or index field is added (the one new key prefix, erasure_job/,
holds erasure receipts that 0.10.3 never reads: after a rollback GET /v1/erasures/{id} answers
404 for jobs 0.10.4 ran, as for any pre-restart job on 0.10.3), and the cohort projection is in
memory only. To roll back, stop 0.10.4 with the usual 120 s grace and start 0.10.3 on the same
data dir; to turn the lanes off without a rollback, unset the flags and restart. A fact extracted
by 0.10.4 carries one more metadata key (extractor_model), which 0.10.3 keeps and ignores; 0.10.3
does not read CORTEX_ENRICHMENT_SCOPE_PREFIXES, so turn enrichment off in its environment
(unset CORTEX_ENRICHMENT_DELAY_SECONDS) before rolling back a store that relied on the
allowlist: 0.10.3 would otherwise enrich every scope.
What was checked, on one data dir, with a graceful stop between each step:
- 0.10.3 wrote captures under several customer scopes.
- 0.10.4 (flags on) read them, wrote more, built a projection and answered exact, joined
(
via) and ranked cohort questions, checked the vocabulary, and confirmed that a leaf recall has nosubtree. - 0.10.3 again opened the store and served everything; its
/v1/cohort/queryanswered404, and a customer forget succeeded. - 0.10.4 again rebuilt the projection without the forgotten customer.
85 of 85 checks passed, with no error line and no panic; the only new top-level entry was
cohort_alias.key.
Known issues
New in 0.10.4:
- Read-all answers are the model's reading of each conversation. The server counts customers
and conversations exactly over those readings, but whether a conversation "shows" a topic, and
which theme it belongs to, is the map model's judgement. Read-all sends every conversation of
the cohort, in full up to
CORTEX_COHORT_SYNTH_CONVERSATION_MAX_BYTES, to the model provider (with no scope, alias or event id); its spend is per question, in the foreground, and grows with the cohort's conversation text (cohort.read.cost_usdstates it). With the answer model as the map model a cohort of several hundred conversations can outrun the 45 s read budget and answerpartial; set a fasterCORTEX_COHORT_SYNTH_MAP_MODEL. - The ranked-sample lanes (
read: "sample", or read-all off) are below their accuracy bar (see "Measurements"): they read a ranked sample of excerpts under an 8,000-token context, and theircitedcustomers are the model's reading of that sample. Use them for exploration, and the exact lane for anything you count or list. - The idempotency store keeps an erased write's key bytes until RocksDB compacts it. An
erasure deletes the idempotency records of the erased events (the
idempotencycheck), but the raw-residual pass does not compactv1_idempotency/, so each deleted record's key (the client'sidempotency_keyand the write's scope path, no memory content) stays in that store's files until RocksDB's own compaction. The receipt lists it among what the erasure keeps. - A long event is read in one 12 KB window unless long-text windowing is on. Enrichment reads
the first
CORTEX_ENRICHMENT_LONG_TEXT_MAX_BYTES(12,000) of an event and, withCORTEX_ENRICHMENT_LONG_TEXT_WINDOWINGunset, dead-letters a longer one asinput_coverage_limit(its first window's facts are kept); a re-derivation then counts it failed. Where conversations run long, setCORTEX_ENRICHMENT_LONG_TEXT_WINDOWING=1(up to 4 windows, 48 KB); the per-event attempt budget then defaults to 16 (CORTEX_ENRICHMENT_MAX_LLM_CALLS_PER_EVENT, see Fixes). - Business terms need the glossary. The rules planner maps only what the vocabulary or
cohort_glossary.jsonspells. A term with two meanings in your business has to be settled there; the plan echo (cohort.notes) always shows which mapping was used. - The number check pairs by position inside a clause and accepts a count that equals another
count of the block (see "Coverage and the number check"). It fails closed: in the final
measurement pass 2 of the 25 exact-lane narrations (a number written as a word, a tied group's
event count) were replaced by the server's own summary, with
narration_unverified_numbersnaming the number; the structured result was right on all 25. - Read-all syntheses are the model's reading. Their counts are complete over every conversation read, but which themes a question gets, and which conversation stands as each theme's example, is a reading by two models (map, then check) and varies between runs; on the test store about four in five cited examples matched an independent labelling. Read the themes as a summary, and use the exact lane for anything you count or list.
- Text-only attributes cannot be counted. Label them at ingest.
- Scopes above 2,000,000 events (
CORTEX_COHORT_MAX_EVENTS) and subtrees above 1,000,000 vectors (CORTEX_SUBTREE_EXACT_VECTOR_MAX) are refused; the large-scope tier is not built. - A synthesis over a cohort of more than 50,000 events answers
413instead of reading a truncated sample. cohort_alias.keyis created without a temporary file. A process killed between creating and writing it leaves an empty file, and every cohort answer then answers503 COHORT_UNAVAILABLE(marked retriable) until the empty file is deleted. After replacing the file, restart the server so every lane reads the new key.cohort.focusresolves stable aliases only: an alias from anaming: "alias_ephemeral"answer answers404 COHORT_ALIAS_NOT_FOUND.- The parent note's M (
parent_pack_unranked_sample: events from K of M) can count a scope whose events were all erased or forgotten (it reads the scope-partition map). The ranked lane'ssubjects_authorizedno longer does, up to 5,000 customers; past that it counts every authorized customer, as before. - Erasure receipts have no delete route. A receipt names the erased scope and event ids (identifiers, not content) and is kept for the life of the data dir, like the audit trail.
/verifyreads what the receipt lists. A receipt keeps at most 250,000 event ids of each kind (event_ids_truncatedpast that); such a proof adds a failedreceipt_idscheck and is neverall_ok, although the scope-level checks still cover a whole-scope erasure's remainder.- An enriched customer's extracted names survive its erasure while
CORTEX_ENTITY_GRAPHis off. Enrichment registers the entity names and aliases it extracts inresolver/(and their vectors inentity_vector/) with or without the graph, and an erasure finds them only through the graph (boot warningENRICHMENT_RESOLVER_ERASURE_RESIDUE). The proof says so: such an erasure's/verifycarries a failedentity_namescheck and its receipt lists the names among what it keeps. A customer that was never enriched proves clean. Server-Timingis not in the CORSAccess-Control-Expose-Headerslist (adding it would change every cross-origin response), so a page on another origin cannot read it; the admin console is same-origin, and a non-browser client reads it as any header. A proxy in front of the server may add metrics of its own: read the header by metric name.- A read-all answer's per-call map time (
cohort.read_all.map.call_p50/call_max) includes the wait for a provider-lane permit, so under concurrent answers it measures queueing as well as the model. cohort.as_ofshows process-wide visibility counters, which move with every tenant's forgets.- With
CORTEX_COHORT_PLANNER=llm, a failed planner call puts the provider's error text incohort.notes; the rules plan is used. - A
POST /v1/cohort/querycursor is not bound to the request body: reuse it only with the same body. - Building a projection while a forget at or above the scope runs keeps the scope warming
(
503 COHORT_WARMING) until the forget finishes. - Non-English questions across customers need
CORTEX_COHORT_PLANNER=auto(orllm). The rules planner's vocabulary, operator words and glossary are English: under the defaultrulesa Hinglish or Hindi cross-customer question is refusedunknown_term(no model call, no cost). Setautowhere users ask in more than one language; English questions the rules plan are answered exactly as underrules. - Spanish text is not detected as Spanish: an untagged Spanish capture is stored as untagged
English (no
language, no gloss). - Not yet run on a copy of a real store: the design's safety probes (a canary tenant, a signed token whose roster excludes some customers, a marker-only partial erasure). The same properties are covered by tests on synthetic stores.
Carried over from 0.10.3: see "Known issues" in RELEASE_NOTES_0.10.3.md.
Clients
No client changes. Every client released with 0.10.3 (Python SDK cortexdbai 0.12.0, TypeScript
SDK cortexdbai 0.10.1, CLI cortexdb-cli 0.6.0, MCP cortexdb-mcp 0.8.0, connectors 0.2.22)
works against 0.10.4 unchanged. None of them sends cohort or subtree yet; call the routes and
fields directly (they are plain JSON). What existing clients can see:
- at parent scopes, the
parent_pack_unranked_samplenote in a recall pack'swarnings[](the MCP server'smemory_searchdoes not surface pack warnings, so an agent using it does not see the note); - an error envelope's
request_idequal to theX-Request-Idthey sent; /v1/recall422s withdetails.part,details.reasonanddetails.field_path.
Notes
Single-node remains the only supported topology.
Recommended for stores whose users ask in more than one language:
CORTEX_COHORT_PLANNER=auto. The default stays rules, so an existing deployment plans every
question as before until it opts in.