diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index f85869b..7615070 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -3,9 +3,10 @@ **Product:** Circe. Repo: `github.com/Absterrg0/circe`. Packages: `@circe/*`. `@circe/core` is a clean-room kernel. The previous Circe implementation is **legacy prior art**, archived separately, and is not a dependency of this architecture. -**Status:** Rev 4 frozen after three reviews. Implementation gate: open. The first 24 -scenarios and 9 properties must fail for real before they are implemented one by one. Do not -"improve" the architecture during implementation unless a test forces it. +**Status:** This is the rev 4 design record. The kernel, accepted-turn pipeline, file-backed +journal recovery and scripted inspector are implemented. Some sections below retain their +original planning language; see `README.md` for current implementation limits. Changes to +contracts should be supported by observable behavior and regression tests. **Process gate:** This document was written *before* inspecting any legacy implementation. Legacy Circe is not a dependency, was not read, and is not referenced by any type in this document. Its lessons enter only as generic evals, later. diff --git a/README.md b/README.md index ea236fb..9d0206f 100644 --- a/README.md +++ b/README.md @@ -18,13 +18,15 @@ first, identifiers second. ## Status -**Architecture rev 4 frozen. First kernel pass is green: all 24 scenarios and 9 properties -pass on deterministic stages only.** The kernel is intentionally small; test-forced contract -extensions are recorded in `ARCHITECTURE.md` §25. +The kernel, accepted-turn compiler, journal recovery, file journal, injected Jev client +adapter and inspector integration are implemented. The architecture document records the +design and its history; passing tests establish the exercised contracts, not a proof that +the runtime is correct under every failure. -**Semantic milestone (in progress): the durable kernel is proven; the semantic environment is -being built above it.** The deterministic classifier is frozen scaffolding, not product -intelligence. +The semantic product is still in progress. The deterministic classifier is frozen +scaffolding, not product intelligence. The inspector uses scripted model answers for a +fixed two-task scenario; its successful execution demonstrates wiring and recovery, not +natural-language accuracy. Phase 4 is green after a corrective pass: `@circe/context` holds the durable record model, an in-memory store with snapshotting and transitive invalidation, the dedicated @@ -44,7 +46,21 @@ with translated dependency references, materialized multi-ref annotations, pinne classes with retrieval completeness exposed on the frame, and a single canonical `entityRefKey` exported from core and used by kernel, resolver, and context alike. -Next: `@circe/perception-jev`. +`@circe/perception-jev` now provides an injected client interface and a read-only evaluation +loop. A host must supply the live SDK/transport implementation. No live model integration +or held-out semantic accuracy claim is included. + +Durability currently covers kernel interactions through `@circe/journal-file`. The semantic +context store remains in memory. The inspector creates a fresh context store per run and +does not persist frames, decision traces or semantic beliefs across process restarts. A +persistent semantic-store adapter and a continuing interaction host remain product work. +The semantic RED corpus still records missing interpretation behavior; tests asserting +that it remains RED must not be counted as successful semantic understanding. Risk +thresholds are initial policy values, not outcome-calibrated guarantees. + +Run `pnpm typecheck`, `pnpm test`, and `pnpm --filter @circe/inspector start` to check the +implemented contracts and scripted demo. `node tools/anti-corruption/check.mjs` checks +package boundaries. File-journal multi-process coordination and compaction are deferred. - `ARCHITECTURE.md` — types, boundaries, state machines, eval plan, decision log. **Start here for the kernel.** - `docs/SEMANTIC-BOUNDARY.md` — what the deterministic resolver may and may not do (frozen). @@ -57,6 +73,9 @@ Next: `@circe/perception-jev`. - `packages/core` — `@circe/core`: zero-dependency kernel. - `packages/context` — `@circe/context`: durable semantic records, ingestion, ContextFrame assembly. - `packages/perception` — `@circe/perception`: decision contracts, policy, traces (no model SDK). +- `packages/perception-jev` — injected model client adapter and read-only evaluation. +- `packages/journal-file` — append-only file journal and crash-tail recovery. +- `apps/inspector` — scripted integration demo with a local authority and executor. - `packages/resolvers-deterministic` — frozen bootstrap classifier + structural reference resolvers. - `packages/evals` — fake tasks/messages adapters, scenario runner, semantic corpus harness. diff --git a/docs/SEMANTIC-CONTEXT-MODEL.md b/docs/SEMANTIC-CONTEXT-MODEL.md index bc2b4e4..5e45649 100644 --- a/docs/SEMANTIC-CONTEXT-MODEL.md +++ b/docs/SEMANTIC-CONTEXT-MODEL.md @@ -6,7 +6,8 @@ candidate-source (seed) layer, bounded ContextFrame assembly with a **materializ semantic neighbourhood, retrieval metrics, transitive dependency invalidation, and authority-safe identity exist and are exercised by the eval suites. Perception contracts, the staged workflow orchestrator, and the acceptance policy live in `@circe/perception`. -The Jev backend is not implemented yet. +The injected Jev client adapter is implemented in `@circe/perception-jev`; a live SDK +integration is still supplied by the host. Corrective pass (review of `e40b533`), all in this revision: @@ -40,8 +41,10 @@ Pre-Jev contracts pass (review of `6d49430`), all in this revision: ## 1. What is durable, and what is not -Circe owns a structured semantic environment that persists independently of any model -conversation. The state is: +Circe's design owns a structured semantic environment independently of any model +conversation. The table below describes the intended durable records. The current context +store is in memory: these records do not survive process restart unless a host supplies a +persistent store implementation. Kernel state has separate journal-backed recovery. | Durable in Circe | Not state | |---|---| @@ -52,7 +55,7 @@ conversation. The state is: | decision traces for semantic judgments | a frontier model's reasoning | The kernel (`@circe/core`) already owns interaction state: scopes, referents, intents, -transactions, effects, provenance. The **context layer** (`@circe/context`, proposed) owns +transactions, effects, provenance. The **context layer** (`@circe/context`) owns the durable world model that interpretation runs against. The two are related but distinct: - kernel state answers "what is the runtime doing" (revisions, gates, frozen effects); diff --git a/packages/context/src/assembly.ts b/packages/context/src/assembly.ts index 4fba3dd..4208b1a 100644 --- a/packages/context/src/assembly.ts +++ b/packages/context/src/assembly.ts @@ -135,6 +135,17 @@ function round4(value: number): number { return Math.round(value * 10000) / 10000; } +function immutableSnapshot(value: T): T { + const copy = structuredClone(value); + const freeze = (entry: unknown): void => { + if (!entry || typeof entry !== "object" || Object.isFrozen(entry)) return; + Object.freeze(entry); + for (const child of Object.values(entry as Record)) freeze(child); + }; + freeze(copy); + return copy; +} + function resolveBudget(policy: AssemblyPolicy, expansionLevel: number): { level: number; budget: AssemblyBudget } { const level = Math.max(0, Math.min(expansionLevel, policy.expansions.length - 1)); return { level, budget: policy.expansions[level] ?? policy.defaults }; @@ -623,7 +634,7 @@ export async function assembleFrame( const hash = hashOf(payload); const retrievedBy = [...new Set(candidates.flatMap((candidate) => candidate.reasons))].sort(); - const frame: ContextFrame = { + const frame: ContextFrame = immutableSnapshot({ id: `frame:${request.interactionId}:${request.observationId}:${level}:${store.version()}:${hash.slice(0, 8)}`, version: store.version(), hash, @@ -642,7 +653,7 @@ export async function assembleFrame( generatedAt: request.now, completeness, }, - }; + }); // Persist the immutable input frame so an interpretation can later prove // which read set and coverage it was based on. store.recordFrame(frame); diff --git a/packages/context/src/store.ts b/packages/context/src/store.ts index 1a291e4..1c28903 100644 --- a/packages/context/src/store.ts +++ b/packages/context/src/store.ts @@ -282,7 +282,16 @@ class InMemoryContextStore implements SemanticContextStoreContract { const record = snapshot({ ...input, id } as SemanticAnnotation); this.annotationRecords.set(id, record); for (const ref of record.refs) this.push(this.annotationsByRef, entityRefKey(ref), id); - const generations = record.refs.map((ref) => this.currentGeneration(ref)); + const frame = this.frameRecords.get(record.evidence.contextFrameId); + // A semantic decision is grounded in the complete frame it received. + // Track every candidate generation so changes to evidence outside the + // annotation's subject tuple still invalidate that exact-frame judgment. + const generations = new Set(record.refs.map((ref) => this.currentGeneration(ref))); + if (frame && (!record.evidence.contextHash || frame.hash === record.evidence.contextHash)) { + for (const candidate of frame.candidates) { + generations.add(candidate.knowledgeGeneration); + } + } this.linkDependencies(id, [...record.dependsOn, ...generations]); // A result published after its evidence changed is stale on arrival; keep // it as history but never let it masquerade as current. The check covers @@ -308,10 +317,12 @@ class InMemoryContextStore implements SemanticContextStoreContract { if (annotation.evidence.contextHash && frame.hash !== annotation.evidence.contextHash) { return false; } - for (const ref of annotation.refs) { - const key = entityRefKey(ref); - const candidate = frame.candidates.find((entry) => entityRefKey(entry.ref) === key); - if (!candidate) return false; + const candidateKeys = new Set(frame.candidates.map((candidate) => entityRefKey(candidate.ref))); + if (annotation.refs.some((ref) => !candidateKeys.has(entityRefKey(ref)))) return false; + // The trace records the exact frame hash, so the full frame read set must + // still be current when its result is published. + for (const candidate of frame.candidates) { + const key = entityRefKey(candidate.ref); if (candidate.knowledgeGeneration !== this.generationByRef.get(key)) return false; } return true; diff --git a/packages/core/src/journal-recovery.ts b/packages/core/src/journal-recovery.ts index 584ad13..014e828 100644 --- a/packages/core/src/journal-recovery.ts +++ b/packages/core/src/journal-recovery.ts @@ -165,6 +165,7 @@ export function recoverState( case "discourse-snapshot": if (entry.interactionId === interactionId) { discourse = structuredClone(entry.state); + if (entry.commandKey !== undefined) appliedKeys.add(entry.commandKey); } break; case "effect-frozen": diff --git a/packages/core/src/journal.ts b/packages/core/src/journal.ts index f77e692..7baaeda 100644 --- a/packages/core/src/journal.ts +++ b/packages/core/src/journal.ts @@ -13,7 +13,9 @@ export function createInMemoryJournal(): InteractionJournal { entries.push(structuredClone(entry)); }, entries(): readonly InteractionLogEntry[] { - return entries; + // The readonly type is only a compile-time promise. Return detached + // records so a consumer cannot rewrite the journal's accepted history. + return structuredClone(entries); }, }; } diff --git a/packages/core/src/runtime.ts b/packages/core/src/runtime.ts index e8942b5..d22ef0d 100644 --- a/packages/core/src/runtime.ts +++ b/packages/core/src/runtime.ts @@ -2629,17 +2629,6 @@ export function createRuntime(deps: RuntimeDeps): SemanticRuntime { } function finalize(ctx: TurnCtx): TurnResult { - // Completing the turn is the durable proof of application; selection - // alone is not. Recorded once, under the actual reservation key. - if (ctx.rememberKey && !ctx.appliedLogged) { - ctx.appliedLogged = true; - log({ - kind: "command-applied", - interactionId: state.interactionId, - key: ctx.rememberKey, - at: ctx.now, - }); - } const scope = activeScope(); const allTransactions = [...state.transactions.values()].filter((tx) => !isTerminal(tx)); const transactionIds = new Set([...ctx.transactions, ...allTransactions.map((tx) => tx.id)]); @@ -2666,7 +2655,6 @@ export function createRuntime(deps: RuntimeDeps): SemanticRuntime { reply: ctx.reply ? { kind: ctx.reply } : undefined, errors: ctx.errors.length > 0 ? [...ctx.errors] : undefined, }; - if (ctx.rememberKey) state.commandResults.set(ctx.rememberKey, structuredClone(result)); const snapshot: DiscourseSnapshot = { scopes: [...state.scopes.values()].map((scope) => structuredClone(scope)), scopeStack: [...state.scopeStack], @@ -2693,8 +2681,22 @@ export function createRuntime(deps: RuntimeDeps): SemanticRuntime { kind: "discourse-snapshot", interactionId: state.interactionId, state: snapshot, + ...(ctx.rememberKey ? { commandKey: ctx.rememberKey } : {}), at: ctx.now, }); + // The snapshot certifies completion atomically with its discourse state. + // Retain the explicit marker for readers of the earlier journal format, + // which must also see that state before suppressing replay. + if (ctx.rememberKey && !ctx.appliedLogged) { + log({ + kind: "command-applied", + interactionId: state.interactionId, + key: ctx.rememberKey, + at: ctx.now, + }); + ctx.appliedLogged = true; + } + if (ctx.rememberKey) state.commandResults.set(ctx.rememberKey, structuredClone(result)); return result; } diff --git a/packages/core/src/types.ts b/packages/core/src/types.ts index dbb40f6..55c8843 100644 --- a/packages/core/src/types.ts +++ b/packages/core/src/types.ts @@ -654,6 +654,8 @@ export type InteractionLogEntry = kind: "discourse-snapshot"; interactionId: string; state: DiscourseSnapshot; + /** Completion and its recovered discourse state share one durable record. */ + commandKey?: string; at: IsoTime; } | { @@ -1095,6 +1097,7 @@ export function decodeJournalEntry(value: unknown): InteractionLogEntry { case "discourse-snapshot": journalString(record.interactionId, `${kind}.interactionId`); validateDiscourseSnapshot(record.state, `${kind}.state`); + if (record.commandKey !== undefined) journalString(record.commandKey, `${kind}.commandKey`); break; case "effect-frozen": journalString(record.interactionId, `${kind}.interactionId`); diff --git a/packages/evals/src/postmerge-context.test.ts b/packages/evals/src/postmerge-context.test.ts new file mode 100644 index 0000000..8ac3c8f --- /dev/null +++ b/packages/evals/src/postmerge-context.test.ts @@ -0,0 +1,110 @@ +import { describe, expect, it } from "vitest"; +import { type EntityRef } from "@circe/core"; +import { assembleFrame, authorityEvidence, createInMemoryContextStore } from "@circe/context"; + +const NOW = "2026-09-24T09:00:00.000Z"; +const a: EntityRef = { namespace: "tasks", type: "task", id: "a", authority: "tracker" }; +const b: EntityRef = { namespace: "tasks", type: "task", id: "b", authority: "tracker" }; + +function recordObservation( + store: ReturnType, + ref: EntityRef, + token: string, + at = NOW, +) { + return store.recordObservation({ + ref, + version: { authority: ref.authority, token, observedAt: at }, + attributes: { title: `${ref.id}-${token}` }, + evidence: authorityEvidence(ref.authority, token, at), + observedAt: at, + }); +} + +describe("post-merge context invariants", () => { + it("invalidates a frame-grounded belief when any candidate in its read set changes", async () => { + const store = createInMemoryContextStore(); + recordObservation(store, a, "a1"); + recordObservation(store, b, "b1"); + const { frame } = await assembleFrame( + store, + { + interactionId: "conversation-1", + observationId: "utterance-1", + utterance: { text: "compare these tasks", source: "text" }, + interaction: { recentReferents: [] }, + hints: { refs: [a, b] }, + now: NOW, + }, + ); + const annotationId = store.addAnnotation({ + refs: [a], + predicate: "same-underlying-issue", + value: true, + evidence: { + class: "semantic", + decisionId: "decision:frame-read", + backend: "test", + model: "test", + confidence: 0.9, + contextFrameId: frame.id, + contextHash: frame.hash, + decidedAt: NOW, + }, + dependsOn: [], + createdAt: NOW, + }); + + recordObservation(store, b, "b2", "2026-09-24T09:01:00.000Z"); + + expect(store.annotation(annotationId)?.invalidatedAt).toBe("2026-09-24T09:01:00.000Z"); + }); + + it("returns an immutable frame snapshot whose content still matches its hash", async () => { + const store = createInMemoryContextStore(); + recordObservation(store, a, "a1"); + const { frame } = await assembleFrame( + store, + { + interactionId: "conversation-2", + observationId: "utterance-2", + utterance: { text: "show this task", source: "text" }, + interaction: { recentReferents: [] }, + hints: { refs: [a] }, + now: NOW, + }, + ); + + expect(Object.isFrozen(frame)).toBe(true); + expect(Object.isFrozen(frame.candidates[0]?.attributes)).toBe(true); + expect(() => { + (frame.candidates[0]?.attributes as Record).title = "changed"; + }).toThrow(); + expect(store.frame(frame.id)?.hash).toBe(frame.hash); + expect(store.frame(frame.id)?.candidates[0]?.attributes.title).toBe("a-a1"); + }); +}); + +it.each(["late-related", "unrelated", "outside-frame"] as const)("checks frame dependencies without widening them: %s", async (scenario) => { + const store = createInMemoryContextStore(); + const outside: EntityRef = { ...a, id: "outside" }; + recordObservation(store, a, "a1"); + recordObservation(store, b, "b1"); + const { frame } = await assembleFrame(store, { + interactionId: "read-set", observationId: "input", utterance: { text: "compare", source: "text" }, + interaction: { recentReferents: [] }, hints: { refs: [a, b] }, now: NOW, + }); + if (scenario === "late-related") recordObservation(store, b, "b2"); + const id = store.addAnnotation({ + refs: [scenario === "outside-frame" ? outside : a], predicate: "related", value: true, + evidence: { class: "semantic", decisionId: "decision:read-set", backend: "test", model: "test", confidence: 1, + contextFrameId: frame.id, contextHash: frame.hash, decidedAt: NOW }, + dependsOn: [], createdAt: NOW, + }); + if (scenario === "unrelated") { + recordObservation(store, outside, "outside1"); + expect(store.annotation(id)?.invalidatedAt).toBeUndefined(); + } else { + expect(store.annotation(id)?.invalidatedAt).toBe(NOW); + } +}); diff --git a/packages/evals/src/postmerge-kernel.test.ts b/packages/evals/src/postmerge-kernel.test.ts new file mode 100644 index 0000000..f9cd8a6 --- /dev/null +++ b/packages/evals/src/postmerge-kernel.test.ts @@ -0,0 +1,52 @@ +import { describe, expect, it } from "vitest"; +import { createInMemoryJournal, createRuntime, type InteractionJournal, type InteractionLogEntry, type Observation } from "@circe/core"; + +describe("post-merge kernel durability regressions", () => { + it("does not expose mutable in-memory journal history", () => { + const journal = createInMemoryJournal(); + const entry: InteractionLogEntry = { + kind: "input", + at: "2026-09-24T00:00:00.000Z", + observation: { + kind: "ui", + id: "event:1", + at: "2026-09-24T00:00:00.000Z", + event: { action: "clear-selection" }, + }, + }; + + journal.append(entry); + const exposed = journal.entries() as InteractionLogEntry[]; + (exposed[0] as Extract).observation.id = "rewritten"; + exposed.push(entry); + + expect(journal.entries()).toEqual([entry]); + }); +}); + +const now = "2026-09-24T00:00:00.000Z"; +function uiRuntime(journal: InteractionJournal) { + return createRuntime({ + interactionId: "postmerge", journal, clock: () => now, + ports: { authorityFor: () => { throw new Error("UI inputs do not observe"); }, authorities: () => [], executorFor: () => undefined, executors: () => [] }, + classifier: { id: "unused", async classify() { throw new Error("UI inputs do not classify"); } }, + }); +} + +it.each(["select", "clear-selection"] as const)("every crash prefix recovers a %s input once", async (action) => { + const journal = createInMemoryJournal(); + const runtime = uiRuntime(journal); + await runtime.ingest({ kind: "ui", id: "select", at: now, event: { action: "select", refs: [{ authority: "a", namespace: "tasks", type: "task", id: "a" }] } }); + const beforeClear = journal.entries().length; + const clear: Observation = { kind: "ui", id: "clear", at: now, event: { action, refs: [{ authority: "a", namespace: "tasks", type: "task", id: "b" }] } }; + await runtime.ingest(clear); + const entries = journal.entries(); + for (let end = beforeClear; end <= entries.length; end++) { + const recoveredJournal = createInMemoryJournal(); + for (const entry of entries.slice(0, end)) recoveredJournal.append(entry); + const recovered = uiRuntime(recoveredJournal); + await recovered.ingest(clear); + expect(recovered.state().currentReferentId, `crash after record ${end}`).toBe(runtime.state().currentReferentId); + expect([...recovered.state().scopes.values()], `scopes after record ${end}`).toEqual([...runtime.state().scopes.values()]); + } +}); diff --git a/packages/evals/src/postmerge-perception.test.ts b/packages/evals/src/postmerge-perception.test.ts new file mode 100644 index 0000000..bfe1e1b --- /dev/null +++ b/packages/evals/src/postmerge-perception.test.ts @@ -0,0 +1,73 @@ +import { describe, expect, it } from "vitest"; +import { assembleFrame, createInMemoryContextStore } from "@circe/context"; +import { + ACCEPTANCE_POLICY_V1, analyzePerceptionResult, runPerceptionWorkflow, + type DecisionSpec, type PerceptionResult, +} from "@circe/perception"; + +const spec: DecisionSpec = { + id: "pick", kind: "choice", decision: "referent.singular", question: "Which one?", + options: [{ id: "a" }, { id: "b" }], +}; +const result = (): PerceptionResult => ({ + status: "RESOLVED", backend: "test", model: "test", latencyMs: 0, + answers: [{ id: "pick", kind: "choice", choice: "a", probabilities: { a: 1, b: 0 }, confidence: 1 }], +}); + +describe("post-merge perception decoding", () => { + it("reads status once and never upgrades an ambiguous response", async () => { + let reads = 0; + const raw = { ...result(), get status() { return ++reads <= 2 ? "AMBIGUOUS" as const : "RESOLVED" as const; } }; + const { frame } = await assembleFrame(createInMemoryContextStore(), { + interactionId: "test", observationId: "input", utterance: { text: "this", source: "text" }, + interaction: { recentReferents: [] }, now: "2026-09-24T00:00:00.000Z", + }); + const run = await runPerceptionWorkflow({ + backend: { id: "test", model: "test", async decide() { return raw; } }, frame, + workflow: { id: "test", stages: [{ id: "pick", specs: () => [spec] }] }, + policy: ACCEPTANCE_POLICY_V1, now: "2026-09-24T00:00:00.000Z", + }); + expect(reads).toBe(1); + expect(run.status).toBe("AMBIGUOUS"); + expect(run.state.accepted.size).toBe(0); + }); + + it("converts unreadable output into a structured decoding failure", () => { + const raw = { ...result(), get answers() { throw new Error("unreadable"); } }; + const analysis = analyzePerceptionResult([spec], raw); + expect(analysis.decoded).toBeUndefined(); + expect(analysis.issues).toEqual([expect.objectContaining({ kind: "malformed-result" })]); + }); + + it("reads an answer identity once", () => { + let reads = 0; + const raw = result(); + raw.answers[0] = { ...raw.answers[0]!, get id() { return ++reads === 1 ? "pick" : "other"; } }; + expect(analyzePerceptionResult([spec], raw).decoded?.answers[0]?.id).toBe("pick"); + expect(reads).toBe(1); + }); + + it("reads score distribution entries once before validating their support", () => { + let reads = 0; + const probabilities = [1, 0]; + Object.defineProperty(probabilities, "0", { get() { return ++reads === 1 ? 1 : 0; } }); + const score: DecisionSpec = { id: "score", kind: "score", decision: "risk", question: "Risk?", levels: ["low", "high"] }; + const raw = { ...result(), answers: [{ id: "score", kind: "score", score: 0, probabilities, confidence: 1 }] }; + expect(analyzePerceptionResult([score], raw).decoded?.answers[0]).toMatchObject({ probabilities: [1, 0] }); + expect(reads).toBe(1); + }); + + it("preserves prototype-named option probabilities as own data", () => { + const special: DecisionSpec = { ...spec, options: [{ id: "__proto__" }] }; + const raw = { ...result(), answers: [{ id: "pick", kind: "choice", choice: "__proto__", probabilities: JSON.parse('{"__proto__":1}'), confidence: 1 }] }; + expect(analyzePerceptionResult([special], raw).issues).toEqual([]); + }); + + it("does not mistake an inherited property for chosen-option probability", () => { + const special: DecisionSpec = { ...spec, options: [{ id: "__proto__" }, { id: "a" }] }; + const raw = { ...result(), answers: [{ id: "pick", kind: "choice", choice: "__proto__", probabilities: { a: 1 }, confidence: 1 }] }; + expect(analyzePerceptionResult([special], raw).issues).toEqual([ + expect.objectContaining({ kind: "invalid-choice" }), + ]); + }); +}); diff --git a/packages/journal-file/src/file-journal-durability.test.ts b/packages/journal-file/src/file-journal-durability.test.ts index e77cb85..236964b 100644 --- a/packages/journal-file/src/file-journal-durability.test.ts +++ b/packages/journal-file/src/file-journal-durability.test.ts @@ -350,6 +350,19 @@ describe("file journal: typed records and owned collections", () => { expect(open(path).entries()).toHaveLength(1); }); + it("retains completion identity on disk and rejects a malformed identity", () => { + const path = tempPath(); + const journal = open(path); + const entry = { kind: "discourse-snapshot" as const, interactionId: "i", at, state: snapshot, commandKey: "observation:select" }; + journal.append(entry); + journal.close(); + expect(open(path).entries()).toEqual([entry]); + + const invalidPath = tempPath(); + writeFileSync(invalidPath, JSON.stringify({ ...entry, commandKey: 42 }) + "\n"); + expect(() => open(invalidPath)).toThrow(/commandKey/); + }); + it("a valid discourse snapshot restores normally", () => { const path = tempPath(); const journal = open(path); diff --git a/packages/perception-jev/src/perception-jev.test.ts b/packages/perception-jev/src/perception-jev.test.ts index 62baf2d..3b7c6b7 100644 --- a/packages/perception-jev/src/perception-jev.test.ts +++ b/packages/perception-jev/src/perception-jev.test.ts @@ -256,15 +256,17 @@ describe("jev seam: adversarial review regressions", () => { retrievedAt: now, }, }); - const { frame } = await assembleFrame(store, { + const { frame: assembledFrame } = await assembleFrame(store, { interactionId: "interaction:mutate", observationId: "event:mutate", utterance: { text: "the safe one", source: "text" }, interaction: { recentReferents: [], activeScope: { id: "s", refs: [target], order: 0 } }, now, }); - // Incomplete retrieval + exhaustive high-risk question must block. - (frame.assembly.completeness as any).candidateTruncated = true; + // Supply an incomplete fixture to exercise the backend boundary. Real + // assembled frames are immutable, so alter a detached fixture only. + const frame = structuredClone(assembledFrame); + frame.assembly.completeness.candidateTruncated = true; const exhaustiveSpec: DecisionSpec = { ...choiceSpec, exhaustive: true, risk: "high" }; const backend = createJevBackend({ client, model: "jev-1" }); const run = await runPerceptionWorkflow({ diff --git a/packages/perception/src/validation.ts b/packages/perception/src/validation.ts index 878a17b..ed00cf3 100644 --- a/packages/perception/src/validation.ts +++ b/packages/perception/src/validation.ts @@ -179,14 +179,13 @@ export function validateDecisionSpecs(specs: DecisionSpec[]): SpecValidationIssu } function decodeAnswer( - spec: DecisionSpec, + id: string, answer: Record, ): DecisionAnswer | undefined { // Each field is read exactly once: a getter cannot present one value to the // validator and another to policy. - const id = answer.id; const kind = answer.kind; - if (typeof id !== "string" || typeof kind !== "string") return undefined; + if (typeof kind !== "string") return undefined; if (kind === "binary") { const probability = answer.probability; return { id, kind: "binary", probability: probability as number }; @@ -196,25 +195,25 @@ function decodeAnswer( const choice = answer.choice; const confidence = answer.confidence; if (!isRecord(rawProbabilities)) return undefined; - const probabilities: Record = {}; - for (const [optionId, value] of Object.entries(rawProbabilities)) { + const entries = Object.entries(rawProbabilities); + for (const [, value] of entries) { if (typeof value !== "number") return undefined; - probabilities[optionId] = value; } + const probabilities = Object.fromEntries(entries) as Record; return { id, kind: "choice", choice: choice as string, probabilities, confidence: confidence as number }; } if (kind === "score") { const rawProbabilities = answer.probabilities; const score = answer.score; const confidence = answer.confidence; - if (!Array.isArray(rawProbabilities) || rawProbabilities.some((value) => typeof value !== "number")) { - return undefined; - } + if (!Array.isArray(rawProbabilities)) return undefined; + const probabilities: unknown[] = [...rawProbabilities]; + if (probabilities.some((value) => typeof value !== "number")) return undefined; return { id, kind: "score", score: score as number, - probabilities: [...(rawProbabilities as number[])], + probabilities: probabilities as number[], confidence: confidence as number, }; } @@ -236,6 +235,16 @@ export function analyzePerceptionResult( specs: DecisionSpec[], result: unknown, ): DecodedPerceptionResult { + try { + return analyzeResult(specs, result); + } catch { + // Accessors and proxies are still untrusted output. Do not invoke their + // string conversion while reporting an unreadable result. + return { issues: [{ specId: "result", kind: "malformed-result", detail: "backend result could not be read" }] }; + } +} + +function analyzeResult(specs: DecisionSpec[], result: unknown): DecodedPerceptionResult { const issues: PerceptionValidationIssue[] = []; if (!isRecord(result)) { @@ -249,17 +258,16 @@ export function analyzePerceptionResult( ], }; } - if ( - typeof result.status !== "string" || - !PERCEPTION_STATUSES.includes(result.status as PerceptionStatus) - ) { + const status = result.status; + const rawAnswers = result.answers; + if (typeof status !== "string" || !PERCEPTION_STATUSES.includes(status as PerceptionStatus)) { issues.push({ specId: specs[0]?.id ?? "result", kind: "malformed-status", - detail: `backend status ${String(result.status)} is not a recognized perception status`, + detail: `backend status ${String(status)} is not a recognized perception status`, }); } - if (!Array.isArray(result.answers)) { + if (!Array.isArray(rawAnswers)) { issues.push({ specId: specs[0]?.id ?? "result", kind: "malformed-answers", @@ -268,13 +276,13 @@ export function analyzePerceptionResult( return { issues }; } const MAX_ANSWERS = 256; - if (result.answers.length > MAX_ANSWERS) { + if (rawAnswers.length > MAX_ANSWERS) { return { issues: [ { specId: specs[0]?.id ?? "result", kind: "malformed-answers", - detail: `backend returned ${result.answers.length} answers (limit ${MAX_ANSWERS})`, + detail: `backend returned too many answers (limit ${MAX_ANSWERS})`, }, ], }; @@ -283,7 +291,7 @@ export function analyzePerceptionResult( const requestedIds = new Set(specs.map((spec) => spec.id)); const byId = new Map(); const malformedIds = new Set(); - for (const raw of result.answers) { + for (const raw of rawAnswers) { if (!isRecord(raw)) { issues.push({ specId: specs[0]?.id ?? "result", @@ -292,23 +300,23 @@ export function analyzePerceptionResult( }); continue; } - if (typeof raw.id === "string" && !requestedIds.has(raw.id)) { + const id = raw.id; + if (typeof id === "string" && !requestedIds.has(id)) { issues.push({ - specId: raw.id, + specId: id, kind: "unexpected-answer", - detail: `backend returned an answer for unrequested decision ${raw.id}`, + detail: `backend returned an answer for unrequested decision ${id}`, }); continue; } - const spec = specs.find((candidate) => candidate.id === raw.id); - const decoded = spec ? decodeAnswer(spec, raw) : undefined; + const decoded = typeof id === "string" ? decodeAnswer(id, raw) : undefined; if (!decoded) { issues.push({ - specId: typeof raw.id === "string" ? raw.id : (specs[0]?.id ?? "result"), + specId: typeof id === "string" ? id : (specs[0]?.id ?? "result"), kind: "malformed-answer", - detail: `answer ${String(raw.id)} has an unrecognized shape`, + detail: `answer ${String(id)} has an unrecognized shape`, }); - if (typeof raw.id === "string") malformedIds.add(raw.id); + if (typeof id === "string") malformedIds.add(id); continue; } if (byId.has(decoded.id)) { @@ -390,7 +398,7 @@ export function analyzePerceptionResult( }); } } - if (optionIds.has(answer.choice) && answer.probabilities[answer.choice] === undefined) { + if (optionIds.has(answer.choice) && !Object.hasOwn(answer.probabilities, answer.choice)) { issues.push({ specId: spec.id, kind: "invalid-choice", @@ -475,17 +483,17 @@ export function analyzePerceptionResult( } answers.push(answer); } + const backend = result.backend; + const model = result.model; + const latencyMs = result.latencyMs; return { issues: [], decoded: { - status: result.status as PerceptionStatus, + status: status as PerceptionStatus, answers, - backend: typeof result.backend === "string" ? result.backend : "unknown", - model: typeof result.model === "string" ? result.model : "unknown", - latencyMs: - typeof result.latencyMs === "number" && Number.isFinite(result.latencyMs) - ? result.latencyMs - : 0, + backend: typeof backend === "string" ? backend : "unknown", + model: typeof model === "string" ? model : "unknown", + latencyMs: typeof latencyMs === "number" && Number.isFinite(latencyMs) ? latencyMs : 0, }, }; }