The backend reduces ContextFrame to four identity fields and coverage flags before calling its only injected client. The utterance, candidate attributes, relations, events, interaction state and capability semantics never reach that interface. Decision specs describe the bounded questions; they do not necessarily contain the evidence needed to answer them.

N1 assembles a real frame for “continue the payment worker” with two materialized candidates. The request contains the generic question “Which candidate does the phrase refer to?” and option IDs A, B and none, but neither the user's phrase nor either candidate's title. This is the same valid spec shape the retained integration tests use. An SDK wrapper implementing only complete(request) cannot infer the intended answer from those IDs and hashes.

Leaving a real interpreter out of scope is reasonable, but the model seam must carry its inputs. Requiring each host to secretly fetch frames by ID or embed all context into every question creates an undocumented second path. Pass an immutable, detached, bounded frame projection with the evidence already assembled by Circe.

Fix direction. Include the utterance and the assembled semantic context in JevDecisionRequest, preserving frame identity and retrieval completeness. Detach and freeze the full request. Keep the client read-only; do not add store access or language rules.

Probes: N1

92:       const request: JevDecisionRequest = {
93:         model: options.model,
94:         frame: {
95:           id: frame.id,
96:           hash: frame.hash,
97:           interactionId: frame.interactionId,
98:           observationId: frame.observationId,
99:         },
100:         // Detached and frozen: the client cannot rewrite the policy inputs
101:         // the orchestrator validated and will evaluate.
102:         decisions: deepFreeze(structuredClone(specs)),
103:         coverage: deepFreeze(structuredClone(frame.assembly.completeness)),
104:       };

The new evaluation metrics copy run.calls and classify failures only from run.failures. run.calls increments after a successful backend return, so a client invocation that throws is not counted. A well-formed result whose status is ERROR also creates no DecisionFailure, so backendFailures stays zero.

N5 invokes the Jev client twice, with one transport failure and one success. The evaluation returns two runs and one backend failure but reports only one call. N6 returns an explicit model ERROR result; errorRuns becomes one while backendFailures is zero. These are ordinary failure paths, not malformed JavaScript objects.

The harness is meant to support model evaluation before enabling execution. Its call counts and backend failure totals currently understate failures and usage. Count attempted calls at the invocation boundary, and represent explicit backend ERROR statuses consistently without double counting thrown or malformed responses.

Fix direction. Count every attempted backend call, including exceptions, and attribute explicit backend ERROR results as backend failures. Keep run counts separate from failure-event counts and document latency semantics for unsuccessful calls.

Probes: N5, N6

88:     metrics.calls += run.calls;
89:     metrics.errors += run.errors.length;
90:     metrics.totalLatencyMs += run.latencyMs;
91:     metrics.runStatuses[run.status] = (metrics.runStatuses[run.status] ?? 0) + 1;
92:     if (run.status === "ERROR") metrics.errorRuns += 1;
93:     for (const failure of run.failures) {
94:       if (failure.reason === "backend-error" || failure.reason === "malformed-backend-output") {
95:         metrics.backendFailures += 1;
96:       } else {
97:         metrics.workflowFailures += 1;
98:       }
99:     }

The evaluation loop awaits each workflow without an exception boundary. The workflow runner records some structural faults, but exceptions from specs, reduce, stop or resolveStatus propagate. A callback error therefore escapes the new batch evaluation API instead of contributing a workflow failure.

N7 evaluates two frames with a reducer that throws only for the first. runReadOnlyEvaluation rejects immediately, returns no metrics or runs, and never evaluates the second frame. A successful backend response is available, so this is a workflow fault rather than a model failure.

For a read-only corpus evaluation, one faulty frame should remain visible in the report instead of discarding the batch. Record a failed run with the correct attribution and retain completed runs while continuing the remaining frames. If a deliberate fail-fast mode is wanted, make it explicit and preserve the accumulated diagnostic output.

Fix direction. Convert workflow callback exceptions into structured per-run workflow failures, then continue the evaluation batch. Preserve frame/run identity and distinguish workflow errors from backend errors.

Probes: N7

78:   for (const [index, frame] of input.frames.entries()) {
79:     const run = await runPerceptionWorkflow({
80:       backend: input.backend,
81:       frame,
82:       workflow: input.workflow,
83:       policy: input.policy,
84:       now: input.now,
85:       runId: runIdFor(frame, index),
86:     });