Jarvis · implementation retrospective
Across 31 commits, Jarvis retired the standalone Companion, removed duplicate report and task state machines, moved control into one validated Controller, consolidated production voice on Pipecat, and fixed the authority bugs found in four follow-up reviews.
1 September 2026
Measured from 076de0643..befa40253 on codex/jarvis-orchestrator.
Architecture
Desktop Controller and standalone Companion each owned routing, voice, reports, setup, packaging, and lifecycle concerns. Server behavior was spread across Manager, Task Desk wrapper, completion/report/queue reactors, report outbox, speaker election and leases, and compatibility transports.
Full and Controller own control and speech. Headless owns execution only. JarvisController adapts a validated closed command into ordinary T3 orchestration. T3 threads remain task truth. Presentation is a live projection, not a second delivery system.
Current request path
utterance → acoustic grounding → Luna semantic proposal → deterministic catalog validation → fresh T3 lookup → ordinary provider/orchestration command
The model proposes action and visible names only. It never supplies IDs, grants approval, chooses an ambiguous target, reads the repository, or dispatches work.
| Area | Removed or collapsed | Replacement |
|---|---|---|
| Product shell | Standalone Companion app, release workflow, setup, updater, relay, voice routing | Full / Controller / Headless presets in one Jarvis product |
| Controller | JarvisManager plus executeWithTaskDesk | One JarvisController |
| Command parsing | Prepare, classify, interpret, plan, project resolver, task resolver pipeline | Schema-constrained semantic intent plus pure validation into a discriminated command |
| Reports | Completion reactor, report reactor, outbox, cursors, acknowledgements, election, speaker lease | Origin-scoped live presentation from durable T3 events |
| Task Desk | Shadow lifecycle, reactor, browser-style history, synthetic task state | Focused identity, bounded recent identities, one pending interaction |
| Follow-ups | Event reconstruction and queue recovery reactor | Normal durable SQL queue with atomic claim and deterministic retry identity |
| Voice | Companion voice and parallel Node Sherpa/Kokoro production runtime | Desktop worker supervising one bundled Pipecat runtime |
| Transport | Standalone Jarvis HTTP and Companion compatibility report subscription | Authenticated typed WebSocket RPC |
Correctness and behavior
Named steer, queue, stop, status, continue, review, reroute, answer, and task focus resolve an exact bounded task, reload its current T3 thread, then use that thread's project, model, runtime, interaction mode, pending request, and lifecycle state.
A named Rivvl task can be controlled while Jarvis is visible in another project. Current UI project no longer gates task commands or project listing. Context mismatch protection remains only for implicit “continue this” requests.
Choosing the second ambiguous task no longer degrades “stop task” into “focus task.” Jarvis pins the chosen task and resumes the original stop, steer, queue, status, continue, or review action.
Task Desk stores identities, not lifecycle snapshots. Missing or deleted threads are filtered instead of being manufactured as synthetic “known” tasks. Canonical focus RPC validates the node and thread before storing focus.
Start, review target, reroute target, and project focus fresh-load the selected project after semantic interpretation. Task-only and no-project commands skip project lookup entirely.
Typed approval and worker-input replies bypass Luna. The selected pending request is rechecked before dispatch, so a request that changed during interpretation cannot receive a stale answer.
Reviews create ordinary provider threads and reciprocal activities. Reroute commands carry source identity and target project identity, then inherit execution details from the fresh source thread.
Luna-low is configurable separately from the coding worker. Structured generation runs in a disposable neutral directory with tools, MCP, apps, hooks, and workspace access disabled. Unsupported unsafe adapters refuse the operation.
Presentation and persistence
T3 owns the result. Jarvis projects finalized provider results, failures, approvals, and input requests to the exact live origin interaction. Disconnecting does not create replay debt. Reconnecting clients inspect the ordinary durable thread.
Desktop voice
node-cpal or macOS renderer PCM → Desktop voice worker → Pipecat → Parakeet ASR / Kokoro TTS → PipeWire or local audio
Legacy native-speech owners, duplicate deadline/lifecycle layers, Node Kokoro worker/client, sherpa-onnx-node, obsolete voice smoke tooling, and Companion voice.
The worker/sidecar process boundary, node-cpal capture, macOS renderer capture, project vocabulary, and platform Sherpa packages used as the frozen Pipecat ONNX Runtime source.
Worker requests have bounded timeouts. Malformed Pipecat output fails immediately instead of becoming a silent timeout. Capture state was consolidated into typed state objects, and completion waits for native output drain.
Production smoke and benchmark commands exercise Pipecat. The frozen host fails closed on source drift, duplicate ONNX runtimes, or oversize output. The documented Linux host dropped from about 267 MiB to 138 MiB.
Other improvements
Evidence
The final review pass ran 39 focused Controller/core/eval tests, targeted lint, and server plus jarvis-core typechecks. The preceding live-authority pass ran 292 focused TypeScript tests and 29 Pipecat tests. Two live Luna-low smoke cases passed: new work with project selection, and exact named-task continuation.
Tests cover named task control, fresh thread state, clarification replay, queue identity, multi-node routing, canonical focus, live presentation, structured failures, worker timeout/failure behavior, and production Pipecat packaging paths.
Honest remaining work
Pre-Pipecat PCM buffering still needs real-device comparison for first-word clipping and push-to-talk latency. Static review cannot settle it.
The 50-case eval exists, but only two live cases were run during the pass to avoid unnecessary model spend. A deliberate benchmark run remains useful.
The manual Desktop voice protocol decoder is verbose but typed and working. Schema-driven cleanup was correctly deferred.
The Controller test file remains large. It is an ergonomics issue, not a product or authority defect.
Implementation record
942e47dab → 51fa45379 → 84595594d → 816117fa6 → 257ca8b1d → 2d3423a12 → a24a73c80 → 80d8bbbc6 → efc53193f → cb95a777c → e9fe39687 → 8e05bbf97 → b60feab9c → 43c4a4b38 → 481b854a9 → befa40253
The omitted hashes in that compact chain are focused documentation, compatibility removal, metadata, follow-up composition, and guidance commits. Git remains the complete record.