Jarvis voice · measured on Intel i7-1255U
The current English model now clears the proposed CPU voice gates in a working benchmark: 94 ms median audible output across 42 expanded utterances, 283 ms worst, 499 MiB peak RSS, 57 ms cancellation and 2.6% transcription WER. Kitten Nano FP32 remains the fallback.
5 September 2026
Native C++ / ONNX Runtime
Current English checkpoint
Recommended: english_2026-04, the optimized stephvax ONNX export, PocketTTS.cpp with the recorded compatibility patch, INT8 language model plus FP32 flow/decoder, temperature 0.3, one flow step, two CPU threads, first chunk one frame and later chunks capped at three frames. Use the first three seconds of Kyutai’s actual Alba casual recording, resampled to 24 kHz, and the bounded leading-silence filter described below.
The original three-prompt corpus scored 6.2% WER across three seeds (276 words), still below the proposed 7% ceiling. The expanded Alba run covered 14 prompts and three seeds, 504 reference words. It had 13 ASR word errors (2.6%); “Done.”, “No.”, “Stopping.” and the disconnection response were retained on every draw. One correction was transcribed as “the test passed, but the bill failed,” and the long single sentence lost words on two draws. These samples need listening review. Kitten had zero ASR errors on the same corpus, so Pocket wins the latency/memory trade, not every quality measure.
Prepared Expresso is a second usable Pocket voice: 165 ms median across the expanded prompts, 430 MiB peak, 69 ms standard cancellation and 2.4% WER, but its worst prompt reached 380 ms. Its one-time voice preparation peaked at 938 MiB; later fresh-runtime runs reused prepared state and stayed around 439 MiB. Prefer the shorter Alba reference to avoid that preparation spike.
Keep Kitten Nano FP32 as the fallback. On the expanded corpus it reached a 557 ms overall median, 730 ms on the standard reply, 4,193 ms worst, 541 MiB peak and 678 ms standard cancellation. Its latest-trigger cancellation took 1,983 ms. It offers better measured transcription accuracy here, but a much weaker interactive response.
This is the best measured Linux CPU configuration in this investigation. It is a working native benchmark candidate, not a finished production integration or a cross-platform release. No installed Jarvis provider was switched.
Earlier Kokoro, Sherpa, Kitten and Piper results remain available. This investigation supersedes the earlier “keep Kokoro” architecture recommendation; it does not change the installed application.
Milliseconds unless marked otherwise. Warm is the median 25-word prompt. Worst audible covers every warm prompt in that run. Cancel is request-to-synthesis-return after the 250 ms trigger; worst cancel covers 25, 250 and 1,000 ms triggers. RSS is MiB. Destroy / trim distinguishes runtime destruction from explicit allocator trimming. Gap is total simulated underrun over all warm outputs. WER is Parakeet transcription error, not a listening score.
| Configuration / raw data | Warm audible | Worst audible | Cancel | Worst cancel | Peak RSS | Destroy / trim RSS | RTF | Gap ms | WER |
|---|---|---|---|---|---|---|---|---|---|
| C++ mixed, Loona, raw | 828 | 1,004 | 65 | 133 | 501 | 318 / 102 | 0.13 | 0 | 10.9% |
| C++ full INT8, Loona, raw | 804 | 909 | 30 | 42 | 444 | 372 / 134 | 0.10 | 0 | 4.7% |
| C++ mixed, Loona, onset filter | 223 | 250 | 62 | 72 | 487 | 402 / 139 | 0.17 | 0 | 9.1% |
| C++ full INT8, Loona, onset filter | 159 | 213 | 63 | 122 | 500 | 327 / 143 | 0.14 | 0 | 9.1% |
| C++ mixed, Expresso 10 s, raw | 449 | 899 | 63 | 77 | 935 | 365 / 121 | 0.17 | 0 | 5.4% |
| C++ mixed, Expresso 10 s, onset filter | 156 | 277 | 51 | 76 | 934 | 372 / 120 | 0.19 | 0 | 4.7% |
| C++ mixed, Alba 3 s, onset filter | 118 | 152 | 57 | 70 | 493 | 345 / 90 | 0.17 | 0 | 6.2% |
| Kevin ONNX mixed, Loona, raw | 683 | 914 | 27 | 124 | 757 | 376 / 149 | 0.40 | 0 | 6.2% |
| Kevin ONNX mixed, Loona, onset filter | 395 | 589 | 5 | 198 | 763 | 475 / 149 | 0.49 | 58 | 7.2% |
| C++ mixed, 15-frame chunks, sensitive filter | 1,000 | 1,205 | 254 | 315 | 625 | 469 / 228 | 0.18 | 1,585 | 13.8% |
| C++ mixed, prepared Expresso voice | 161 | 282 | 79 | 79 | 439 | 383 / 107 | 0.17 | 0 | 5.2% |
| C++ mixed, Alba 3 s, expanded prompts | 118 | 283 | 57 | 70 | 499 | 370 / 130 | 0.17 | 17 | 2.6% |
| C++ mixed, prepared Expresso, expanded prompts | 165 | 380 | 69 | 73 | 430 | 344 / 111 | 0.18 | 56 | 2.4% |
| C++ mixed, Loona, expanded prompts | 196 | 310 | 49 | 70 | 538 | 387 / 130 | 0.18 | 0 | 2.2% |
| Kitten Nano FP32, expanded prompts | 730 | 4,193 | 678 | 1,983 | 541 | 175 / 70 | 0.13 | 0 | 0.0% |
| C++ mixed, Loona, 12-cycle soak | 222 | 288 | 69 | 69 | 518 | 377 / 146 | 0.17 | 0 | pending |
| cpp-int8-c3-trim | 808 | 912 | 47 | 58 | 501 | 397 / 146 | 0.13 | 0 | 9.8% |
| cpp-mixed-c3-trim | 831 | 1,012 | 58 | 68 | 481 | 386 / 124 | 0.16 | 0 | 9.8% |
| kevin-int8-c3-trim | 680 | 957 | 24 | 114 | 690 | 376 / 150 | 0.36 | 0 | 6.2% |
| kevin-mixed-c3-trim | 688 | 919 | 6 | 128 | 753 | 404 / 149 | 0.40 | 0 | 6.2% |
Historical Kitten Nano FP32: 725 ms warm, 745 ms cancellation, 433 MiB peak, 2.2% WER over 92 words. The new Kitten row above uses the expanded prompts and a fresh run. Model, speaker, text and hardware affect every number; do not treat author benchmarks on other machines as comparable measurements.
C++ produced its first buffer in roughly 26 ms, but the Loona output often began with 590–970 ms of near-silence. A bounded streaming filter waits for a 10 ms window at −50 dBFS and retains 40 ms before it. It removes only the leading prefix, stops removing after two seconds, and preserves every later pause. The −60 dBFS attempt opened on a faint decoder transient and did not help.
The runtime needs the English bundle’s learned BOS voice vector and no legacy space padding for short text. Mixed precision uses INT8 language-model weights and FP32 flow/decoder. The optimized export uses compact FP16 circular KV state. The C++ source was adapted explicitly; these are not measurements of an untouched upstream binary.
Three-frame chunks produced much faster cancellation than the 15-frame setting. Generation, decoder output, emitted PCM and completed worker shutdown are different events; cancellation was timed through the final thread join.
The short Loona reference met latency and memory targets but exceeded the quality target on the shared long paragraph. Reference identity was corrected during this study: initial files labelled “alba” actually used Expresso. The report labels them accurately. Actual Alba uses alba-mackenna/casual.wav. A prepared fixed voice avoids re-encoding a long reference on every fresh runtime.
These are generated artifacts, not recorded speaker output. Compare the voice, consonants, sentence ends, and the meaning of the correction prompt. Automated transcription cannot decide naturalness or personal preference.
Each configuration ran in a separate process, CPU affinity 0,2, with a two-thread inference budget. Heavy inference and builds ran sequentially. Pocket used english_2026-04, temperature 0.3, one flow step. C++ ONNX Runtime was 1.23.2; Kevin Python/ORT used 1.24.4, CPU execution, two intra-op threads and disabled spin waiting. This measures whole runtime configurations, not an isolated language-only performance comparison.
The original prompts contain 92 words; each three-draw quality result covers 276 words. The expanded corpus adds one-word responses, numbers, task ambiguity, disconnection, negation, contractions and a long single sentence. Each draw uses a different seed. NumPy and C++ random generators differ, so numeric seeds do not establish identical latent noise. Parakeet receives resampled 16 kHz PCM and no hotwords. It can misrecognize correct speech; every transcript is downloadable.
RSS includes the Python/NumPy measurement host. C++ inference itself needs no Python. Disk state is disabled in uncached rows; prepared-voice rows explicitly reuse model-specific state created by the preparation run. The library remains loaded after model destruction. Full worker exit would release its entire address space, but that was not substituted for the reported destruction measurements.
No microphone, speaker/DAC, OS permissions, desktop hotkey, mobile runtime, or other machine was tested. These are Linux CPU measurements. The 24-layer English checkpoint, Rust/Candle, XN and Apple MLX were not benchmarked here; no claim of a universal fastest model is made. The current compact English model and two ONNX runtimes address the concrete unresolved choice from the earlier review.
Exact source and model revisions, compatibility changes and reference corrections · Initial quality transcripts · Final candidate and Kitten transcripts
The C++ API needs a bounded output queue, explicit stream-error propagation, safe exclusive use of each runtime, and cancellation independent of the next audio read. Its current callback queue is unbounded and exceptions are printed rather than delivered as typed failures. The C++ sentence splitter also needs the bundle’s token bound for unrestricted text. These are production integration requirements, even if the measured prompts pass.
Integrate behind the existing Pipecat TTS service and preserve the ordinary voice lifecycle: Full and Controller own speech, Headless remains idle, disabling voice releases the worker, and cancellation discards buffered playback. Keep Kitten Nano FP32 as the fallback choice if native packaging or target-machine acceptance fails. Do not extend the Sherpa Pocket prototype.
PocketTTS.cpp source · Optimized ONNX bundle · Kevin runtime and bundle · Kyutai models and alternative runtimes · Voice references · KittenTTS