Voice control for coding agents is usually a gimmick. You say something, it transcribes poorly, sends text into a chat box, and the agent treats it like any other message. The interesting question is different: what would it take to actually steer an agent session with your voice, including interrupting it mid-turn, resuming threads, and managing its context window, all from the terminal?
Public repository: github.com/1Pio/codex-voice-steer
That is what cxv does. It is a local CLI that wraps voice around Codex's app-server protocol. You launch it, speak a wake word, and your words become a routed turn inside a live agent session, with the ability to steer, interrupt, and resume.
The product surface
The command is cxv. Run it plain and you get a foreground listener with a mini TUI. The wake word fires, Silero VAD detects speech, MacParakeet transcribes the finalized segment, and the text is delivered to Codex through the app-server protocol. You can also type directly with cxv text if you want precision without speaking. Session management handles thread resume and binding. cxv doctor checks readiness across Codex, the STT engine, the wake model, and the microphone. There are no launch agents, no cron jobs, no containers, and no autostart. You run it when you want it.
The key design decision was using finalized segment transcription rather than live hot-mic streaming. MacParakeet is excellent for finalized segments but poor for continuous live transcription, so the pipeline segments with VAD, writes a temporary WAV internally, transcribes the finished segment, and sends the final text. That choice made the system reliable instead of flaky.
Schema, not guesswork
The most valuable lesson in this project came from a failure. Early in development, I guessed the Codex app-server payload source enum based on intuition. It was wrong. The fix was to inspect the live installed schema using codex app-server generate-json-schema and work from the actual thread/start, turn/start, turn/steer, and turn/interrupt shapes.
That became a rule I carry into every protocol integration: read the schema, do not infer it. Guessing at API contracts is how you ship bugs that only surface at runtime.
Training a wake model honestly
The default wake word is Scarlett, a personal name I chose and trained a custom OpenWakeWord model for. The training process was real and the results were honestly mixed. The best committed candidate hit 216 out of 219 real positive samples, falsely triggered on 10 of 82 real negatives, and scored zero false positives across 500 synthetic negatives and 60 noise samples.
Those numbers are useful for daily local use but not flawless, and the important lesson was about the process. Perfect recall with zero hard-negative false hits requires relabeling or re-recording marginal samples, not blindly training candidate after candidate. One positive sample transcribed like "That's it" and scored near zero across every candidate, which means the training data itself was the bottleneck, not the model architecture. Knowing when to stop tuning the model and start fixing the data is a skill worth more than any single model.
Interrupting an agent mid-turn
The app-server integration supports turn/start, turn/steer, and turn/interrupt, which means you can redirect or stop a Codex turn while it is running. A later slice added automatic context compaction: the daemon listens for token-usage updates and compact events, schedules compaction after completed turns, and cancels it if you send new input. The system manages its own context window without you thinking about it.
Debugging the lifecycle taught me about cooperative cancellation, condition variables replacing busy-waits, and the difference between perceived latency and actual pipeline cost. Most perceived Codex latency turned out to be the Codex turn itself, not the STT or the handoff, which changed where optimization effort was worth spending.
The boundary that makes it work
cxv is voice-to-Codex control. It does not do text-to-speech. That responsibility belongs to msd, a separate project that I designed alongside cxv from the start. The naming is deliberate: cxv text sends typed input into Codex, while msd say produces spoken output. Two tools, two directions, one clean boundary.
The project also refuses scope creep by design. No launchd, no cron, no Hermes router integration, no Codex patches, no hosted voice product. cxv treats Codex as the runtime and wraps voice around it. Everything else is deliberately out of scope, and that discipline is what kept the project shippable instead of sprawling into a half-finished voice assistant.
What I take from it
Building cxv sharpened my instincts about three things: read the schema, fix the data before fixing the model, and draw product boundaries early. The tool works because it knows exactly what it is and refuses to be anything else.
