bingshuo/grok-build-upstream-mirror
Watch
1
0
Fork
You've already forked grok-build-upstream-mirror
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Synced from monorepo

Browse source 
Changes:
- Non-blocking coding-data sharing upsell banner
- Consolidate remediation in Doctor
- Auto mode defers fail-closed gate asks to the classifier
- Coalesce marketplace list fetches
- Allow removing a marketplace source by name
- Contain hung git marketplace sources (timeouts, non-blocking refresh, unbrick modal)
- Label failed workspace RPCs with error_kind
- Drop redundant explicit tonic/prost deps from xai-grok-shell
- Report real exit codes for completed background shells
- Narrow the date-rollover reminder to date-bearing templates
- Wire toolOverrides through the session and agent
- Security: Bash(git:*) allowlist matches whole command chain by prefix
- Split prompt-trigger telemetry and record classifier provenance
- Raise connectors-manager timeout to 60s
- Auto classifier honors recorded approvals for repeat actions
- Apply doctor fixes in the TUI
- Auto-mode classifier timeouts prompt instead of silently denying
- Scope subagent completion drains to the owning session
- Add the toolOverrides wire types
- Set client_identifier=grok-agent-sdk
- Accept both spellings of the workspace-teleport kill switch
- Persist one-shot occurrence journal
- Stop turns that poll the exact same tool call 16x in a row
- Copy compaction checkpoint files when forking sessions
- Auto-focus permission prompt from scrollback
- Esc cancels the running turn in non-vim and minimal modes
- List Ctrl+Z undo and redo in keyboard shortcuts
- Out-of-process macOS mic capture
- Show active auth mode on session-info
- Install the npm binary under $GROK_HOME
- Remove hover/click dead zones between dashboard items
- Route startup warnings to doctor
- Document [feedback.user] author identity config
- Extend bang command timeout
- Close combine-queued edit-hold race
- Integrate relocation recovery
- Expose privacy notice rollout flag
- Break harness discovery ref cycle so connections can idle-evict
- Shift/Alt+Enter inserts newline when editing a queued prompt
- Gate project Claude permissions on folder trust
- Echo response.create.event_id on response.created
- Toast when session creation fails from disk full
- Add shared test process lifecycle
- Enable dynamic workflows by default
- Add relocation transaction state machine
- Add shared test sandbox
- Surface auth failures on model-switch compact
- Persist durable scheduler expiry
- Confirm before removing extensions-modal items
- Re-run compact and prompt after login when compact hit expired auth
- Recap sends hosted tools under backend search
This commit is contained in:
grokkybara[bot] 2026-07-22 19:18:53 +01:00
commit a5727c5960
482 changed files with 37627 additions and 13402 deletions
Download patch file Download diff file Expand all files Collapse all files

228
crates/codegen/xai-grok-voice/src/audio/capture.rs
View file

@ -5,9 +5,19 @@
//! 48 kHz stereo F32 on macOS) and downmixes + resamples to 16 kHz mono for the
//! STT API. cpal streams are not `Send` on all platforms; capture runs on a
//! dedicated std thread and forwards PCM chunks through a sync channel.
//!
//! # Two roles: in-process backend and `__mic-capture` child
//!
//! On Windows this module is the capture backend itself (WASAPI's in-process
//! memory cost is modest). On macOS, opening CoreAudio in-process permanently
//! dirties several MB that the OS never returns after the stream drops, so
//! [`super::capture_subprocess`] re-execs the binary as a short-lived
//! `__mic-capture` helper instead; this module provides that child
//! ([`run_capture_child_cli`]) and the in-process fallback for when self-exec
//! is unavailable (e.g. the on-disk binary was replaced by an update).
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, AtomicU16, AtomicUsize, Ordering};
use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
use std::sync::mpsc::TrySendError;
use std::thread::{self, JoinHandle};
use std::time::Duration;
@ -23,15 +33,9 @@ pub struct CaptureHandle {
stop: Arc<AtomicBool>,
thread: Option<JoinHandle<()>>,
bridge: tokio::task::JoinHandle<()>,
peak: Arc<AtomicU16>,
}
impl CaptureHandle {
/// Session peak of device-delivered PCM (see [`meter_and_send`]).
pub fn peak_meter(&self) -> Arc<AtomicU16> {
Arc::clone(&self.peak)
}
/// Stop capture and wait for the thread to exit.
///
/// Dropping a `CaptureHandle` also stops capture (see the `Drop` impl), but
@ -82,11 +86,9 @@ pub fn spawn_pcm_capture(
let stop = Arc::new(AtomicBool::new(false));
let stop_flag = Arc::clone(&stop);
let peak = Arc::new(AtomicU16::new(0));
let peak_cb = Arc::clone(&peak);
let (ready_tx, ready_rx) = std::sync::mpsc::sync_channel::<Result<(), VoiceError>>(1);
let thread = thread::spawn(move || {
run_capture_loop(sample_rate, sync_tx, stop_flag, peak_cb, ready_tx);
run_capture_loop(sample_rate, sync_tx, stop_flag, ready_tx);
});
// Wait briefly for the device to actually open (mirrors the STT
@ -114,7 +116,6 @@ pub fn spawn_pcm_capture(
stop,
thread: Some(thread),
bridge,
peak,
})
}
@ -151,14 +152,7 @@ pub fn capture_pcm_for_duration(
let stop_flag = Arc::clone(&stop);
let (ready_tx, ready_rx) = std::sync::mpsc::sync_channel::<Result<(), VoiceError>>(1);
let thread = thread::spawn(move || {
// Duration probe does not read the session peak.
run_capture_loop(
sample_rate,
sync_tx,
stop_flag,
Arc::new(AtomicU16::new(0)),
ready_tx,
);
run_capture_loop(sample_rate, sync_tx, stop_flag, ready_tx);
});
// Surface device-open failures before recording instead of returning empty.
@ -198,7 +192,6 @@ struct CaptureStreamParams<'a> {
target_rate: u32,
sync_tx: std::sync::mpsc::SyncSender<Vec<u8>>,
stop: Arc<AtomicBool>,
peak: Arc<AtomicU16>,
/// Count of PCM chunks dropped because the channel was full. Logged off the
/// audio thread by `run_capture_loop`.
dropped: Arc<AtomicUsize>,
@ -208,7 +201,6 @@ fn run_capture_loop(
sample_rate: u32,
sync_tx: std::sync::mpsc::SyncSender<Vec<u8>>,
stop: Arc<AtomicBool>,
peak: Arc<AtomicU16>,
ready_tx: std::sync::mpsc::SyncSender<Result<(), VoiceError>>,
) {
let dropped = Arc::new(AtomicUsize::new(0));
@ -220,7 +212,6 @@ fn run_capture_loop(
sample_rate,
sync_tx,
Arc::clone(&stop),
peak,
Arc::clone(&dropped),
) {
Ok(v) => {
@ -240,11 +231,10 @@ fn run_capture_loop(
/// Open the input device, build, and start the cpal capture stream. All
/// device/config/permission failures surface here as a `VoiceError` so the
/// caller can report them before entering the steady-state loop.
fn open_capture_stream(
pub(super) fn open_capture_stream(
sample_rate: u32,
sync_tx: std::sync::mpsc::SyncSender<Vec<u8>>,
stop: Arc<AtomicBool>,
peak: Arc<AtomicU16>,
dropped: Arc<AtomicUsize>,
) -> Result<(cpal::Stream, String), VoiceError> {
let device = default_input_device()?;
@ -284,7 +274,6 @@ fn open_capture_stream(
target_rate: sample_rate,
sync_tx,
stop,
peak,
dropped,
};
@ -391,7 +380,6 @@ where
target_rate,
sync_tx,
stop,
peak,
dropped,
} = params;
let channels = in_channels as usize;
@ -413,7 +401,7 @@ where
if pcm.is_empty() {
return;
}
meter_and_send(&pcm, &peak, &sync_tx, &dropped);
send_pcm(&pcm, &sync_tx, &dropped);
},
|err| {
tracing::warn!(error = %err, "voice capture stream error");
@ -425,15 +413,9 @@ where
Ok(stream)
}
/// Meter then non-blocking send. Peak is updated **before** load-shed so the
/// silence guard sees what the mic delivered, not what survived backpressure.
fn meter_and_send(
pcm: &[i16],
peak: &AtomicU16,
sync_tx: &std::sync::mpsc::SyncSender<Vec<u8>>,
dropped: &AtomicUsize,
) {
peak.fetch_max(crate::pcm::peak_abs_i16(pcm), Ordering::Relaxed);
/// Non-blocking send from the real-time audio callback: shed load (and count
/// it) rather than ever blocking the device thread.
fn send_pcm(pcm: &[i16], sync_tx: &std::sync::mpsc::SyncSender<Vec<u8>>, dropped: &AtomicUsize) {
let bytes: Vec<u8> = pcm.iter().flat_map(|s| s.to_le_bytes()).collect();
match sync_tx.try_send(bytes) {
Ok(()) => {}
@ -494,20 +476,184 @@ fn resample_mono_i16(samples: &[i16], input_rate: u32, output_rate: u32) -> Vec<
output
}
// ---------------------------------------------------------------------------
// `__mic-capture` child mode (see the module docs and `capture_subprocess`).
// ---------------------------------------------------------------------------
/// Run the `__mic-capture` helper child. `args` is argv after the subcommand:
/// `--rate <N>` streams PCM16 mono LE at `N` Hz to stdout; `--device-info`
/// prints the default input device instead (one line, no stream opened).
///
/// Wire protocol (stdout): one status header line, then raw PCM.
/// - `READY <device>\n` followed by the PCM byte stream, or
/// - `INFO <name>\t<detail>\n` for `--device-info`, or
/// - `ERR <message>\n` and a non-zero exit on any failure.
///
/// The child exits when its stdout write fails (parent closed the pipe or
/// died) or when the parent kills it — it never outlives the capture session.
pub(crate) fn run_capture_child_cli(args: Vec<String>) -> i32 {
// Route the child's tracing (device open info, cpal warnings) to stderr,
// which the parent drains into its debug log — plain text, since the
// reader is a pipe, not a terminal. Stdout is the protocol channel and
// must stay clean.
let _ = tracing_subscriber::fmt()
.with_writer(std::io::stderr)
.with_ansi(false)
.without_time()
.try_init();
match parse_child_args(&args) {
Ok(ChildMode::DeviceInfo) => run_device_info_child(),
Ok(ChildMode::Capture { rate }) => run_capture_child(rate),
Err(msg) => {
emit_header(&super::protocol::err_line(&msg));
2
}
}
}
/// Write a header line to stdout without panicking: `println!` aborts on
/// EPIPE, and a helper whose parent died must exit quietly, not crash.
fn emit_header(line: &str) {
use std::io::Write;
let mut out = std::io::stdout().lock();
let _ = writeln!(out, "{line}");
let _ = out.flush();
}
/// What the helper child was asked to do (parsed from its argv).
#[derive(Debug, PartialEq, Eq)]
enum ChildMode {
Capture { rate: u32 },
DeviceInfo,
}
/// Parse the helper argv. Pure so the contract is unit-testable.
fn parse_child_args(args: &[String]) -> Result<ChildMode, String> {
let mut rate: u32 = crate::config::DEFAULT_SAMPLE_RATE;
let mut i = 0;
while i < args.len() {
match args[i].as_str() {
"--device-info" => return Ok(ChildMode::DeviceInfo),
"--rate" => {
i += 1;
rate = args
.get(i)
.and_then(|v| v.parse().ok())
.filter(|r| *r > 0)
.ok_or_else(|| "bad --rate".to_string())?;
}
other => return Err(format!("unknown mic-capture arg: {other}")),
}
i += 1;
}
Ok(ChildMode::Capture { rate })
}
fn run_device_info_child() -> i32 {
match input_device_info() {
Ok(info) => {
emit_header(&super::protocol::info_line(&info.name, &info.detail));
0
}
Err(e) => {
emit_header(&super::protocol::err_line(&e.to_string()));
1
}
}
}
fn run_capture_child(rate: u32) -> i32 {
use std::io::Write;
let (sync_tx, sync_rx) = std::sync::mpsc::sync_channel::<Vec<u8>>(64);
let stop = Arc::new(AtomicBool::new(false));
let stream = match open_capture_stream(
rate,
sync_tx,
Arc::clone(&stop),
Arc::new(AtomicUsize::new(0)),
) {
Ok((stream, device_name)) => {
emit_header(&super::protocol::ready_line(&device_name));
stream
}
Err(e) => {
emit_header(&super::protocol::err_line(&e.to_string()));
return 1;
}
};
let mut out = std::io::stdout().lock();
// Flush per chunk: chunks are small (~10 ms of PCM) and streaming STT
// wants them promptly, not batched by the stdout buffer.
loop {
match sync_rx.recv_timeout(Duration::from_secs(2)) {
Ok(chunk) => {
if out.write_all(&chunk).and_then(|()| out.flush()).is_err() {
break; // parent closed the pipe / died → stop capturing
}
}
// A silent device produces no writes, so parent death would go
// unnoticed and orphan this child; poll for reparenting (the
// parent normally kills us long before this fires).
Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {
#[cfg(unix)]
if std::os::unix::process::parent_id() == 1 {
break;
}
}
Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => break,
}
}
stop.store(true, Ordering::Release);
drop(stream);
0
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn meter_and_send_meters_shed_chunks() {
fn child_args_default_to_capture_at_default_rate() {
assert_eq!(
parse_child_args(&[]),
Ok(ChildMode::Capture {
rate: crate::config::DEFAULT_SAMPLE_RATE
})
);
let args = vec!["--rate".to_string(), "24000".to_string()];
assert_eq!(
parse_child_args(&args),
Ok(ChildMode::Capture { rate: 24000 })
);
}
#[test]
fn child_args_reject_bad_rate_and_unknown_flags() {
assert!(parse_child_args(&["--rate".to_string()]).is_err());
assert!(parse_child_args(&["--rate".to_string(), "0".to_string()]).is_err());
assert!(parse_child_args(&["--rate".to_string(), "x".to_string()]).is_err());
assert!(parse_child_args(&["--bogus".to_string()]).is_err());
}
#[test]
fn child_args_device_info_wins() {
assert_eq!(
parse_child_args(&["--device-info".to_string()]),
Ok(ChildMode::DeviceInfo)
);
}
#[test]
fn send_pcm_counts_shed_chunks() {
let (tx, _rx) = std::sync::mpsc::sync_channel::<Vec<u8>>(1);
let peak = AtomicU16::new(0);
let dropped = AtomicUsize::new(0);
meter_and_send(&[100], &peak, &tx, &dropped); // fills the channel
meter_and_send(&[-9_000], &peak, &tx, &dropped); // shed, but metered
send_pcm(&[100], &tx, &dropped); // fills the channel
send_pcm(&[200], &tx, &dropped); // shed
assert_eq!(dropped.load(Ordering::Relaxed), 1);
assert_eq!(peak.load(Ordering::Relaxed), 9_000);
}
#[test]

210
crates/codegen/xai-grok-voice/src/audio/capture_linux.rs
View file

@ -18,20 +18,16 @@
use std::io::Read;
use std::process::{Child, Command, Stdio};
use std::sync::atomic::{AtomicBool, AtomicU16, Ordering};
use std::sync::atomic::AtomicBool;
use std::sync::{Arc, Mutex};
use std::thread::{self, JoinHandle};
use std::thread;
use std::time::{Duration, Instant};
use tokio::sync::mpsc as async_mpsc;
use super::pipe::{self, READ_CHUNK};
use crate::error::VoiceError;
/// PCM read size from the recorder's stdout (bytes) — ~64 ms at 16 kHz mono
/// PCM16. Small enough to stream responsively, large enough to avoid syscall
/// churn on the reader thread.
const READ_CHUNK: usize = 2048;
/// How long to wait after spawning before deciding the recorder started cleanly.
/// A missing device or a stopped audio server makes the recorder exit within a
/// few ms; this surfaces that as an error instead of a session that "listens"
@ -142,11 +138,15 @@ fn require_recorder() -> Result<Recorder, VoiceError> {
fn spawn_recorder(sample_rate: u32) -> Result<(Recorder, Child), VoiceError> {
let recorder = require_recorder()?;
let mut child = Command::new(recorder.program())
.args(recorder.args(sample_rate))
let mut cmd = Command::new(recorder.program());
cmd.args(recorder.args(sample_rate))
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.stderr(Stdio::piped());
// setsid detach via the sanctioned helper (workspace subprocess rule): the
// recorder writes to a pipe and must not share the pager's controlling TTY.
xai_tty_utils::detach_std_command(&mut cmd);
let mut child = cmd
.spawn()
.map_err(|e| VoiceError::Config(format!("failed to start {}: {e}", recorder.program())))?;
@ -177,61 +177,7 @@ fn spawn_recorder(sample_rate: u32) -> Result<(Recorder, Child), VoiceError> {
}
/// Stop handle for the recorder subprocess (owns the child + reader thread).
pub struct CaptureHandle {
/// `Some` until `stop()` or `Drop` consumes it (kill + reap).
child: Option<Child>,
stop: Arc<AtomicBool>,
reader: Option<JoinHandle<()>>,
peak: Arc<AtomicU16>,
}
impl CaptureHandle {
/// Session peak of recorder-delivered PCM (metered before load-shed).
pub fn peak_meter(&self) -> Arc<AtomicU16> {
Arc::clone(&self.peak)
}
/// Stop capture: kill the recorder, reap it, and join the reader thread so
/// the input device is released before returning.
///
/// Dropping a `CaptureHandle` also kills and reaps the recorder (see
/// `Drop`), but without joining the reader; call `stop()` when you must be
/// sure the device is freed before continuing.
pub fn stop(mut self) {
self.stop.store(true, Ordering::Release);
if let Some(mut child) = self.child.take() {
let _ = child.kill();
let _ = child.wait();
}
if let Some(reader) = self.reader.take() {
let _ = reader.join();
}
}
}
impl Drop for CaptureHandle {
fn drop(&mut self) {
// Always kill the recorder so the mic is released even when `stop()` was
// never called — e.g. the STT session ended on its own (server close /
// error). Killing closes the child's stdout, so the reader thread's
// blocking `read` returns 0 and it exits. `Drop` must never block (it
// may run on an async executor), so the reap happens on a detached
// thread — without it every drop-path teardown (session supersede, STT
// error, connect failure) would leave a zombie until the pager exits.
self.stop.store(true, Ordering::Release);
if let Some(mut child) = self.child.take() {
let _ = child.kill();
// `Builder::spawn` (not `thread::spawn`) so spawn failure under
// thread exhaustion degrades to kill-without-reap instead of a
// panic — a panicking `Drop` during unwind would abort.
let _ = thread::Builder::new()
.name("voice-capture-reap".into())
.spawn(move || {
let _ = child.wait();
});
}
}
}
pub use super::pipe::ChildCaptureHandle as CaptureHandle;
/// Spawn subprocess capture; PCM16 LE chunks are forwarded to `pcm_tx`.
pub fn spawn_pcm_capture(
@ -239,20 +185,21 @@ pub fn spawn_pcm_capture(
pcm_tx: async_mpsc::Sender<Vec<u8>>,
) -> Result<CaptureHandle, VoiceError> {
let (recorder, mut child) = spawn_recorder(sample_rate)?;
let stdout = child
.stdout
.take()
.ok_or_else(|| VoiceError::Config(format!("{} produced no stdout", recorder.program())))?;
let Some(stdout) = child.stdout.take() else {
let _ = child.kill();
let _ = child.wait();
return Err(VoiceError::Config(format!(
"{} produced no stdout",
recorder.program()
)));
};
drain_stderr(&mut child, recorder.program());
pipe::drain_stderr(&mut child, recorder.program());
let stop = Arc::new(AtomicBool::new(false));
let stop_reader = Arc::clone(&stop);
let peak = Arc::new(AtomicU16::new(0));
let peak_reader = Arc::clone(&peak);
let device = recorder.program();
let reader =
thread::spawn(move || forward_pcm(stdout, pcm_tx, stop_reader, peak_reader, device));
let reader = thread::spawn(move || pipe::forward_pcm(stdout, pcm_tx, stop_reader, device));
tracing::info!(
recorder = recorder.program(),
@ -260,82 +207,7 @@ pub fn spawn_pcm_capture(
"voice capture stream (subprocess)"
);
Ok(CaptureHandle {
child: Some(child),
stop,
reader: Some(reader),
peak,
})
}
/// Drain the recorder's stderr to EOF on a detached thread so a chatty recorder
/// (xrun/underrun warnings, etc.) can't fill the pipe buffer and block its own
/// writes — which would stall capture, since the hot path never reads stderr.
/// Non-empty output is logged at debug for diagnostics. The thread ends on its
/// own when the child exits (EOF), so it is not joined.
fn drain_stderr(child: &mut Child, device: &'static str) {
let Some(mut stderr) = child.stderr.take() else {
return;
};
thread::spawn(move || {
let mut buf = String::new();
if stderr.read_to_string(&mut buf).is_ok() {
let msg = buf.trim();
if !msg.is_empty() {
tracing::debug!(device, stderr = msg, "voice recorder stderr");
}
}
});
}
/// Forward raw PCM from the recorder's stdout to the async STT sender until the
/// recorder stops (EOF on kill), the consumer goes away, or `stop` is set.
/// Generic over the reader for tests; production passes the child's stdout.
fn forward_pcm(
mut stdout: impl Read,
pcm_tx: async_mpsc::Sender<Vec<u8>>,
stop: Arc<AtomicBool>,
peak: Arc<AtomicU16>,
device: &'static str,
) {
let mut buf = vec![0u8; READ_CHUNK];
let mut dropped = 0u64;
loop {
if stop.load(Ordering::Acquire) {
break;
}
match stdout.read(&mut buf) {
// EOF: the recorder closed stdout (killed by teardown or exited).
Ok(0) => break,
Ok(n) => {
// Before try_send: shed chunks must still move the peak meter.
peak.fetch_max(crate::pcm::peak_abs_i16_le(&buf[..n]), Ordering::Relaxed);
// Never park this thread on the channel: `stop()` joins it, so a
// send that waits on a stalled STT consumer would turn teardown
// into a hang. Shed load instead when the consumer is behind —
// the same strategy as the cpal backend's real-time callback.
// (`read` itself is unblocked by the kill-on-stop path: killing
// the recorder closes stdout, so a waiting `read` returns 0.)
match pcm_tx.try_send(buf[..n].to_vec()) {
Ok(()) => {}
Err(async_mpsc::error::TrySendError::Full(_)) => dropped += 1,
// Consumer is gone: the session ended; stop capturing.
Err(async_mpsc::error::TrySendError::Closed(_)) => break,
}
}
Err(e) => {
tracing::warn!(device, error = %e, "voice capture read error");
break;
}
}
}
if dropped > 0 {
tracing::warn!(
device,
dropped,
"voice capture dropped PCM chunks (slow consumer)"
);
}
Ok(CaptureHandle::new(child, stop, reader))
}
/// Recorder that would be spawned, without recording ([`crate::probe::input_device_info`]).
@ -353,11 +225,15 @@ pub fn capture_pcm_for_duration(
seconds: u32,
) -> Result<(Vec<u8>, u32), VoiceError> {
let (recorder, mut child) = spawn_recorder(sample_rate)?;
let mut stdout = child
.stdout
.take()
.ok_or_else(|| VoiceError::Config(format!("{} produced no stdout", recorder.program())))?;
drain_stderr(&mut child, recorder.program());
let Some(mut stdout) = child.stdout.take() else {
let _ = child.kill();
let _ = child.wait();
return Err(VoiceError::Config(format!(
"{} produced no stdout",
recorder.program()
)));
};
pipe::drain_stderr(&mut child, recorder.program());
let duration = Duration::from_secs(seconds.max(1) as u64);
let deadline = Instant::now() + duration;
@ -407,30 +283,6 @@ pub fn capture_pcm_for_duration(
mod tests {
use super::*;
#[test]
fn forward_pcm_meters_shed_chunks() {
// One loud sample per READ_CHUNK read; capacity 1 forces the second
// read to shed. Both must register in the peak meter.
let mut pcm = vec![0u8; 2 * READ_CHUNK];
pcm[..2].copy_from_slice(&5_000i16.to_le_bytes());
pcm[READ_CHUNK..READ_CHUNK + 2].copy_from_slice(&(-9_000i16).to_le_bytes());
let (tx, mut rx) = async_mpsc::channel::<Vec<u8>>(1);
let peak = Arc::new(AtomicU16::new(0));
forward_pcm(
std::io::Cursor::new(pcm),
tx,
Arc::new(AtomicBool::new(false)),
Arc::clone(&peak),
"test",
);
assert_eq!(peak.load(Ordering::Relaxed), 9_000);
let first = rx.try_recv().expect("first chunk forwarded");
assert_eq!(crate::pcm::peak_abs_i16_le(&first), 5_000);
assert!(rx.try_recv().is_err(), "second chunk shed (channel full)");
}
#[test]
fn arecord_args_are_raw_s16_mono() {
let args = Recorder::Arecord.args(16_000);

395
crates/codegen/xai-grok-voice/src/audio/capture_subprocess.rs Normal file
View file

@ -0,0 +1,395 @@
//! Microphone capture on macOS via a short-lived self-exec helper process.
//!
//! Opening CoreAudio in-process permanently dirties the pager's memory
//! footprint: several MB for the HAL plus device capture buffers (tens of MB
//! with some input routes), none of it returned to the OS after the stream is
//! dropped. Capture therefore runs out of process, like the Linux recorder
//! backend: the pager spawns `current_exe __mic-capture --rate N`, the child
//! streams raw PCM16 mono LE to stdout behind a one-line `READY`/`ERR` header
//! (see [`super::capture::run_capture_child_cli`]), and all audio-stack
//! memory is freed when the child exits with the utterance. The helper is the
//! same executable, so the terminal's mic permission grant applies unchanged.
//!
//! In-process capture ([`super::capture`]) remains the fallback when the
//! helper cannot run at all — self-exec unavailable, or the spawned binary
//! doesn't speak the helper protocol (e.g. it was replaced by an update mid
//! run) — and can be forced with `GROK_VOICE_CAPTURE=inprocess`.
use std::io::Read;
use std::process::{Child, ChildStdout, Command, Stdio};
use std::sync::Arc;
use std::sync::atomic::AtomicBool;
use std::thread::{self, JoinHandle};
use std::time::Duration;
use tokio::sync::mpsc as async_mpsc;
use super::pipe::{self, ChildCaptureHandle};
use super::protocol;
use crate::error::VoiceError;
/// Env escape hatch: `GROK_VOICE_CAPTURE=inprocess` forces the legacy
/// in-process cpal backend (accepting its permanent footprint cost).
const CAPTURE_BACKEND_ENV: &str = "GROK_VOICE_CAPTURE";
/// How long to wait for the helper's status header. Device open takes
/// hundreds of ms; exec of the (usually page-cached) binary adds tens more.
/// Matches the in-process backend's 5 s open handshake.
const READY_TIMEOUT: Duration = Duration::from_secs(5);
/// Stop handle for a capture session: the helper child, or the in-process
/// fallback stream.
pub enum CaptureHandle {
Child(ChildCaptureHandle),
InProcess(super::capture::CaptureHandle),
}
impl CaptureHandle {
/// Stop capture and wait until the device is released.
pub fn stop(self) {
match self {
CaptureHandle::Child(h) => h.stop(),
CaptureHandle::InProcess(h) => h.stop(),
}
}
}
/// Whether the env escape hatch forces the in-process backend.
fn force_inprocess() -> bool {
std::env::var(CAPTURE_BACKEND_ENV).is_ok_and(|v| v.eq_ignore_ascii_case("inprocess"))
}
/// Why the helper handshake produced no `READY`/`INFO` payload.
#[derive(Debug)]
enum HandshakeFailure {
/// The helper ran and reported `ERR` (a real device/permission error), or
/// timed out opening the device. Surfaced as-is; an in-process retry
/// would fail identically.
Reported(VoiceError),
/// The helper could not run or doesn't speak the protocol (spawn failure,
/// EOF/garbage/oversized header — e.g. the binary was replaced by an
/// update mid-run). The caller falls back to in-process capture.
Broken(VoiceError),
}
/// Spawn the helper (detached from the TTY, stdin null, stdout/stderr piped)
/// and hand back its stdout. Kills the child on the defensive missing-stdout
/// path so it can never outlive the error.
fn spawn_helper(args: &[&str]) -> Result<(Child, ChildStdout), VoiceError> {
let exe = std::env::current_exe()
.map_err(|e| VoiceError::Config(format!("current_exe for mic helper: {e}")))?;
let mut cmd = Command::new(exe);
cmd.arg(crate::MIC_CAPTURE_SUBCOMMAND)
.args(args)
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::piped());
// The helper must not share the pager's controlling TTY.
xai_tty_utils::detach_std_command(&mut cmd);
let mut child = cmd
.spawn()
.map_err(|e| VoiceError::Config(format!("spawn mic helper: {e}")))?;
let Some(stdout) = child.stdout.take() else {
let _ = child.kill();
let _ = child.wait();
return Err(VoiceError::Config("mic helper produced no stdout".into()));
};
pipe::drain_stderr(&mut child, "mic-helper");
Ok((child, stdout))
}
/// Read the helper's one-line status header. Byte-at-a-time so no PCM after
/// the newline is consumed from the stream.
fn read_header(stdout: &mut impl Read) -> Result<String, HandshakeFailure> {
let mut line = Vec::with_capacity(64);
let mut byte = [0u8; 1];
// Cap far above any real header so a corrupt child can't feed us forever.
while line.len() < 4096 {
match stdout.read(&mut byte) {
Ok(0) => {
return Err(HandshakeFailure::Broken(VoiceError::Config(
"mic helper exited before ready".into(),
)));
}
Ok(_) if byte[0] == b'\n' => {
let text = String::from_utf8_lossy(&line);
let text = text.trim_end_matches('\r');
return match text.split_once(' ') {
Some((tag, payload)) if tag == protocol::READY || tag == protocol::INFO => {
Ok(payload.to_string())
}
Some((tag, message)) if tag == protocol::ERR => Err(
HandshakeFailure::Reported(VoiceError::Config(message.to_string())),
),
_ => Err(HandshakeFailure::Broken(VoiceError::Config(format!(
"unexpected mic helper header: {text:?}"
)))),
};
}
Ok(_) => line.push(byte[0]),
Err(e) => {
return Err(HandshakeFailure::Broken(VoiceError::Config(format!(
"read mic helper header: {e}"
))));
}
}
}
Err(HandshakeFailure::Broken(VoiceError::Config(
"oversized mic helper header".into(),
)))
}
/// Kill + reap a handshake-failed child, then join its reader (the kill
/// closes stdout, so a blocked header read returns EOF and the join
/// completes).
fn teardown(mut child: Child, reader: JoinHandle<()>) {
let _ = child.kill();
let _ = child.wait();
let _ = reader.join();
}
/// Run the handshake with a deadline: a reader thread does the blocking read
/// and sends the outcome (plus the stdout, for the PCM stream that follows)
/// over a channel. On failure the child is killed, reaped, and joined.
/// `timeout_what` names the operation in the timeout error (capture vs
/// device-info).
fn handshake(
child: Child,
mut stdout: ChildStdout,
timeout_what: &str,
) -> Result<(Child, String, ChildStdout), HandshakeFailure> {
type Outcome = (Result<String, HandshakeFailure>, ChildStdout);
let (tx, rx) = std::sync::mpsc::sync_channel::<Outcome>(1);
let reader = thread::spawn(move || {
let outcome = read_header(&mut stdout);
let _ = tx.send((outcome, stdout));
});
// A result that lands just as the timeout fires must not be discarded as
// a timeout, so the deadline arm re-checks the channel once before
// tearing down.
let outcome = rx
.recv_timeout(READY_TIMEOUT)
.or_else(|_| rx.try_recv())
.map_err(|_| {
HandshakeFailure::Reported(VoiceError::Config(format!(
"{timeout_what} did not start within {}s",
READY_TIMEOUT.as_secs()
)))
});
match outcome {
Ok((Ok(payload), stdout)) => {
let _ = reader.join();
Ok((child, payload, stdout))
}
Ok((Err(failure), _stdout)) => {
teardown(child, reader);
Err(failure)
}
Err(timeout) => {
teardown(child, reader);
Err(timeout)
}
}
}
/// Spawn helper capture; PCM16 LE chunks are forwarded to `pcm_tx`.
///
/// Falls back to in-process cpal capture when the helper cannot run at all
/// (spawn failure or broken protocol). Device/permission errors reported by a
/// working helper — and handshake timeouts, which an in-process retry of the
/// same stuck device would only double — surface as-is.
pub fn spawn_pcm_capture(
sample_rate: u32,
pcm_tx: async_mpsc::Sender<Vec<u8>>,
) -> Result<CaptureHandle, VoiceError> {
if force_inprocess() {
tracing::info!("voice capture forced in-process ({CAPTURE_BACKEND_ENV}=inprocess)");
return super::capture::spawn_pcm_capture(sample_rate, pcm_tx)
.map(CaptureHandle::InProcess);
}
let rate = sample_rate.to_string();
let handshaken = spawn_helper(&["--rate", &rate])
.map_err(HandshakeFailure::Broken)
.and_then(|(child, stdout)| handshake(child, stdout, "voice capture"));
let (child, device, stdout) = match handshaken {
Ok(up) => up,
Err(HandshakeFailure::Broken(e)) => {
tracing::warn!(error = %e, "mic helper unavailable; falling back to in-process capture");
return super::capture::spawn_pcm_capture(sample_rate, pcm_tx)
.map(CaptureHandle::InProcess);
}
Err(HandshakeFailure::Reported(e)) => return Err(e),
};
tracing::info!(
device = %device,
sample_rate,
"voice capture stream (mic helper subprocess)"
);
let stop = Arc::new(AtomicBool::new(false));
let stop_reader = Arc::clone(&stop);
let reader =
thread::spawn(move || pipe::forward_pcm(stdout, pcm_tx, stop_reader, "mic-helper"));
Ok(CaptureHandle::Child(ChildCaptureHandle::new(
child, stop, reader,
)))
}
/// Default input device via the helper (`--device-info`), so `/doctor` in the
/// long-lived TUI doesn't pay the permanent in-process CoreAudio enumeration
/// cost. Falls back to in-process enumeration when the helper cannot run.
pub fn input_device_info() -> Result<crate::probe::InputDeviceInfo, VoiceError> {
if force_inprocess() {
return super::capture::input_device_info();
}
let handshaken = spawn_helper(&["--device-info"])
.map_err(HandshakeFailure::Broken)
.and_then(|(child, stdout)| handshake(child, stdout, "mic device lookup"));
let payload = match handshaken {
Ok((mut child, payload, _stdout)) => {
// Info mode: the child prints its one line and exits on its own.
// Kill defensively before reaping (a no-op when already exited) so
// a confused child that streams PCM can never wedge the `wait`.
let _ = child.kill();
let _ = child.wait();
payload
}
Err(HandshakeFailure::Broken(e)) => {
tracing::debug!(error = %e, "mic helper unavailable; enumerating in-process");
return super::capture::input_device_info();
}
Err(HandshakeFailure::Reported(e)) => return Err(e),
};
let (name, detail) = payload
.split_once(protocol::INFO_FIELD_SEPARATOR)
.unwrap_or((payload.as_str(), ""));
Ok(crate::probe::InputDeviceInfo {
name: name.to_string(),
detail: detail.to_string(),
})
}
#[cfg(test)]
mod tests {
use super::*;
fn header(bytes: &[u8]) -> Result<String, HandshakeFailure> {
read_header(&mut std::io::Cursor::new(bytes.to_vec()))
}
#[test]
fn header_parses_ready_info_and_err() {
let mut ok = std::io::Cursor::new(b"READY Built-in Microphone\nPCM".to_vec());
assert_eq!(read_header(&mut ok).unwrap(), "Built-in Microphone");
// The PCM byte after the newline must remain unread.
let mut rest = Vec::new();
ok.read_to_end(&mut rest).unwrap();
assert_eq!(rest, b"PCM");
assert_eq!(
header(b"INFO Mic\t44100 Hz, 1 ch\n").unwrap(),
"Mic\t44100 Hz, 1 ch"
);
match header(b"ERR no default input audio device\n") {
Err(HandshakeFailure::Reported(VoiceError::Config(msg))) => {
assert_eq!(msg, "no default input audio device");
}
other => panic!("expected Reported, got {other:?}"),
}
}
#[test]
fn header_treats_eof_garbage_and_oversize_as_broken() {
for bytes in [
b"".as_slice(), // EOF before any header
b"bogus header\n".as_slice(), // unknown tag
&[b'x'; 5000], // no newline within the cap
] {
assert!(
matches!(header(bytes), Err(HandshakeFailure::Broken(_))),
"input {:?}... must be Broken",
&bytes[..bytes.len().min(12)]
);
}
}
/// Drive `handshake` against real scripted children, covering the
/// concurrent recv/teardown paths that the pure header tests cannot:
/// success (with the stdout handed back intact), a reported error, a
/// protocol-broken child, and a child that never answers (timeout).
#[test]
fn handshake_resolves_scripted_children() {
let spawn_sh = |script: &str| -> (Child, ChildStdout) {
let mut cmd = std::process::Command::new("sh");
cmd.arg("-c")
.arg(script)
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::null());
xai_tty_utils::detach_std_command(&mut cmd);
let mut child = cmd.spawn().expect("spawn sh");
let stdout = child.stdout.take().expect("stdout");
(child, stdout)
};
// READY → payload plus the byte stream after the header, unconsumed.
let (child, stdout) = spawn_sh("printf 'READY fake-mic\\nPCM'; sleep 5");
let (mut child, payload, mut stdout) =
handshake(child, stdout, "test").expect("ready handshake");
assert_eq!(payload, "fake-mic");
let mut pcm = [0u8; 3];
stdout.read_exact(&mut pcm).expect("post-header bytes");
assert_eq!(&pcm, b"PCM");
let _ = child.kill();
let _ = child.wait();
// ERR → Reported, child reaped by handshake.
let (child, stdout) = spawn_sh("printf 'ERR no such device\\n'");
match handshake(child, stdout, "test") {
Err(HandshakeFailure::Reported(VoiceError::Config(msg))) => {
assert_eq!(msg, "no such device");
}
Ok(_) => panic!("expected Reported, got READY"),
Err(other) => panic!("expected Reported, got {other:?}"),
}
// Garbage → Broken (the in-process fallback trigger).
let (child, stdout) = spawn_sh("printf 'not-a-header\\n'");
assert!(matches!(
handshake(child, stdout, "test"),
Err(HandshakeFailure::Broken(_))
));
}
/// A child that produces no header within the deadline is killed and the
/// timeout surfaces as `Reported`, naming the caller's operation. Costs a
/// full `READY_TIMEOUT` (5 s), so it is ignored by default.
#[test]
#[ignore = "takes READY_TIMEOUT (5s); run with --ignored"]
fn handshake_times_out_on_silent_child() {
let mut cmd = std::process::Command::new("sh");
cmd.arg("-c")
.arg("sleep 30")
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::null());
xai_tty_utils::detach_std_command(&mut cmd);
let mut child = cmd.spawn().expect("spawn sh");
let stdout = child.stdout.take().expect("stdout");
match handshake(child, stdout, "test capture") {
Err(HandshakeFailure::Reported(VoiceError::Config(msg))) => {
assert!(msg.contains("test capture"), "{msg}");
assert!(msg.contains("did not start"), "{msg}");
}
Ok(_) => panic!("expected timeout, got READY"),
Err(other) => panic!("expected timeout Reported, got {other:?}"),
}
}
}

47
crates/codegen/xai-grok-voice/src/audio/mod.rs
View file

@ -1,16 +1,49 @@
//! Microphone capture (optional `audio` feature).
//!
//! Two backends share one interface (`spawn_pcm_capture`,
//! `capture_pcm_for_duration`, `CaptureHandle`):
//! - non-Linux (macOS/Windows): `cpal` (coreaudio/wasapi), linked into the binary;
//! - Linux: a subprocess recorder (`pw-record`/`parec`/`arecord`), because the
//! static-musl release binary cannot link `cpal` -> `alsa-sys`. See
//! [`capture_linux`] for the full rationale.
//! Three backends share one interface (`spawn_pcm_capture`,
//! `capture_pcm_for_duration`, `input_device_info`, `CaptureHandle`):
//!
//! - **Linux**: a subprocess recorder (`pw-record`/`parec`/`arecord`) — the
//! static-musl release binary cannot link `cpal` → `alsa-sys`; see
//! [`capture_linux`].
//! - **macOS**: a subprocess too — the self-exec `__mic-capture` helper —
//! because in-process CoreAudio memory is never returned after the stream
//! drops; see [`capture_subprocess`].
//! - **Windows**: `cpal` (WASAPI) in-process; its memory cost is modest.
//!
//! The fixed-duration probe capture stays in-process on macOS/Windows: it only
//! runs in short-lived diagnostic processes, where the memory dies at exit.
//!
//! `CaptureHandle` is deliberately one name per platform, resolved by the
//! re-exports below:
//! - Linux → `pipe::ChildCaptureHandle` (recorder subprocess);
//! - macOS → `capture_subprocess::CaptureHandle`, an enum over the helper
//! subprocess and the in-process fallback;
//! - Windows → `capture::CaptureHandle` (in-process cpal stream).
// cpal-based capture: the Windows backend, the macOS fallback, and the macOS
// `__mic-capture` child implementation.
#[cfg(not(target_os = "linux"))]
mod capture;
// Wire protocol shared by the `__mic-capture` child (writer, in `capture`)
// and the macOS parent (parser, in `capture_subprocess`).
#[cfg(not(target_os = "linux"))]
pub use capture::{CaptureHandle, capture_pcm_for_duration, input_device_info, spawn_pcm_capture};
mod protocol;
#[cfg(not(target_os = "linux"))]
pub use capture::capture_pcm_for_duration;
#[cfg(not(target_os = "linux"))]
pub(crate) use capture::run_capture_child_cli;
#[cfg(target_os = "windows")]
pub use capture::{CaptureHandle, input_device_info, spawn_pcm_capture};
// Shared PCM-over-pipe plumbing for the two subprocess backends.
#[cfg(any(target_os = "linux", target_os = "macos"))]
mod pipe;
#[cfg(target_os = "macos")]
mod capture_subprocess;
#[cfg(target_os = "macos")]
pub use capture_subprocess::{CaptureHandle, input_device_info, spawn_pcm_capture};
#[cfg(target_os = "linux")]
mod capture_linux;

169
crates/codegen/xai-grok-voice/src/audio/pipe.rs Normal file
View file

@ -0,0 +1,169 @@
//! Shared PCM-over-pipe plumbing for the subprocess capture backends
//! (Linux system recorder, macOS `__mic-capture` helper): the capture child's
//! stop handle, a reader-thread loop that forwards the child's stdout to the
//! async STT sender, and a stderr drain.
use std::io::Read;
use std::process::Child;
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use std::thread::{self, JoinHandle};
use tokio::sync::mpsc as async_mpsc;
/// Stop handle for a capture child process (recorder or self-exec helper) and
/// its PCM reader thread.
pub struct ChildCaptureHandle {
/// `Some` until `stop()` or `Drop` consumes it (kill + reap).
child: Option<Child>,
stop: Arc<AtomicBool>,
reader: Option<JoinHandle<()>>,
}
impl ChildCaptureHandle {
pub(super) fn new(child: Child, stop: Arc<AtomicBool>, reader: JoinHandle<()>) -> Self {
Self {
child: Some(child),
stop,
reader: Some(reader),
}
}
/// Stop capture: kill the child, reap it, and join the reader thread so
/// the input device is released before returning.
pub fn stop(mut self) {
self.stop.store(true, Ordering::Release);
if let Some(mut child) = self.child.take() {
let _ = child.kill();
let _ = child.wait();
}
if let Some(reader) = self.reader.take() {
let _ = reader.join();
}
}
}
impl Drop for ChildCaptureHandle {
fn drop(&mut self) {
// Always kill the child so the mic is released even when `stop()` was
// never called (e.g. the STT session ended on its own). Killing closes
// the child's stdout, so the reader thread's blocking `read` returns 0
// and it exits. `Drop` must never block (it may run on an async
// executor), so the reap happens on a detached thread — without it,
// drop-path teardowns would leave zombies until the pager exits.
self.stop.store(true, Ordering::Release);
if let Some(mut child) = self.child.take() {
let _ = child.kill();
// `Builder::spawn` so spawn failure under thread exhaustion
// degrades to kill-without-reap instead of a panicking `Drop`.
let _ = thread::Builder::new()
.name("voice-capture-reap".into())
.spawn(move || {
let _ = child.wait();
});
}
}
}
/// PCM read size from the child's stdout (bytes) — ~64 ms at 16 kHz mono
/// PCM16. Small enough to stream responsively, large enough to avoid syscall
/// churn on the reader thread.
pub(super) const READ_CHUNK: usize = 2048;
/// Forward raw PCM from the child's stdout to the async STT sender until the
/// child stops (EOF on kill), the consumer goes away, or `stop` is set.
/// Generic over the reader for tests; production passes the child's stdout.
pub(super) fn forward_pcm(
mut stdout: impl Read,
pcm_tx: async_mpsc::Sender<Vec<u8>>,
stop: Arc<AtomicBool>,
device: &'static str,
) {
let mut buf = vec![0u8; READ_CHUNK];
let mut dropped = 0u64;
loop {
if stop.load(Ordering::Acquire) {
break;
}
match stdout.read(&mut buf) {
// EOF: the child closed stdout (killed by teardown or exited).
Ok(0) => break,
Ok(n) => {
// Never park this thread on the channel: `stop()` joins it, so
// a send that waits on a stalled STT consumer would turn
// teardown into a hang. Shed load instead. (`read` itself is
// unblocked by the kill-on-stop path: killing the child closes
// stdout, so a waiting `read` returns 0.)
match pcm_tx.try_send(buf[..n].to_vec()) {
Ok(()) => {}
Err(async_mpsc::error::TrySendError::Full(_)) => dropped += 1,
// Consumer is gone: the session ended; stop capturing.
Err(async_mpsc::error::TrySendError::Closed(_)) => break,
}
}
Err(e) => {
tracing::warn!(device, error = %e, "voice capture read error");
break;
}
}
}
if dropped > 0 {
tracing::warn!(
device,
dropped,
"voice capture dropped PCM chunks (slow consumer)"
);
}
}
/// Drain the child's stderr to EOF on a detached thread so a chatty child
/// can't fill the pipe buffer and block its own writes (the hot path never
/// reads stderr). Non-empty output is logged at debug. The thread ends on its
/// own when the child exits, so it is not joined.
pub(super) fn drain_stderr(child: &mut Child, device: &'static str) {
let Some(mut stderr) = child.stderr.take() else {
return;
};
thread::spawn(move || {
let mut buf = String::new();
if stderr.read_to_string(&mut buf).is_ok() {
let msg = buf.trim();
if !msg.is_empty() {
tracing::debug!(device, stderr = msg, "voice capture child stderr");
}
}
});
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn forward_pcm_sheds_when_consumer_is_behind() {
// Two reads into a capacity-1 channel: first forwarded, second shed.
let pcm = vec![7u8; 2 * READ_CHUNK];
let (tx, mut rx) = async_mpsc::channel::<Vec<u8>>(1);
forward_pcm(
std::io::Cursor::new(pcm),
tx,
Arc::new(AtomicBool::new(false)),
"test",
);
assert_eq!(rx.try_recv().expect("first chunk").len(), READ_CHUNK);
assert!(rx.try_recv().is_err(), "second chunk shed (channel full)");
}
#[test]
fn forward_pcm_stops_when_consumer_closes() {
let (tx, rx) = async_mpsc::channel::<Vec<u8>>(1);
drop(rx);
// Endless reader: must exit via the Closed arm, not spin forever.
forward_pcm(
std::io::repeat(0),
tx,
Arc::new(AtomicBool::new(false)),
"test",
);
}
}

62
crates/codegen/xai-grok-voice/src/audio/protocol.rs Normal file
View file

@ -0,0 +1,62 @@
//! Wire protocol between the `__mic-capture` helper child and its parent.
//!
//! One status header line on stdout, then (in capture mode) raw PCM:
//! - `READY <device>` — capture stream open, PCM follows;
//! - `INFO <name>\t<detail>` — device lookup result (no stream);
//! - `ERR <message>` — failure, non-zero exit.
//!
//! The child builds lines with the helpers here and the parent parses with
//! the same tags, so the two sides cannot drift.
/// Capture stream is open; raw PCM follows this line.
pub(super) const READY: &str = "READY";
/// Device lookup result; fields separated by [`INFO_FIELD_SEPARATOR`].
pub(super) const INFO: &str = "INFO";
/// Failure; the payload is the error message.
pub(super) const ERR: &str = "ERR";
/// Separates the device name from its detail in an `INFO` payload.
pub(super) const INFO_FIELD_SEPARATOR: char = '\t';
pub(super) fn ready_line(device: &str) -> String {
format!("{READY} {}", sanitize(device))
}
pub(super) fn info_line(name: &str, detail: &str) -> String {
format!(
"{INFO} {}{INFO_FIELD_SEPARATOR}{}",
sanitize(name),
sanitize(detail)
)
}
pub(super) fn err_line(message: &str) -> String {
format!("{ERR} {}", sanitize(message))
}
/// Header payloads must stay single-line for the parent's line-oriented
/// handshake, and must not contain the `INFO` field separator (a device name
/// with a tab would otherwise bleed into the detail field).
fn sanitize(s: &str) -> String {
s.replace(['\n', '\r', INFO_FIELD_SEPARATOR], " ")
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn lines_carry_tag_and_sanitized_payload() {
assert_eq!(ready_line("Mic\nName"), "READY Mic Name");
assert_eq!(err_line("boom\r"), "ERR boom ");
assert_eq!(info_line("USB Mic", "44100 Hz"), "INFO USB Mic\t44100 Hz");
}
#[test]
fn sanitize_strips_the_info_field_separator() {
// A tab inside a device name must not create a phantom third field.
assert_eq!(
info_line("Evil\tMic", "48000 Hz"),
"INFO Evil Mic\t48000 Hz"
);
}
}

16
crates/codegen/xai-grok-voice/src/bin/voice_probe.rs
View file

@ -11,8 +11,20 @@ use xai_grok_voice::{
StaticVoiceAuth, VoiceConfig, VoiceProbeOptions, format_probe_report, run_streaming_probe,
};
#[tokio::main]
async fn main() -> anyhow::Result<()> {
fn main() -> anyhow::Result<()> {
// Hidden mic-capture helper intercept (macOS): the capture backend
// re-execs the current binary — here, voice-probe itself. Runs before any
// runtime/TLS init so the capture child stays minimal.
if let Some(code) = xai_grok_voice::maybe_run_capture_subprocess() {
std::process::exit(code);
}
tokio::runtime::Builder::new_multi_thread()
.enable_all()
.build()?
.block_on(run())
}
async fn run() -> anyhow::Result<()> {
// Standalone binary: install the process-level rustls provider (the pager
// does this in its own main), or the first TLS/WSS connect panics with
// "Could not automatically determine the process-level CryptoProvider".

6
crates/codegen/xai-grok-voice/src/config.rs
View file

@ -2,6 +2,10 @@ use serde::{Deserialize, Serialize};
use crate::error::VoiceError;
/// Default STT capture rate (Hz). Shared with the `__mic-capture` helper's
/// argv default so parent and child agree when `--rate` is omitted.
pub const DEFAULT_SAMPLE_RATE: u32 = 16_000;
/// Voice settings for the STT transport.
///
/// Prefer **https** `api_base` (same shape as chat). [`Self::stt_ws_url`] derives
@ -34,7 +38,7 @@ impl Default for VoiceConfig {
api_base: "https://api.x.ai".into(),
stt_ws_path: "/v1/stt".into(),
language: "en".into(),
sample_rate: 16_000,
sample_rate: DEFAULT_SAMPLE_RATE,
stt_endpointing_ms: 400,
stt_interim_results: true,
client_identifier: String::new(),

80
crates/codegen/xai-grok-voice/src/lib.rs
View file

@ -2,6 +2,10 @@
//! [`run_voice_pipeline`] task that emits [`VoiceEvent`]s for the pager.
//!
//! Voice is dictation only: mic → streaming STT → transcript into the prompt box.
//!
//! On macOS and Linux the microphone is opened in a short-lived subprocess so
//! the long-lived TUI never pays the platform audio stack's permanent memory
//! cost (see [`audio`] and [`maybe_run_capture_subprocess`]).
#[cfg(feature = "audio")]
pub mod audio;
@ -10,7 +14,6 @@ pub mod config;
pub mod error;
pub mod event;
pub mod language;
pub mod pcm;
pub mod pipeline;
pub mod probe;
pub mod stt;
@ -41,3 +44,78 @@ pub use probe::{
/// is actually installed is reported when a session starts. Consumers gate voice
/// on this so a no-audio build never advertises a mic it can't open.
pub const AUDIO_SUPPORTED: bool = cfg!(feature = "audio");
/// Hidden subcommand consumers re-exec themselves with to capture microphone
/// audio in a short-lived helper process (macOS; see
/// [`audio::capture_subprocess`](audio) for why capture is out of process).
/// Intercepted via [`maybe_run_capture_subprocess`] at the very top of `main`,
/// before any TUI/agent/tokio init, so the child stays minimal.
pub const MIC_CAPTURE_SUBCOMMAND: &str = "__mic-capture";
/// If this process was re-exec'd as the hidden mic-capture helper, run it and
/// return `Some(exit_code)`; otherwise `None` (a normal invocation). Call at
/// the very top of `main` in every binary that links this crate with `audio`
/// (the pager composition root and `voice-probe`), mirroring the pager's
/// mermaid render child intercept.
pub fn maybe_run_capture_subprocess() -> Option<i32> {
let argv: Vec<std::ffi::OsString> = std::env::args_os().collect();
if !is_capture_subcommand(&argv) {
return None;
}
#[cfg(all(feature = "audio", not(target_os = "linux")))]
{
// Skip argv[0] (binary) and argv[1] (subcommand); the rest are flags.
let args: Vec<String> = argv
.into_iter()
.skip(2)
.map(|a| a.to_string_lossy().into_owned())
.collect();
Some(audio::run_capture_child_cli(args))
}
#[cfg(not(all(feature = "audio", not(target_os = "linux"))))]
{
// Never spawned by this build's own parent backend (Linux uses system
// recorders; no-audio builds have no capture). Reachable only by hand.
// `write!` not `println!`: never panic on a closed pipe.
use std::io::Write;
let _ = writeln!(
std::io::stdout(),
"ERR mic-capture helper unavailable in this build"
);
Some(2)
}
}
/// Whether `argv` (the full process argv, incl. argv[0]) invokes the hidden
/// mic-capture helper — i.e. argv[1] is [`MIC_CAPTURE_SUBCOMMAND`]. Pure so the
/// dispatch decision is unit-testable without mutating the process's real args.
fn is_capture_subcommand(argv: &[std::ffi::OsString]) -> bool {
argv.get(1).and_then(|a| a.to_str()) == Some(MIC_CAPTURE_SUBCOMMAND)
}
#[cfg(test)]
mod intercept_tests {
use super::*;
fn argv(items: &[&str]) -> Vec<std::ffi::OsString> {
items.iter().map(std::ffi::OsString::from).collect()
}
#[test]
fn capture_subcommand_matches_only_argv1() {
assert!(is_capture_subcommand(&argv(&["grok", "__mic-capture"])));
assert!(is_capture_subcommand(&argv(&[
"grok",
"__mic-capture",
"--rate",
"16000"
])));
assert!(!is_capture_subcommand(&argv(&["grok"])));
assert!(!is_capture_subcommand(&argv(&["grok", "chat"])));
assert!(!is_capture_subcommand(&argv(&[
"grok",
"chat",
"__mic-capture"
])));
}
}

63
crates/codegen/xai-grok-voice/src/pcm.rs
View file

@ -1,63 +0,0 @@
//! PCM16 peak-level helpers for silence detection.
//!
//! Denied mic permission (macOS feeds unauthorized apps zeros), muted input, or
//! a dead device yields ~zero peak. A working mic still has a noise floor, so
//! peak separates "mic misconfigured" from "user didn't speak".
/// Peaks at or below this many PCM16 counts count as digital silence.
/// Small allowance for dither on an otherwise dead input; far below a real
/// mic's noise floor.
pub const SILENCE_PEAK_MAX: u16 = 3;
/// Peak absolute sample of little-endian mono PCM16. Empty → 0; trailing odd
/// byte ignored. `u16` because `i16::MIN.unsigned_abs()` is 32768.
pub fn peak_abs_i16_le(pcm_le: &[u8]) -> u16 {
pcm_le
.chunks_exact(2)
.map(|b| i16::from_le_bytes([b[0], b[1]]).unsigned_abs())
.max()
.unwrap_or(0)
}
/// [`peak_abs_i16_le`] for samples not yet encoded as bytes.
pub fn peak_abs_i16(samples: &[i16]) -> u16 {
samples.iter().map(|s| s.unsigned_abs()).max().unwrap_or(0)
}
/// Whether a peak is digital silence (see [`SILENCE_PEAK_MAX`]).
pub fn is_silence(peak: u16) -> bool {
peak <= SILENCE_PEAK_MAX
}
#[cfg(test)]
mod tests {
use super::*;
fn pcm(samples: &[i16]) -> Vec<u8> {
samples.iter().flat_map(|s| s.to_le_bytes()).collect()
}
#[test]
fn peak_abs_i16_le_edges() {
assert_eq!(peak_abs_i16_le(&[]), 0);
assert_eq!(peak_abs_i16_le(&pcm(&[10, -500, 300])), 500);
assert_eq!(peak_abs_i16_le(&pcm(&[i16::MIN])), 32768);
let mut bytes = pcm(&[7]);
bytes.push(0xFF);
assert_eq!(peak_abs_i16_le(&bytes), 7);
}
#[test]
fn peak_abs_i16_edges() {
assert_eq!(peak_abs_i16(&[]), 0);
assert_eq!(peak_abs_i16(&[10, -500, 300]), 500);
assert_eq!(peak_abs_i16(&[i16::MIN]), 32768);
}
#[test]
fn silence_threshold_boundary() {
assert!(is_silence(0));
assert!(is_silence(SILENCE_PEAK_MAX));
assert!(!is_silence(SILENCE_PEAK_MAX + 1));
}
}

79
crates/codegen/xai-grok-voice/src/pipeline.rs
View file

@ -7,8 +7,6 @@
#[cfg(feature = "audio")]
use std::collections::VecDeque;
#[cfg(feature = "audio")]
use std::sync::atomic::Ordering;
use tokio::sync::mpsc;
use tokio::task::JoinHandle;
@ -96,7 +94,7 @@ pub async fn run_voice_pipeline(
};
// The reader task owns the capture handle; signalling it lets the
// reader stop the mic and send `audio.done` in a single place,
// matching the silence-guard teardown below.
// matching the no-speech-watchdog teardown below.
let _ = session.finish_tx.send(()).await;
}
}
@ -193,30 +191,21 @@ async fn forward_pcm(
}
}
/// Silence → short toast (+ long OS hint); non-silence → "try again".
/// Toast may be the only surface (dashboard / `--minimal`), so on macOS it
/// includes grant + restart — the long hint has the full Settings path.
/// How long a session may run without any transcript before it is torn down
/// (instead of streaming a dead mic until the user gives up). Disarmed by the
/// first transcript, so long dictation with pauses is unaffected.
#[cfg(feature = "audio")]
fn silence_guard_error(peak: u16) -> (String, Option<String>) {
if crate::pcm::is_silence(peak) {
let message = if cfg!(target_os = "macos") {
"microphone delivered only silence — allow terminal mic access, then restart it"
} else {
"microphone delivered only silence — check mic permission"
};
(
message.to_string(),
Some(format!(
"To fix voice dictation, {}",
crate::probe::mic_silence_help()
)),
)
} else {
(
"heard audio but no speech was detected — try again".to_string(),
None,
)
}
const NO_SPEECH_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(10);
/// Message and permission guidance for a session torn down by
/// [`NO_SPEECH_TIMEOUT`]. A denied grant is indistinguishable from not speaking
/// because macOS may return silence instead of an error.
#[cfg(feature = "audio")]
fn no_speech_error() -> (String, Option<String>) {
(
"No speech was detected. Voice stopped.".to_owned(),
Some(crate::probe::mic_fix_help().to_owned()),
)
}
#[cfg(feature = "audio")]
@ -256,7 +245,6 @@ async fn start_capture_session(
)));
}
};
let peak = capture.peak_meter();
let mut stt = connect_res?;
// Hand the live sender to the forwarder; it flushes the backlog then streams.
@ -278,9 +266,9 @@ async fn start_capture_session(
handle.stop();
}
};
// No transcript in first 10s → diagnose from peak. Disarmed after speech.
let silence_deadline = tokio::time::Instant::now() + std::time::Duration::from_secs(10);
let mut silence_check = true;
// No transcript within the timeout → tear down. Disarmed after speech.
let no_speech_deadline = tokio::time::Instant::now() + NO_SPEECH_TIMEOUT;
let mut awaiting_speech = true;
// Chunk-final (`is_final && !speech_final`) text is locked: the server
// sends it as a delta of the turn. Stitch those deltas into the live
// preview so a long pauseless utterance keeps accumulating on screen
@ -293,19 +281,19 @@ async fn start_capture_session(
tokio::select! {
msg = finish_rx.recv() => {
if msg.is_some() {
// User ended the turn; stop watching for initial silence.
silence_check = false;
// User ended the turn; stop the no-speech watchdog.
awaiting_speech = false;
stop_capture(&mut capture);
stt.finish_audio();
} else {
return;
}
}
_ = tokio::time::sleep_until(silence_deadline), if silence_check => {
// Tear down rather than streaming silence until the user stops.
_ = tokio::time::sleep_until(no_speech_deadline), if awaiting_speech => {
// Tear down rather than streaming a dead mic until the user stops.
stop_capture(&mut capture);
stt.finish_audio();
let (message, hint) = silence_guard_error(peak.load(Ordering::Relaxed));
let (message, hint) = no_speech_error();
let _ = out.send(VoiceEvent::Error { message, hint }).await;
return;
}
@ -316,8 +304,8 @@ async fn start_capture_session(
if text.is_empty() {
continue;
}
// Real speech arrived: disarm the initial-silence guard.
silence_check = false;
// Real speech arrived: disarm the no-speech watchdog.
awaiting_speech = false;
let event = if p.speech_final {
locked_prefix.clear();
@ -348,7 +336,7 @@ async fn start_capture_session(
Some(StreamingSttEvent::Done { text }) => {
locked_prefix.clear();
if !text.trim().is_empty() {
silence_check = false;
awaiting_speech = false;
let _ = out.send(VoiceEvent::UtteranceFinal { text }).await;
}
}
@ -422,16 +410,9 @@ mod tests {
}
#[test]
fn silence_guard_error_matches_metered_level() {
let (message, hint) = silence_guard_error(0);
assert!(message.contains("only silence"));
if cfg!(target_os = "macos") {
assert!(message.contains("restart"), "{message}");
}
assert!(hint.is_some_and(|h| h.contains(crate::probe::mic_silence_help())));
let (message, hint) = silence_guard_error(2_000);
assert!(message.contains("no speech was detected"));
assert!(hint.is_none());
fn no_speech_error_carries_permission_hint() {
let (message, hint) = no_speech_error();
assert_eq!(message, "No speech was detected. Voice stopped.");
assert!(hint.is_some_and(|hint| hint.contains(crate::probe::mic_fix_help())));
}
}

13
crates/codegen/xai-grok-voice/src/probe.rs
View file

@ -152,14 +152,13 @@ pub fn input_device_info() -> Result<InputDeviceInfo, VoiceError> {
))
}
/// Platform-specific fix text for a silent mic. On macOS the grant is for the
/// terminal app and only applies after that app restarts.
pub fn mic_silence_help() -> &'static str {
/// Platform-specific fix text for a mic that isn't being picked up. On macOS
/// the grant is for the terminal app and only applies after that app restarts.
pub fn mic_fix_help() -> &'static str {
if cfg!(target_os = "macos") {
"grant your terminal app microphone access in System Settings → \
Privacy & Security → Microphone, then restart the terminal. If it's \
already allowed, check the input device and level in System Settings \
→ Sound → Input."
"Allow microphone access for your terminal in System Settings → Privacy & Security → \
Microphone, then restart the terminal. If access is already on, check the input device \
and level in System Settings → Sound → Input."
} else if cfg!(target_os = "windows") {
"allow microphone access in Settings → Privacy & security → \
Microphone, and check the input device and level in Settings → \