Publish harness and TUI open-source

initial sync from the monorepo

crates/codegen/xai-grok-test-support/Cargo.toml

@@ -0,0 +1,36 @@
+[package]
+license = "Apache-2.0"
+name = "xai-grok-test-support"
+version = "0.1.0"
+edition.workspace = true
+description = "Shared test-support for grok-build crates: mock inference server, SSE generators, ACP stdio client, headless runner, env sandbox"
+publish = false
+
+[dependencies]
+agent-client-protocol = { workspace = true }
+anyhow = { workspace = true }
+async-trait = { workspace = true }
+axum = { workspace = true }
+clap = { workspace = true }
+futures-util = { workspace = true }
+serde = { workspace = true }
+serde_json = { workspace = true }
+tempfile = { workspace = true }
+tokio = { workspace = true }
+tokio-tungstenite = { workspace = true }
+tokio-util = { workspace = true, features = ["compat"] }
+tracing = { workspace = true }
+tracing-subscriber = { workspace = true, features = ["fmt"] }
+xai-acp-lib = { workspace = true }
+
+[dev-dependencies]
+reqwest = { workspace = true }
+[features]
+default = []
+default-bazel = []
+
+[target.'cfg(unix)'.dependencies]
+libc = { workspace = true }
+
+[lints]
+workspace = true

crates/codegen/xai-grok-test-support/README.md

@@ -0,0 +1,64 @@
+# xai-grok-test-support
+
+Shared test infrastructure for the grok-build crates: mock inference server,
+SSE wire-format generators, ACP stdio clients, headless
+runner, and sandboxed process env. Consumed by `xai-grok-shell` integration
+tests, `xai-grok-pager-pty-harness` (`ContentController`), and `xai-grok-sampler`
+tests.
+
+> **Freshness rule:** update this README in the same PR that changes `src/` —
+> reviewers should treat a `src/` diff without a README diff as incomplete.
+
+How-to-test discovery lives with the pager PTY harness crate
+(`xai-grok-pager-pty-harness`). This file is the API reference for the shared
+test-support surface.
+
+## Module map
+
+| Module | What it provides |
+|--------|------------------|
+| `mock_server` | `MockInferenceServer` — `/v1/chat/completions`, `/v1/responses`, `/v1/messages`, `/v1/models`, `/v1/settings`, `/v1/user` on `127.0.0.1:0`. `/v1/models` entries are `MockModelEntry` (re-exported as `MockModel` for PTY tests): `new(id)` / `with_agent_type(id, ty)` plus chainable `with_api_backend`, `with_supports_backend_search(bool)` → `supportsBackendSearch`, `with_supports_reasoning_effort(bool)` → `supportsReasoningEffort`, `with_reasoning_effort(&str)` → `reasoningEffort`, `with_reasoning_efforts(Vec<Value>)` → `reasoningEfforts` (raw option tables/bare strings), all emitted top-level as `parse_remote_model_value` reads them. Three response modes for the inference endpoints, precedence **scripted > required-auth > mode**: (1) **echo** (default) streams `Echo: <last user message>`, whitespace-collapsing; (2) **fixed** via `set_response(text)`, byte-exact delta reconstruction (newlines preserved — fenced code blocks survive); (3) **scripted** via `enqueue_response`. Constructors (`start`, `start_with_models`, `start_with_required_auth`) return `anyhow::Result`. Settings are 404-until-set (`set_settings(impl Serialize)`, `preset_allow_access()` for the `{"allow_access": true}` gate); scripted `/v1/settings` one-shots (`enqueue_response`) take precedence over the steady-state value (stale-snapshot tests). `/v1/user` serves a minimal `UserInfo` whose `subscriptionTier` is controlled by `set_user_subscription_tier(Option<&str>)` (`None` = free); its log entries keep the query string (e.g. `/v1/user?include=subscription`) so subscription-check cadence is countable. Request log: `requests()` (`LogEntry` with body, `authorization`, full POST headers + `header(name)` accessor), `request_bodies()`, `request_count()`, `has_chat_completion_request()` / `has_responses_request()` (exact, per endpoint), `messages_request_count()`, `last_system_prompt()`, `request_log_summary()`. **Storage:** `POST /v1/storage` with flippable 401 (`set_storage_unauthorized`); accepted uploads via `storage_uploads()` → `StorageUpload { path, size, body, authorization }` (`body` retained up to 256 KiB, empty above; `authorization` is the raw header). Runtime knobs: `set_models`, `set_messages_stop_reason`. Shuts down on drop. |
+| `scripted` | Data-only scripting (no axum types in the surface): `SseEvent { event, data }` (`::data`, `::with_event`), `ScriptedBody::{Json, Sse, Raw}` (`Raw` = byte-controllable malformed SSE), `ScriptedResponse { status, headers, body }` (`::sse`, `::json`, `::text`). `enqueue_response(path, response)` queues FIFO **per path**; consumed by the three inference endpoints, falling back to the active mode when empty. Scripted SSE bodies honor the server's `set_chunk_delay` pacing, same as the echo/fixed modes. Validation is eager — bad status/header panics at the enqueue call site. Seed of the phase-2 scriptable-mock format. |
+| `sse` | The three wire formats as event-list builders: `chat_completion_events` / `responses_api_events` / `messages_api_events(text, model, stop_reason)` (echo-style, whitespace-collapsing) plus byte-exact variants `chat_completion_events_exact` / `responses_api_events_exact` (messages is single-delta, byte-exact by construction). The exact/echo split is load-bearing — see the in-module byte-exactness tests. Also the scripted-scenario builders returning `SseEvent`s (for `ScriptedResponse::sse`): `responses_api_reasoning_only_events(reasoning, model)` — reasoning summary deltas completing with a `reasoning` item but no message/output-text, so the shell collector classifies the turn `EmptyReason::ReasoningOnly` (the model-doomloop trigger); `responses_api_reasoning_and_text_events(reasoning, text, model)` — reasoning deltas then a normal text answer (the ordinary reasoning-model turn); `responses_api_reasoning_then_tool_call_events(reasoning, call_id, name, arguments, model)` + its Chat Completions twin `chat_completions_reasoning_then_tool_call_events(...)` — reasoning deltas then one tool call (the think-then-call turn whose tool call finishes the thought and keeps the turn non-empty); the doom-loop check trio: `responses_api_doom_loop_check_events(triggers, reasoning, model)` — a doomed reasoning-only turn with NAMED `response.doom_loop_check` frames re-sent per cumulative prefix of `triggers` plus the terminal `doom_loop_check.triggers` copy on `response.completed`, `responses_api_doom_loop_terminal_only_events(triggers, reasoning, text, model)` — a normal answer whose terminal response alone carries the field, and `responses_api_with_doom_loop_frame(check_frame_data, reasoning, text, model)` — splices one named check frame with a caller-supplied payload (byte-exact `xai_grok_sampling_types::doom_loop::SAMPLE_CHECK_EVENT_DATA{,_CUMULATIVE}` fixtures or malformed variants) into an ordinary turn. |
+| `acp_client` | `GrokStdioClient` — drives `grok agent stdio` over real pipes through `agent-client-protocol`: spawn variants (`spawn`, `spawn_with_home`, `spawn_with_home_and_env`, `spawn_with_home_env_and_args`), initialize/authenticate, session create/load, prompt, `*_with_timeout` wrappers, captured text + stderr. `RawStdioClient` — raw-wire sibling for bytes the typed `ClientSideConnection` can never produce (escaped-slash methods `"session\/prompt"`, string UUID ids — the Xcode/Foundation shape): `send_line` writes a line verbatim; `response_for_id` matches the response by exact string id (the match IS the id-echo assertion), skips notifications, auto-refuses agent→client requests with `-32601`, and panics on timeout with skipped-traffic diagnostics (count + last lines; `0 other messages` = true silence). Both spawn through one hermetic `spawn_agent_process` (sandbox env + debug-log kill-list exists once) atop `process::spawn_piped_with_stderr_capture` (crate-internal `process` module: pipes, `kill_on_drop`, stderr drain — also used by `leader::LeaderStdioClient`). |
+| `headless` | `run_headless(server, args, cwd)` / `run_headless_with_cmd(cmd)` → `HeadlessResult { status, stdout, stderr, timed_out }` (60s cap), `assert_headless_success`, `assert_no_crashes` (panic/SIGSEGV/linker patterns), `stderr_tail`. |
+| `env` | `grok_binary()` (`GROK_BINARY` env → `CARGO_BIN_EXE` → local debug build of `xai-grok-pager`), `git_workdir()` (temp git repo, forces full libgit2 init), `test_env_cmd_tokio(cmd, mock_url, home)` (sandboxed HOME **and GROK_HOME** — Windows resolves `~` via USERPROFILE, so HOME alone doesn't sandbox — + mock endpoints + telemetry kill-switches). |
+| `leader` | Unix-only `LeaderStdioClient` (`grok agent --leader stdio`, `env_clear`-hermetic, sandboxed `GROK_LEADER_SOCKET`; `spawn_with_binary` runs an explicit binary for version-skew lanes, per-role resolution via `leader_binary()` / `client_binary()` honoring `GROK_BINARY_LEADER` / `GROK_BINARY_CLIENT`) + lock-file helpers: `leader_lock_path`, `read_leader_pid`, `pid_alive`, `wait_for_live_leader`, `wait_for_new_leader`, `wait_for_replay_notifications`, `leader_log`. |
+| `uds_proxy` | Unix-only `UdsProxy` — frame-aware (4-byte BE length prefix) man-in-the-middle for leader IPC sockets. `UdsProxy::spawn(proxy_path, upstream_path, FaultPlan)`; `FaultPlan { direction, drop_frame, sever_mid_frame, delay, duplicate_frame }` (1-based frame index, per connection per direction); runtime `FaultHandle::sever_now()` + `forwarded(direction)` counters; frame bodies capped at 64 MiB (leader-transport parity — corrupt lengths error instead of allocating). Zero production changes: point `LeaderClient::connect` / `GROK_LEADER_SOCKET` at the proxy path. |
+
+## Consumer matrix
+
+| Consumer | Uses | Notes |
+|----------|------|-------|
+| `xai-grok-shell` `tests/*.rs` | Everything | Direct imports (`use xai_grok_test_support::*` or module paths); no local shim. |
+| `xai-grok-pager-pty-harness` `src/content.rs` | `MockInferenceServer`, `MockModelEntry` (re-exported as `MockModel`) | `ContentController` wraps the server and **keeps the HOME-sandbox `TempDir` + `env_for_pager()` harness-side**; presets `allow_access` + a fixed default response at construction. |
+| `xai-grok-sampler` `tests/test_actor.rs` | `sse` generators | Happy-path payloads only; the actor keeps its own router for stall/conditional fixtures. |
+
+## Adding a capability
+
+**A response mode** (`mock_server.rs`): extend the private `ResponseMode` enum
++ add the setter; wire the new arm into all **three** inference handlers (the
+match in each route); scripted responses must still win. Extend the in-crate
+tests: an HTTP round-trip for the new mode plus a leg in
+`scripted_responses_serve_fifo_per_path_then_fall_back` proving fallback
+reaches it. The echo pinning test (`echo_mode_echoes_last_user_message`) must
+pass unmodified — echo bytes are frozen.
+
+**A wire format** (`sse.rs`): add the echo-style builder and, if clients
+reconstruct text byte-for-byte, an `_exact` variant built on a delta fn;
+then add the serving arm in `mock_server` (all modes) and a route if it is a
+new endpoint. Extend the byte-exactness pins
+(`deltas_reconstruct_multiline_response_byte_for_byte`,
+`deltas_preserve_runs_of_whitespace`) — they are the contract that fenced
+code blocks (mermaid) survive streaming. A **scripted-scenario builder** (one
+that models a specific completion the echo/fixed modes can't express, e.g.
+`responses_api_reasoning_only_events`) instead returns `SseEvent`s for
+`ScriptedResponse::sse`, needs no `mock_server` mode wiring, and ships with an
+in-module shape test asserting its event shape.
+
+**A scripted-body kind** (`scripted.rs`): new `ScriptedBody` variant + render
+arm in `into_response_paced` + eager checks in `validate` if the data can be
+invalid. Add an in-crate test asserting client-visible bytes (the `Raw`
+byte-exactness test is the template) and keep
+`scripted_response_takes_precedence_over_required_auth` green — precedence is
+part of the contract.

crates/codegen/xai-grok-test-support/src/acp_client.rs

@@ -0,0 +1,514 @@
+//! ACP stdio clients for testing grok sessions end-to-end: the typed
+//! [`GrokStdioClient`] (`agent-client-protocol::ClientSideConnection` —
+//! authentication, session lifecycle, permissions, notification streaming) and
+//! the raw-wire [`RawStdioClient`] (verbatim JSON-RPC lines for shapes the
+//! typed client can't produce), plus the shared subprocess spawn/stderr-capture
+//! plumbing used by every harness in this crate.
+
+use std::path::Path;
+use std::sync::Arc;
+use std::sync::atomic::{AtomicU32, Ordering};
+use std::time::Duration;
+
+use agent_client_protocol::{self as acp, Agent as _};
+use tempfile::TempDir;
+use tokio_util::compat::{TokioAsyncReadCompatExt, TokioAsyncWriteCompatExt};
+use xai_acp_lib::LineBufferedRead;
+
+use crate::env::{grok_binary, test_env_cmd_tokio};
+use crate::headless::stderr_tail;
+use crate::mock_server::MockInferenceServer;
+use crate::process::spawn_piped_with_stderr_capture;
+
+/// Spawn `grok agent stdio` with the canonical hermetic test env: the sandbox
+/// from [`test_env_cmd_tokio`] plus the debug-logging kill-list, so the
+/// hermeticity setup exists exactly once for the typed ([`GrokStdioClient`])
+/// and raw ([`RawStdioClient`]) harnesses. `leading_args` go before the
+/// `agent stdio` subcommand (global flags); `extra_env` is applied after the
+/// kill-list so a test can still set e.g. `GROK_DEBUG_LOG=1` explicitly.
+fn spawn_agent_process(
+    server: &MockInferenceServer,
+    cwd: &Path,
+    home: &Path,
+    extra_env: &[(&str, &str)],
+    leading_args: &[&str],
+) -> (tokio::process::Child, Arc<std::sync::Mutex<Vec<u8>>>) {
+    let binary = grok_binary();
+
+    let mut cmd = tokio::process::Command::new(&binary);
+    cmd.args(leading_args)
+        .args(["agent", "stdio"])
+        .current_dir(cwd);
+    test_env_cmd_tokio(&mut cmd, &server.url(), home);
+    // Hermetic firehose env: clear inherited debug-logging knobs so a test
+    // controls logging only via `extra_env` / `leading_args` (mirrors the
+    // headless `debug_cmd`).
+    for k in [
+        "GROK_DEBUG_LOG",
+        "GROK_LOG_FILE",
+        "GROK_LOG_SAMPLING",
+        "GROK_HOOKS_LOG",
+    ] {
+        cmd.env_remove(k);
+    }
+    for (k, v) in extra_env {
+        cmd.env(k, v);
+    }
+
+    spawn_piped_with_stderr_capture(cmd)
+}
+
+#[derive(Default)]
+struct TextCapture {
+    chunks: std::sync::Mutex<Vec<String>>,
+    notification_count: AtomicU32,
+}
+
+/// ACP client impl: auto-approves permissions, captures text chunks.
+struct TestAcpClient {
+    capture: Arc<TextCapture>,
+}
+
+#[async_trait::async_trait(?Send)]
+impl acp::Client for TestAcpClient {
+    async fn request_permission(
+        &self,
+        args: acp::RequestPermissionRequest,
+    ) -> acp::Result<acp::RequestPermissionResponse> {
+        // Auto-approve: pick AllowOnce if available, otherwise first option.
+        let outcome = args
+            .options
+            .iter()
+            .find(|o| o.kind == acp::PermissionOptionKind::AllowOnce)
+            .or(args.options.first())
+            .map(|o| {
+                acp::RequestPermissionOutcome::Selected(acp::SelectedPermissionOutcome::new(
+                    o.option_id.clone(),
+                ))
+            })
+            .unwrap_or(acp::RequestPermissionOutcome::Cancelled);
+
+        Ok(acp::RequestPermissionResponse::new(outcome))
+    }
+
+    async fn session_notification(&self, args: acp::SessionNotification) -> acp::Result<()> {
+        self.capture
+            .notification_count
+            .fetch_add(1, Ordering::SeqCst);
+
+        if let acp::SessionUpdate::AgentMessageChunk(acp::ContentChunk { content, .. }) =
+            args.update
+            && let acp::ContentBlock::Text(text_content) = content
+            && !text_content.text.is_empty()
+        {
+            self.capture.chunks.lock().unwrap().push(text_content.text);
+        }
+        Ok(())
+    }
+}
+
+/// Drives `grok agent stdio` via the ACP protocol over pipes.
+///
+/// Handles the full lifecycle: spawn → initialize → authenticate → session → prompt.
+/// Child process is killed on drop.
+pub struct GrokStdioClient {
+    conn: acp::ClientSideConnection,
+    _child: tokio::process::Child,
+    home: Option<TempDir>,
+    capture: Arc<TextCapture>,
+    stderr: Arc<std::sync::Mutex<Vec<u8>>>,
+}
+
+impl GrokStdioClient {
+    pub async fn spawn(server: &MockInferenceServer, cwd: &Path) -> Self {
+        let home = TempDir::new().expect("create temp home");
+        Self::spawn_with_home(server, cwd, home).await
+    }
+
+    pub async fn spawn_with_home(server: &MockInferenceServer, cwd: &Path, home: TempDir) -> Self {
+        Self::spawn_with_home_and_env(server, cwd, home, &[]).await
+    }
+
+    /// Like [`spawn_with_home`] but applies extra environment variables to the
+    /// child process (after the standard test env). Used by tests that toggle
+    /// behavior via env vars (e.g. the vendor-compat suite).
+    pub async fn spawn_with_home_and_env(
+        server: &MockInferenceServer,
+        cwd: &Path,
+        home: TempDir,
+        extra_env: &[(&str, &str)],
+    ) -> Self {
+        Self::spawn_with_home_env_and_args(server, cwd, home, extra_env, &[]).await
+    }
+
+    /// Like [`spawn_with_home_and_env`] but also prepends `leading_args` before
+    /// the `agent stdio` subcommand. Used to drive top-level global flags (e.g.
+    /// `--debug`) so a test can exercise the flag's master switch, not just env.
+    pub async fn spawn_with_home_env_and_args(
+        server: &MockInferenceServer,
+        cwd: &Path,
+        home: TempDir,
+        extra_env: &[(&str, &str)],
+        leading_args: &[&str],
+    ) -> Self {
+        let (mut child, stderr) =
+            spawn_agent_process(server, cwd, home.path(), extra_env, leading_args);
+
+        let outgoing = child.stdin.take().unwrap().compat_write();
+        let incoming = child.stdout.take().unwrap().compat();
+
+        let capture = Arc::new(TextCapture::default());
+        let client = TestAcpClient {
+            capture: capture.clone(),
+        };
+
+        let incoming = LineBufferedRead::spawn_local(incoming);
+        let (conn, handle_io) = acp::ClientSideConnection::new(client, outgoing, incoming, |fut| {
+            tokio::task::spawn_local(fut);
+        });
+        tokio::task::spawn_local(handle_io);
+
+        Self {
+            conn,
+            _child: child,
+            home: Some(home),
+            capture,
+            stderr,
+        }
+    }
+
+    /// Initialize and authenticate (picks `api_key` auth method).
+    pub async fn initialize(&self) -> acp::InitializeResponse {
+        let init_resp = self
+            .conn
+            .initialize(
+                acp::InitializeRequest::new(acp::ProtocolVersion::V1)
+                    .client_capabilities(
+                        acp::ClientCapabilities::new()
+                            .fs(acp::FileSystemCapabilities::new())
+                            .terminal(false),
+                    )
+                    .meta(
+                        serde_json::json!({
+                            "startupHints": {
+                                "nonInteractive": true,
+                                "skipGitStatus": true,
+                                "skipProjectLayout": true
+                            },
+                            "clientType": "test-client",
+                            "clientVersion": "0.0.0-test"
+                        })
+                        .as_object()
+                        .cloned(),
+                    ),
+            )
+            .await
+            .expect("initialize failed");
+
+        let api_key_method = init_resp
+            .auth_methods
+            .iter()
+            .find(|m| &*m.id().0 == "xai.api_key")
+            .unwrap_or_else(|| {
+                let ids: Vec<_> = init_resp.auth_methods.iter().map(|m| &m.id().0).collect();
+                panic!(
+                    "expected auth method 'xai.api_key' but got: {ids:?}\n\
+                     If the method ID changed, update this test."
+                )
+            });
+
+        self.conn
+            .authenticate(
+                acp::AuthenticateRequest::new(api_key_method.id().clone())
+                    .meta(serde_json::json!({"headless": true}).as_object().cloned()),
+            )
+            .await
+            .expect("authenticate failed");
+
+        init_resp
+    }
+
+    pub async fn create_session(&self, cwd: &Path) -> acp::SessionId {
+        let resp = self
+            .conn
+            .new_session(acp::NewSessionRequest::new(cwd.to_path_buf()).mcp_servers(vec![]))
+            .await
+            .expect("session/new failed");
+        resp.session_id
+    }
+
+    /// Create a session with a specific model pre-selected.
+    pub async fn create_session_with_model(&self, cwd: &Path, model_id: &str) -> acp::SessionId {
+        let resp = self
+            .conn
+            .new_session(
+                acp::NewSessionRequest::new(cwd.to_path_buf())
+                    .mcp_servers(vec![])
+                    .meta(
+                        serde_json::json!({ "modelId": model_id })
+                            .as_object()
+                            .cloned(),
+                    ),
+            )
+            .await
+            .expect("session/new with modelId failed");
+        resp.session_id
+    }
+
+    /// Switch model on an existing session via the typed ACP `session/set_model`.
+    pub async fn set_model(
+        &self,
+        session_id: &acp::SessionId,
+        model_id: &str,
+    ) -> acp::Result<acp::SetSessionModelResponse> {
+        use acp::Agent as _;
+        self.conn
+            .set_session_model(acp::SetSessionModelRequest::new(
+                session_id.clone(),
+                acp::ModelId::new(model_id),
+            ))
+            .await
+    }
+
+    pub async fn prompt(
+        &self,
+        session_id: &acp::SessionId,
+        text: &str,
+    ) -> acp::Result<acp::PromptResponse> {
+        self.conn
+            .prompt(acp::PromptRequest::new(
+                session_id.clone(),
+                vec![acp::ContentBlock::Text(acp::TextContent::new(
+                    text.to_string(),
+                ))],
+            ))
+            .await
+    }
+
+    pub fn captured_text(&self) -> String {
+        self.capture.chunks.lock().unwrap().join("")
+    }
+
+    pub fn notification_count(&self) -> u32 {
+        self.capture.notification_count.load(Ordering::SeqCst)
+    }
+
+    pub fn stderr(&self) -> String {
+        String::from_utf8_lossy(&self.stderr.lock().unwrap()).into_owned()
+    }
+
+    pub fn take_home(&mut self) -> TempDir {
+        self.home.take().expect("test home already taken")
+    }
+
+    /// Return the home directory path (for cache invalidation between phases).
+    pub fn home_path(&self) -> &std::path::Path {
+        self.home.as_ref().expect("test home already taken").path()
+    }
+
+    pub async fn initialize_with_timeout(&self) -> acp::InitializeResponse {
+        tokio::time::timeout(Duration::from_secs(20), self.initialize())
+            .await
+            .unwrap_or_else(|_| panic!("initialize timed out\nstderr:\n{}", self.stderr()))
+    }
+
+    pub async fn create_session_with_timeout(&self, cwd: &Path) -> acp::SessionId {
+        tokio::time::timeout(Duration::from_secs(20), self.create_session(cwd))
+            .await
+            .unwrap_or_else(|_| panic!("session/new timed out\nstderr:\n{}", self.stderr()))
+    }
+
+    pub async fn create_session_with_model_timeout(
+        &self,
+        cwd: &Path,
+        model_id: &str,
+    ) -> acp::SessionId {
+        tokio::time::timeout(
+            Duration::from_secs(20),
+            self.create_session_with_model(cwd, model_id),
+        )
+        .await
+        .unwrap_or_else(|_| {
+            panic!(
+                "session/new with modelId={model_id} timed out\nstderr:\n{}",
+                self.stderr()
+            )
+        })
+    }
+
+    pub async fn set_model_with_timeout(
+        &self,
+        session_id: &acp::SessionId,
+        model_id: &str,
+    ) -> acp::Result<acp::SetSessionModelResponse> {
+        tokio::time::timeout(
+            Duration::from_secs(20),
+            self.set_model(session_id, model_id),
+        )
+        .await
+        .unwrap_or_else(|_| {
+            panic!(
+                "session/set_model({model_id}) timed out\nstderr:\n{}",
+                self.stderr()
+            )
+        })
+    }
+
+    pub async fn prompt_with_timeout(
+        &self,
+        session_id: &acp::SessionId,
+        text: &str,
+    ) -> acp::Result<acp::PromptResponse> {
+        tokio::time::timeout(Duration::from_secs(30), self.prompt(session_id, text))
+            .await
+            .unwrap_or_else(|_| panic!("prompt timed out\nstderr:\n{}", self.stderr()))
+    }
+
+    pub async fn load_session_with_timeout(
+        &self,
+        session_id: &acp::SessionId,
+        cwd: &Path,
+    ) -> acp::LoadSessionResponse {
+        // 60s: session/load replays history and is slower under Rosetta
+        // (macos-x86_64 lifecycle CI). 20s flaked repeatedly there.
+        tokio::time::timeout(
+            Duration::from_secs(60),
+            self.conn.load_session(
+                acp::LoadSessionRequest::new(session_id.clone(), cwd.to_path_buf())
+                    .mcp_servers(vec![]),
+            ),
+        )
+        .await
+        .unwrap_or_else(|_| panic!("session/load timed out\nstderr:\n{}", self.stderr()))
+        .expect("session/load failed")
+    }
+
+    pub async fn ext_method(
+        &self,
+        method: &str,
+        params: serde_json::Value,
+    ) -> acp::Result<acp::ExtResponse> {
+        let raw = serde_json::value::RawValue::from_string(params.to_string())
+            .expect("serialize ext params");
+        self.conn
+            .ext_method(acp::ExtRequest::new(method, std::sync::Arc::from(raw)))
+            .await
+    }
+}
+
+/// Drives `grok agent stdio` with verbatim newline-delimited JSON-RPC lines.
+///
+/// Exists for wire shapes the typed [`GrokStdioClient`] (`ClientSideConnection`,
+/// integer ids) can never produce — e.g. Xcode's Swift/Foundation `JSONEncoder`
+/// output: escaped-slash methods (`"session\/prompt"`) and string UUID request
+/// ids. Child process is killed on drop.
+pub struct RawStdioClient {
+    stdin: tokio::process::ChildStdin,
+    stdout: tokio::io::BufReader<tokio::process::ChildStdout>,
+    stderr: Arc<std::sync::Mutex<Vec<u8>>>,
+    _child: tokio::process::Child,
+    _home: TempDir,
+}
+
+impl RawStdioClient {
+    pub async fn spawn(server: &MockInferenceServer, cwd: &Path) -> Self {
+        let home = TempDir::new().expect("create temp home");
+        let (mut child, stderr) = spawn_agent_process(server, cwd, home.path(), &[], &[]);
+
+        let stdin = child.stdin.take().expect("child stdin missing");
+        let child_stdout = child.stdout.take().expect("child stdout missing");
+
+        Self {
+            stdin,
+            stdout: tokio::io::BufReader::new(child_stdout),
+            stderr,
+            _child: child,
+            _home: home,
+        }
+    }
+
+    pub fn stderr(&self) -> String {
+        String::from_utf8_lossy(&self.stderr.lock().unwrap()).into_owned()
+    }
+
+    /// Write `line` verbatim followed by `\n`, and flush.
+    pub async fn send_line(&mut self, line: &str) {
+        use tokio::io::AsyncWriteExt as _;
+
+        self.stdin
+            .write_all(line.as_bytes())
+            .await
+            .expect("write line to agent stdin");
+        self.stdin.write_all(b"\n").await.expect("write newline");
+        self.stdin.flush().await.expect("flush agent stdin");
+    }
+
+    /// Read stdout lines until the response to `id` arrives (no `method` key +
+    /// exact string-id match) — returning IS the id-echo assertion: an id
+    /// echoed with different bytes or as a different JSON type never matches
+    /// and surfaces in the timeout diagnostics instead. Notifications are
+    /// skipped; any agent→client request is refused with a JSON-RPC error so a
+    /// turn can never hang on this capability-less client. On timeout the
+    /// panic reports how much non-matching traffic was seen (0 = true
+    /// silence, the acp-0.6 escaped-method symptom) plus the last few lines.
+    pub async fn response_for_id(
+        &mut self,
+        id: &str,
+        what: &str,
+        timeout: Duration,
+    ) -> serde_json::Value {
+        use tokio::io::AsyncBufReadExt as _;
+
+        let deadline = tokio::time::Instant::now() + timeout;
+        let mut line = String::new();
+        let mut skipped = 0_usize;
+        let mut skipped_tail: Vec<String> = Vec::new();
+        loop {
+            line.clear();
+            let next_line = self.stdout.read_line(&mut line);
+            let Ok(io_result) = tokio::time::timeout_at(deadline, next_line).await else {
+                panic!(
+                    "{what}: no matching response within {timeout:?} ({skipped} other messages \
+                     seen; last: {skipped_tail:?})\nstderr:\n{}",
+                    stderr_tail(&self.stderr(), 1200)
+                );
+            };
+            let read =
+                io_result.unwrap_or_else(|e| panic!("{what}: agent stdout read failed: {e}"));
+            if read == 0 {
+                panic!(
+                    "{what}: agent closed stdout before responding ({skipped} other messages \
+                     seen)\nstderr:\n{}",
+                    stderr_tail(&self.stderr(), 1200)
+                );
+            }
+            let Ok(msg) = serde_json::from_str::<serde_json::Value>(line.trim_end()) else {
+                push_skipped_tail(&mut skipped, &mut skipped_tail, &line);
+                continue;
+            };
+            let is_response = msg.get("method").is_none();
+            if is_response && msg.get("id").and_then(|v| v.as_str()) == Some(id) {
+                return msg;
+            }
+            push_skipped_tail(&mut skipped, &mut skipped_tail, &line);
+            if !is_response && let Some(req_id) = msg.get("id") {
+                let refusal = serde_json::json!({
+                    "jsonrpc": "2.0",
+                    "id": req_id,
+                    "error": { "code": -32601, "message": "unsupported by raw test client" },
+                });
+                self.send_line(&refusal.to_string()).await;
+            }
+        }
+    }
+}
+
+/// Record a non-matching line for [`RawStdioClient::response_for_id`]'s timeout
+/// diagnostics: bump the count, keep the last 3 lines (truncated).
+fn push_skipped_tail(skipped: &mut usize, tail: &mut Vec<String>, line: &str) {
+    *skipped += 1;
+    if tail.len() == 3 {
+        tail.remove(0);
+    }
+    tail.push(line.trim_end().chars().take(200).collect());
+}

crates/codegen/xai-grok-test-support/src/counting_server.rs

@@ -0,0 +1,68 @@
+//! Minimal connection-counting HTTP/1.1 server for wire-level tests that need
+//! to assert TCP connection reuse (e.g. shared-client pooling): it counts
+//! accepted connections and records each request's header block.
+
+use std::sync::atomic::{AtomicUsize, Ordering};
+use std::sync::{Arc, Mutex};
+
+use tokio::io::{AsyncReadExt, AsyncWriteExt};
+use tokio::net::TcpListener;
+
+/// Minimal keep-alive HTTP/1.1 server: counts accepted connections and
+/// records each request's header block.
+pub async fn spawn_counting_server() -> (String, Arc<AtomicUsize>, Arc<Mutex<Vec<String>>>) {
+    let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
+    let base_url = format!("http://{}/v1", listener.local_addr().unwrap());
+    let accepts = Arc::new(AtomicUsize::new(0));
+    let heads: Arc<Mutex<Vec<String>>> = Arc::new(Mutex::new(Vec::new()));
+    let (accepts_l, heads_l) = (Arc::clone(&accepts), Arc::clone(&heads));
+    tokio::spawn(async move {
+        loop {
+            let Ok((mut sock, _)) = listener.accept().await else {
+                return;
+            };
+            accepts_l.fetch_add(1, Ordering::SeqCst);
+            let heads = Arc::clone(&heads_l);
+            tokio::spawn(async move {
+                let mut buf: Vec<u8> = Vec::new();
+                loop {
+                    // Read one full request: header block, then content-length body bytes.
+                    let head_end = loop {
+                        if let Some(i) = buf.windows(4).position(|w| w == b"\r\n\r\n") {
+                            break i + 4;
+                        }
+                        let mut chunk = [0u8; 4096];
+                        match sock.read(&mut chunk).await {
+                            Ok(0) | Err(_) => return,
+                            Ok(n) => buf.extend_from_slice(&chunk[..n]),
+                        }
+                    };
+                    let head = String::from_utf8_lossy(&buf[..head_end]).to_string();
+                    let body_len: usize = head
+                        .lines()
+                        .find_map(|l| {
+                            l.to_ascii_lowercase()
+                                .strip_prefix("content-length:")
+                                .and_then(|v| v.trim().parse().ok())
+                        })
+                        .unwrap_or(0);
+                    while buf.len() < head_end + body_len {
+                        let mut chunk = [0u8; 4096];
+                        match sock.read(&mut chunk).await {
+                            Ok(0) | Err(_) => return,
+                            Ok(n) => buf.extend_from_slice(&chunk[..n]),
+                        }
+                    }
+                    buf.drain(..head_end + body_len);
+                    heads.lock().unwrap().push(head);
+                    let resp =
+                        b"HTTP/1.1 200 OK\r\ncontent-type: application/json\r\ncontent-length: 2\r\n\r\n{}";
+                    if sock.write_all(resp).await.is_err() {
+                        return;
+                    }
+                }
+            });
+        }
+    });
+    (base_url, accepts, heads)
+}

crates/codegen/xai-grok-test-support/src/env.rs

@@ -0,0 +1,177 @@
+//! Shared environment helpers: binary resolution, git workdirs, env var setup.
+
+use std::ffi::{OsStr, OsString};
+use std::path::{Path, PathBuf};
+use std::process::Command;
+
+use tempfile::TempDir;
+
+/// RAII guard for a single environment variable in `#[serial]` tests: snapshots
+/// the prior value on construction, applies the change, then restores the prior
+/// value (or unsets it) on drop — even if an assertion panics. Restoring rather
+/// than always unsetting avoids clobbering vars a parent process/harness set
+/// (e.g. `RUST_LOG`).
+///
+/// Callers MUST be `#[serial_test::serial]`: the `unsafe` `set_var`/`remove_var`
+/// are sound only when no other thread accesses the environment concurrently.
+pub struct EnvGuard {
+    key: &'static str,
+    prior: Option<OsString>,
+}
+
+impl EnvGuard {
+    /// Set `key` to `value` for the guard's lifetime. Accepts `&str`, `&Path`,
+    /// `String`, etc. via `AsRef<OsStr>`.
+    pub fn set(key: &'static str, value: impl AsRef<OsStr>) -> Self {
+        let prior = std::env::var_os(key);
+        // SAFETY: callers are `#[serial]`, so no other thread touches the env.
+        unsafe { std::env::set_var(key, value) };
+        Self { key, prior }
+    }
+
+    /// Unset `key` for the guard's lifetime.
+    pub fn unset(key: &'static str) -> Self {
+        let prior = std::env::var_os(key);
+        // SAFETY: see [`EnvGuard::set`].
+        unsafe { std::env::remove_var(key) };
+        Self { key, prior }
+    }
+}
+
+impl Drop for EnvGuard {
+    fn drop(&mut self) {
+        // SAFETY: see [`EnvGuard::set`].
+        match self.prior.take() {
+            Some(v) => unsafe { std::env::set_var(self.key, v) },
+            None => unsafe { std::env::remove_var(self.key) },
+        }
+    }
+}
+
+fn workspace_root() -> PathBuf {
+    // nth(3): crate is nested three levels below the cargo workspace root.
+    PathBuf::from(env!("CARGO_MANIFEST_DIR"))
+        .ancestors()
+        .nth(3)
+        .expect("workspace root")
+        .to_path_buf()
+}
+
+fn target_dir() -> PathBuf {
+    std::env::var_os("CARGO_TARGET_DIR")
+        .map(PathBuf::from)
+        .unwrap_or_else(|| workspace_root().join("target"))
+}
+
+fn local_grok_binary_path() -> PathBuf {
+    target_dir()
+        .join("debug")
+        .join(format!("xai-grok-pager{}", std::env::consts::EXE_SUFFIX))
+}
+
+fn ensure_local_grok_binary(binary: &Path) {
+    if binary.exists() {
+        return;
+    }
+
+    let cargo = std::env::var("CARGO").unwrap_or_else(|_| "cargo".to_string());
+    let output = Command::new(&cargo)
+        .current_dir(workspace_root())
+        .args(["build", "-p", "xai-grok-pager", "--bin", "xai-grok-pager"])
+        .output()
+        .unwrap_or_else(|e| panic!("failed to spawn {cargo} to build xai-grok-pager: {e}"));
+
+    assert!(
+        output.status.success(),
+        "failed to build xai-grok-pager for lifecycle tests (exit {:?})\nstdout:\n{}\nstderr:\n{}",
+        output.status.code(),
+        String::from_utf8_lossy(&output.stdout),
+        String::from_utf8_lossy(&output.stderr),
+    );
+    assert!(
+        binary.exists(),
+        "xai-grok-pager build completed but binary missing at {}",
+        binary.display()
+    );
+}
+
+/// Resolve grok binary: `GROK_BINARY` env (CI) or a locally built `xai-grok-pager` binary.
+pub fn grok_binary() -> PathBuf {
+    if let Ok(path) = std::env::var("GROK_BINARY") {
+        let p = PathBuf::from(path);
+        assert!(p.exists(), "GROK_BINARY does not exist: {}", p.display());
+        return p;
+    }
+
+    if let Ok(path) = std::env::var("CARGO_BIN_EXE_xai-grok-pager") {
+        let p = PathBuf::from(path);
+        if p.exists() {
+            return p;
+        }
+    }
+
+    let binary = local_grok_binary_path();
+    ensure_local_grok_binary(&binary);
+    binary
+}
+
+/// Temp dir with a git repo + one committed file.
+/// Forces libgit2 to fully init (the codepath that breaks with bad OpenSSL linking).
+pub fn git_workdir() -> TempDir {
+    let dir = TempDir::new().expect("create temp dir");
+    let path = dir.path();
+
+    fn run_git(args: &[&str], dir: &Path) {
+        let output = Command::new("git")
+            .args(args)
+            .current_dir(dir)
+            .output()
+            .unwrap_or_else(|e| panic!("failed to spawn git {}: {e}", args.join(" ")));
+        assert!(
+            output.status.success(),
+            "git {} failed (exit {:?}):\n{}",
+            args.join(" "),
+            output.status.code(),
+            String::from_utf8_lossy(&output.stderr),
+        );
+    }
+
+    run_git(&["init"], path);
+    // Configure git user for commits (required in CI where no global config exists)
+    run_git(&["config", "user.email", "test@test.com"], path);
+    run_git(&["config", "user.name", "Test"], path);
+
+    std::fs::write(path.join("README.md"), "test file\n").expect("write test file");
+
+    run_git(&["add", "-A"], path);
+    run_git(&["commit", "-m", "init", "--no-gpg-sign"], path);
+
+    dir
+}
+
+/// Point grok at the mock server with a fake API key and telemetry disabled.
+pub fn test_env_cmd_tokio(
+    cmd: &mut tokio::process::Command,
+    mock_url: &str,
+    home: &std::path::Path,
+) {
+    cmd.env("HOME", home)
+        // HOME alone does not sandbox grok on Windows: the product resolves
+        // `~` via `USERPROFILE`/Known Folders (`std::env::home_dir()`), so
+        // without an explicit GROK_HOME every spawned child shares the real
+        // `%USERPROFILE%\.grok` — test 1's models_cache.json (which embeds
+        // its per-test mock-server URL) then poisons every later test's
+        // prompt (the windows-x86_64 lifecycle "prompt timed out" failure).
+        // Mirrors `leader.rs` and the pty-harness `env_for_pager`.
+        .env("GROK_HOME", home.join(".grok"))
+        .env("GROK_CLI_CHAT_PROXY_BASE_URL", mock_url)
+        .env("GROK_XAI_API_BASE_URL", mock_url)
+        .env("XAI_API_KEY", "test-key-for-ci")
+        .env("GROK_TELEMETRY_ENABLED", "false")
+        .env("GROK_FEEDBACK_ENABLED", "false")
+        .env("GROK_TRACE_UPLOAD", "false")
+        .env("GROK_INSTRUMENTATION", "disabled")
+        // Release binaries (CI lifecycle tests) otherwise spawn a background
+        // update check that hits the network and can add latency under Rosetta.
+        .env("GROK_DISABLE_AUTOUPDATER", "1");
+}

crates/codegen/xai-grok-test-support/src/headless.rs

@@ -0,0 +1,154 @@
+//! Headless mode (`grok -p`) test runner.
+//!
+//! Runs the grok binary as a subprocess with the mock server, captures output.
+
+use std::path::Path;
+use std::process::ExitStatus;
+use std::time::Duration;
+
+use tempfile::TempDir;
+use tokio::io::AsyncReadExt as _;
+
+use crate::env::{grok_binary, test_env_cmd_tokio};
+use crate::mock_server::MockInferenceServer;
+
+pub struct HeadlessResult {
+    pub status: ExitStatus,
+    pub stdout: String,
+    pub stderr: String,
+    pub timed_out: bool,
+}
+
+const HEADLESS_TIMEOUT_SECS: u64 = 60;
+
+/// Run `grok` with the given args against the mock server, with a 60s timeout.
+/// Uses an isolated HOME and disables telemetry.
+pub async fn run_headless(
+    server: &MockInferenceServer,
+    args: &[&str],
+    cwd: &Path,
+) -> HeadlessResult {
+    let home = TempDir::new().expect("create temp home");
+    let mut cmd = tokio::process::Command::new(grok_binary());
+    cmd.args(args)
+        .current_dir(cwd)
+        .stdin(std::process::Stdio::null())
+        .stdout(std::process::Stdio::piped())
+        .stderr(std::process::Stdio::piped())
+        .kill_on_drop(true);
+    test_env_cmd_tokio(&mut cmd, &server.url(), home.path());
+    run_headless_with_cmd(cmd).await
+}
+
+pub async fn run_headless_with_cmd(mut cmd: tokio::process::Command) -> HeadlessResult {
+    let binary = grok_binary();
+    let mut child = cmd
+        .spawn()
+        .unwrap_or_else(|e| panic!("failed to spawn grok binary at {}: {e}", binary.display()));
+
+    let stdout = child.stdout.take().expect("child stdout missing");
+    let stderr = child.stderr.take().expect("child stderr missing");
+
+    let stdout_handle = tokio::spawn(async move {
+        let mut stdout = stdout;
+        let mut stdout_buf = Vec::new();
+        stdout.read_to_end(&mut stdout_buf).await?;
+        Ok::<Vec<u8>, std::io::Error>(stdout_buf)
+    });
+    let stderr_handle = tokio::spawn(async move {
+        let mut stderr = stderr;
+        let mut stderr_buf = Vec::new();
+        stderr.read_to_end(&mut stderr_buf).await?;
+        Ok::<Vec<u8>, std::io::Error>(stderr_buf)
+    });
+
+    let (status, timed_out) = match tokio::time::timeout(
+        Duration::from_secs(HEADLESS_TIMEOUT_SECS),
+        child.wait(),
+    )
+    .await
+    {
+        Ok(result) => (
+            result.unwrap_or_else(|e| {
+                panic!("failed to wait for grok binary {}: {e}", binary.display())
+            }),
+            false,
+        ),
+        Err(_) => {
+            let _ = child.kill().await;
+            let status = child.wait().await.unwrap_or_else(|e| {
+                panic!(
+                    "failed to kill timed out grok binary {}: {e}",
+                    binary.display()
+                )
+            });
+            (status, true)
+        }
+    };
+
+    let stdout_bytes = match stdout_handle.await {
+        Ok(Ok(bytes)) => bytes,
+        Ok(Err(err)) => panic!("failed to read stdout from {}: {err}", binary.display()),
+        Err(err) => panic!("stdout task join failed for {}: {err}", binary.display()),
+    };
+    let stderr_bytes = match stderr_handle.await {
+        Ok(Ok(bytes)) => bytes,
+        Ok(Err(err)) => panic!("failed to read stderr from {}: {err}", binary.display()),
+        Err(err) => panic!("stderr task join failed for {}: {err}", binary.display()),
+    };
+
+    HeadlessResult {
+        status,
+        stdout: String::from_utf8_lossy(&stdout_bytes).into_owned(),
+        stderr: String::from_utf8_lossy(&stderr_bytes).into_owned(),
+        timed_out,
+    }
+}
+
+const CRASH_PATTERNS: &[&str] = &[
+    "panicked at",
+    "SIGSEGV",
+    "segfault",
+    "undefined symbol",
+    "SIGABRT",
+    "cannot open shared object",
+];
+
+/// Diagnostic helper: format the tail of stderr for assertion messages.
+pub fn stderr_tail(stderr: &str, max_chars: usize) -> &str {
+    &stderr[stderr.len().saturating_sub(max_chars)..]
+}
+
+/// Assert that a headless run succeeded (non-timeout, zero exit code).
+pub fn assert_headless_success(
+    result: &HeadlessResult,
+    label: &str,
+    server: Option<&MockInferenceServer>,
+) {
+    assert!(
+        !result.timed_out,
+        "{label}: timed out after {HEADLESS_TIMEOUT_SECS}s\nstderr tail:\n{}",
+        stderr_tail(&result.stderr, 500)
+    );
+    assert!(
+        result.status.success(),
+        "{label}: exited with {:?}\nstderr tail:\n{}\n{}",
+        result.status.code(),
+        stderr_tail(&result.stderr, 1000),
+        server
+            .map(|s| format!("request log:\n{}", s.request_log_summary()))
+            .unwrap_or_default()
+    );
+}
+
+/// Panic if stderr contains any crash/linking-failure indicators.
+pub fn assert_no_crashes(stderr: &str) {
+    let lower = stderr.to_lowercase();
+    for pattern in CRASH_PATTERNS {
+        assert!(
+            !lower.contains(&pattern.to_lowercase()),
+            "stderr contains crash indicator '{pattern}':\n{}",
+            stderr_tail(stderr, 500)
+        );
+    }
+}

crates/codegen/xai-grok-test-support/src/leader.rs

@@ -0,0 +1,354 @@
+//! Leader-mode (`grok agent --leader stdio`) test harness.
+//!
+//! Spawns the real binary as a stdio client whose bridge elects a leader
+//! subprocess hosting the actual sessions, speaks ACP over pipes, and
+//! exposes lock-file helpers for leader-lifecycle assertions. Unix-only:
+//! the leader transport is a unix socket.
+
+use std::path::{Path, PathBuf};
+use std::sync::Arc;
+use std::sync::atomic::{AtomicU32, Ordering};
+use std::time::Duration;
+
+use agent_client_protocol::{self as acp, Agent as _};
+use tokio_util::compat::{TokioAsyncReadCompatExt, TokioAsyncWriteCompatExt};
+use xai_acp_lib::LineBufferedRead;
+
+use crate::env::grok_binary;
+use crate::mock_server::MockInferenceServer;
+use crate::process::spawn_piped_with_stderr_capture;
+
+/// Env var naming the binary that elects/hosts the leader in a two-binary
+/// (version-skew) test. Falls back to [`grok_binary`]'s resolution.
+pub const LEADER_BINARY_ENV: &str = "GROK_BINARY_LEADER";
+
+/// Env var naming the binary for the second (usually newer) client in a
+/// two-binary test. Falls back to [`grok_binary`]'s resolution.
+pub const CLIENT_BINARY_ENV: &str = "GROK_BINARY_CLIENT";
+
+fn role_binary(env_key: &str) -> PathBuf {
+    if let Ok(path) = std::env::var(env_key) {
+        let p = PathBuf::from(path);
+        assert!(p.exists(), "{env_key} does not exist: {}", p.display());
+        return p;
+    }
+    grok_binary()
+}
+
+/// Binary for the leader-electing side of a version-skew test
+/// (`GROK_BINARY_LEADER`, else the shared [`grok_binary`] resolution).
+pub fn leader_binary() -> PathBuf {
+    role_binary(LEADER_BINARY_ENV)
+}
+
+/// Binary for the client side of a version-skew test (`GROK_BINARY_CLIENT`,
+/// else the shared [`grok_binary`] resolution).
+pub fn client_binary() -> PathBuf {
+    role_binary(CLIENT_BINARY_ENV)
+}
+
+/// Capture for notifications + reconnect signals.
+#[derive(Default)]
+pub struct Capture {
+    chunks: std::sync::Mutex<Vec<String>>,
+    notification_count: AtomicU32,
+    reconnected_count: AtomicU32,
+}
+
+struct LeaderAcpClient {
+    capture: Arc<Capture>,
+}
+
+#[async_trait::async_trait(?Send)]
+impl acp::Client for LeaderAcpClient {
+    async fn request_permission(
+        &self,
+        args: acp::RequestPermissionRequest,
+    ) -> acp::Result<acp::RequestPermissionResponse> {
+        let outcome = args
+            .options
+            .iter()
+            .find(|o| o.kind == acp::PermissionOptionKind::AllowOnce)
+            .or(args.options.first())
+            .map(|o| {
+                acp::RequestPermissionOutcome::Selected(acp::SelectedPermissionOutcome::new(
+                    o.option_id.clone(),
+                ))
+            })
+            .unwrap_or(acp::RequestPermissionOutcome::Cancelled);
+        Ok(acp::RequestPermissionResponse::new(outcome))
+    }
+
+    async fn session_notification(&self, args: acp::SessionNotification) -> acp::Result<()> {
+        self.capture
+            .notification_count
+            .fetch_add(1, Ordering::SeqCst);
+        if let acp::SessionUpdate::AgentMessageChunk(acp::ContentChunk { content, .. }) =
+            args.update
+            && let acp::ContentBlock::Text(t) = content
+        {
+            self.capture.chunks.lock().unwrap().push(t.text);
+        }
+        Ok(())
+    }
+
+    async fn ext_notification(&self, args: acp::ExtNotification) -> acp::Result<()> {
+        if &*args.method == "x.ai/leader_reconnected" {
+            self.capture
+                .reconnected_count
+                .fetch_add(1, Ordering::SeqCst);
+        }
+        Ok(())
+    }
+}
+
+/// A `grok agent --leader stdio` client subprocess speaking ACP over pipes.
+/// The leader subprocess it elects hosts the actual sessions.
+pub struct LeaderStdioClient {
+    pub conn: acp::ClientSideConnection,
+    // Exposed for PID assertions.
+    pub child: tokio::process::Child,
+    capture: Arc<Capture>,
+    stderr: Arc<std::sync::Mutex<Vec<u8>>>,
+}
+
+impl LeaderStdioClient {
+    pub async fn spawn(server: &MockInferenceServer, cwd: &Path, home: &Path) -> Self {
+        Self::spawn_with_binary(&grok_binary(), server, cwd, home).await
+    }
+
+    /// [`Self::spawn`] with an explicit binary, for two-binary version-skew
+    /// tests (pair with [`leader_binary`] / [`client_binary`]).
+    pub async fn spawn_with_binary(
+        binary: &Path,
+        server: &MockInferenceServer,
+        cwd: &Path,
+        home: &Path,
+    ) -> Self {
+        let mut cmd = tokio::process::Command::new(binary);
+        cmd.args(["agent", "--leader", "stdio"])
+            .current_dir(cwd)
+            // Hermetic env: the developer's shell may export GROK_* vars
+            // (e.g. GROK_LEADER_SOCKET pointing at a REAL leader on this
+            // machine). env_clear + explicit allowlist guarantees the test
+            // can never touch a leader outside its sandbox home.
+            .env_clear()
+            .env("PATH", std::env::var("PATH").unwrap_or_default())
+            .env("HOME", home)
+            .env("GROK_HOME", home.join(".grok"))
+            // Pin the socket inside the sandbox. The lock file is the
+            // sibling `.lock` (leader.sock -> leader.lock), and the spawned
+            // leader subprocess inherits/forwards this env var, so every
+            // (re-)elected leader binds the same sandboxed path.
+            .env("GROK_LEADER_SOCKET", home.join(".grok").join("leader.sock"))
+            .env("GROK_CLI_CHAT_PROXY_BASE_URL", server.url())
+            .env("GROK_XAI_API_BASE_URL", server.url())
+            .env("XAI_API_KEY", "test-key-for-ci")
+            .env("GROK_TELEMETRY_ENABLED", "false")
+            .env("GROK_FEEDBACK_ENABLED", "false")
+            .env("GROK_TRACE_UPLOAD", "false")
+            .env("GROK_INSTRUMENTATION", "disabled")
+            // Inherited by the spawned leader, whose stderr goes to
+            // ~/.grok/leader.log — keep it chatty for diagnosis.
+            .env("RUST_LOG", "xai_grok_shell=debug");
+
+        let (mut child, stderr) = spawn_piped_with_stderr_capture(cmd);
+
+        let outgoing = child.stdin.take().unwrap().compat_write();
+        let incoming = child.stdout.take().unwrap().compat();
+
+        let capture = Arc::new(Capture::default());
+        let client = LeaderAcpClient {
+            capture: capture.clone(),
+        };
+        let incoming = LineBufferedRead::spawn_local(incoming);
+        let (conn, handle_io) = acp::ClientSideConnection::new(client, outgoing, incoming, |fut| {
+            tokio::task::spawn_local(fut);
+        });
+        tokio::task::spawn_local(handle_io);
+
+        Self {
+            conn,
+            child,
+            capture,
+            stderr,
+        }
+    }
+
+    pub fn stderr_text(&self) -> String {
+        String::from_utf8_lossy(&self.stderr.lock().unwrap()).into_owned()
+    }
+
+    pub fn captured_text(&self) -> String {
+        self.capture.chunks.lock().unwrap().join("")
+    }
+
+    pub async fn initialize(&self) -> acp::InitializeResponse {
+        let init = tokio::time::timeout(
+            Duration::from_secs(60),
+            self.conn.initialize(
+                acp::InitializeRequest::new(acp::ProtocolVersion::V1)
+                    .client_capabilities(
+                        acp::ClientCapabilities::new()
+                            .fs(acp::FileSystemCapabilities::new())
+                            .terminal(false),
+                    )
+                    .meta(
+                        serde_json::json!({
+                            "startupHints": {
+                                "nonInteractive": true,
+                                "skipGitStatus": true,
+                                "skipProjectLayout": true
+                            },
+                            "clientType": "test-client",
+                            "clientVersion": "0.0.0-test"
+                        })
+                        .as_object()
+                        .cloned(),
+                    ),
+            ),
+        )
+        .await
+        .unwrap_or_else(|_| panic!("initialize timed out\nstderr:\n{}", self.stderr_text()))
+        .expect("initialize failed");
+
+        let api_key_method = init
+            .auth_methods
+            .iter()
+            .find(|m| &*m.id().0 == "xai.api_key")
+            .expect("xai.api_key auth method");
+        self.conn
+            .authenticate(
+                acp::AuthenticateRequest::new(api_key_method.id().clone())
+                    .meta(serde_json::json!({"headless": true}).as_object().cloned()),
+            )
+            .await
+            .expect("authenticate failed");
+        init
+    }
+
+    pub async fn create_session(&self, cwd: &Path) -> acp::SessionId {
+        self.create_session_inner(cwd, None).await
+    }
+
+    pub async fn create_session_with_model(&self, cwd: &Path, model_id: &str) -> acp::SessionId {
+        self.create_session_inner(
+            cwd,
+            serde_json::json!({ "modelId": model_id })
+                .as_object()
+                .cloned(),
+        )
+        .await
+    }
+
+    async fn create_session_inner(&self, cwd: &Path, meta: Option<acp::Meta>) -> acp::SessionId {
+        tokio::time::timeout(
+            Duration::from_secs(30),
+            self.conn.new_session(
+                acp::NewSessionRequest::new(cwd.to_path_buf())
+                    .mcp_servers(vec![])
+                    .meta(meta),
+            ),
+        )
+        .await
+        .unwrap_or_else(|_| panic!("session/new timed out\nstderr:\n{}", self.stderr_text()))
+        .expect("session/new failed")
+        .session_id
+    }
+
+    pub async fn prompt(
+        &self,
+        session_id: &acp::SessionId,
+        text: &str,
+    ) -> acp::Result<acp::PromptResponse> {
+        tokio::time::timeout(
+            Duration::from_secs(30),
+            self.conn.prompt(acp::PromptRequest::new(
+                session_id.clone(),
+                vec![acp::ContentBlock::Text(acp::TextContent::new(
+                    text.to_string(),
+                ))],
+            )),
+        )
+        .await
+        .unwrap_or_else(|_| panic!("prompt timed out\nstderr:\n{}", self.stderr_text()))
+    }
+
+    pub fn reconnected_count(&self) -> u32 {
+        self.capture.reconnected_count.load(Ordering::SeqCst)
+    }
+
+    pub fn notification_count(&self) -> u32 {
+        self.capture.notification_count.load(Ordering::SeqCst)
+    }
+}
+
+pub fn leader_lock_path(home: &Path) -> PathBuf {
+    home.join(".grok").join("leader.lock")
+}
+
+pub fn read_leader_pid(home: &Path) -> Option<u32> {
+    std::fs::read_to_string(leader_lock_path(home))
+        .ok()?
+        .trim()
+        .parse()
+        .ok()
+}
+
+pub fn pid_alive(pid: u32) -> bool {
+    unsafe { libc::kill(pid as i32, 0) == 0 }
+}
+
+/// Wait until the leader lock file contains a live PID, return it.
+pub async fn wait_for_live_leader(home: &Path, timeout: Duration) -> Option<u32> {
+    let deadline = tokio::time::Instant::now() + timeout;
+    while tokio::time::Instant::now() < deadline {
+        if let Some(pid) = read_leader_pid(home)
+            && pid_alive(pid)
+        {
+            return Some(pid);
+        }
+        tokio::time::sleep(Duration::from_millis(100)).await;
+    }
+    None
+}
+
+/// Wait until the leader lock file contains a live PID *different* from `old_pid`.
+pub async fn wait_for_new_leader(home: &Path, old_pid: u32, timeout: Duration) -> Option<u32> {
+    let deadline = tokio::time::Instant::now() + timeout;
+    while tokio::time::Instant::now() < deadline {
+        if let Some(pid) = read_leader_pid(home)
+            && pid != old_pid
+            && pid_alive(pid)
+        {
+            return Some(pid);
+        }
+        tokio::time::sleep(Duration::from_millis(100)).await;
+    }
+    None
+}
+
+/// Wait for evidence that the bridge finished its reconnect replay.
+///
+/// The `x.ai/leader_reconnected` ext notification is dropped by the typed
+/// `ClientSideConnection` (bare `x.ai/*` methods are rejected by the ACP
+/// decoder), so we wait for the replayed `session/load` to emit session
+/// notifications instead: the notification count rises above `baseline`.
+pub async fn wait_for_replay_notifications(
+    client: &LeaderStdioClient,
+    baseline: u32,
+    timeout: Duration,
+) -> bool {
+    let deadline = tokio::time::Instant::now() + timeout;
+    while tokio::time::Instant::now() < deadline {
+        if client.reconnected_count() > 0 || client.notification_count() > baseline {
+            return true;
+        }
+        tokio::time::sleep(Duration::from_millis(100)).await;
+    }
+    false
+}
+
+pub fn leader_log(home: &Path) -> String {
+    std::fs::read_to_string(home.join(".grok").join("leader.log")).unwrap_or_default()
+}

crates/codegen/xai-grok-test-support/src/lib.rs

@@ -0,0 +1,41 @@
+#![allow(
+    unused_imports,
+    unused_variables,
+    unused_mut,
+    unreachable_code,
+    dead_code
+)]
+//! Shared test utilities for grok-build crates: mock inference server, SSE
+//! generators, ACP stdio client, headless runner, env sandbox.
+//!
+//! Provides:
+//! - [`MockInferenceServer`] — Mock /v1/chat/completions + /v1/responses with request logging
+//! - [`GrokStdioClient`] — ACP client that drives `grok agent stdio` as a subprocess
+//! - [`RawStdioClient`] — raw-wire ACP driver for bytes the typed client can't
+//!   produce (Foundation `\/` methods, string UUID ids)
+//! - [`leader::LeaderStdioClient`] — ACP client that drives `grok agent --leader stdio` (unix)
+//! - [`run_headless`] — Run `grok -p` against the mock server and capture output
+//! - [`git_workdir`] — Create a temp directory with git repo (forces libgit2 init)
+//! - [`grok_binary`] — Resolve the grok binary path (GROK_BINARY env or cargo_bin)
+//! - [`spawn_counting_server`] — Connection-counting HTTP/1.1 server for wire/pooling tests
+//! - [`uds_proxy::UdsProxy`] — Frame-aware fault-injection proxy for leader IPC sockets (unix)
+pub mod acp_client;
+pub mod counting_server;
+pub mod env;
+pub mod headless;
+#[cfg(unix)]
+pub mod leader;
+pub mod mock_server;
+mod process;
+pub mod scripted;
+pub mod sse;
+#[cfg(unix)]
+pub mod uds_proxy;
+pub use acp_client::{GrokStdioClient, RawStdioClient};
+pub use counting_server::spawn_counting_server;
+pub use env::{EnvGuard, git_workdir, grok_binary};
+pub use headless::{
+    HeadlessResult, assert_headless_success, assert_no_crashes, run_headless,
+    run_headless_with_cmd, stderr_tail,
+};
+pub use mock_server::{MockInferenceServer, MockModelEntry, ScriptedResponse, SseEvent};

crates/codegen/xai-grok-test-support/src/process.rs

@@ -0,0 +1,46 @@
+//! General subprocess plumbing shared by the harnesses in this crate.
+
+use std::sync::Arc;
+
+/// Pipe all three stdio handles, `kill_on_drop`, spawn, and drain the child's
+/// stderr into the returned buffer on a background task. The one spawn path
+/// shared by every subprocess harness in this crate (`GrokStdioClient`,
+/// `RawStdioClient`, `leader::LeaderStdioClient`); env/args stay with the
+/// callers, whose hermeticity models differ (sandbox-inherit vs `env_clear`).
+/// The drain future is `Send`, so this works on and off a `LocalSet`.
+pub(crate) fn spawn_piped_with_stderr_capture(
+    mut cmd: tokio::process::Command,
+) -> (tokio::process::Child, Arc<std::sync::Mutex<Vec<u8>>>) {
+    cmd.stdin(std::process::Stdio::piped())
+        .stdout(std::process::Stdio::piped())
+        .stderr(std::process::Stdio::piped())
+        .kill_on_drop(true);
+
+    // Derived from `cmd` itself so the panic can never name a different binary
+    // than the one actually spawned.
+    let program = cmd.as_std().get_program().to_string_lossy().into_owned();
+    let mut child = cmd
+        .spawn()
+        .unwrap_or_else(|e| panic!("failed to spawn grok at {program}: {e}"));
+
+    let stderr = Arc::new(std::sync::Mutex::new(Vec::new()));
+    let stderr_capture = stderr.clone();
+    let mut child_stderr = child.stderr.take().expect("child stderr missing");
+    tokio::spawn(async move {
+        use tokio::io::AsyncReadExt as _;
+
+        let mut buf = [0_u8; 1024];
+        loop {
+            match child_stderr.read(&mut buf).await {
+                Ok(0) => break,
+                Ok(read) => stderr_capture
+                    .lock()
+                    .unwrap()
+                    .extend_from_slice(&buf[..read]),
+                Err(_) => break,
+            }
+        }
+    });
+
+    (child, stderr)
+}

crates/codegen/xai-grok-test-support/src/scripted.rs

@@ -0,0 +1,138 @@
+//! Data-driven scripted responses for the mock inference server: plain
+//! status/header/body triples queued per path and rendered to HTTP at serve
+//! time. Pure data — no router or handler types in the public surface.
+
+use std::convert::Infallible;
+
+use axum::Json;
+use axum::http::{HeaderName, HeaderValue, StatusCode};
+use axum::response::sse::{KeepAlive, Sse};
+use axum::response::{IntoResponse, Response};
+use futures_util::stream;
+use serde_json::Value;
+
+/// One SSE event as data: optional `event:` name plus the `data:` payload.
+#[derive(Debug, Clone)]
+pub struct SseEvent {
+    pub event: Option<String>,
+    pub data: String,
+}
+
+impl SseEvent {
+    /// Event with a `data:` payload only.
+    pub fn data(data: impl Into<String>) -> Self {
+        Self {
+            event: None,
+            data: data.into(),
+        }
+    }
+
+    /// Event with an `event:` name and a `data:` payload.
+    pub fn with_event(event: impl Into<String>, data: impl Into<String>) -> Self {
+        Self {
+            event: Some(event.into()),
+            data: data.into(),
+        }
+    }
+}
+
+/// Body of a [`ScriptedResponse`].
+#[derive(Debug, Clone)]
+pub enum ScriptedBody {
+    Json(Value),
+    Sse(Vec<SseEvent>),
+    /// Raw body bytes, served verbatim (byte-controllable malformed SSE etc.).
+    Raw(String),
+}
+
+/// A scripted reply for a single request on one path, consumed FIFO.
+/// Takes precedence over the response mode AND the required-auth check —
+/// a script is full control over the next reply.
+#[derive(Debug, Clone)]
+pub struct ScriptedResponse {
+    pub status: u16,
+    pub headers: Vec<(String, String)>,
+    pub body: ScriptedBody,
+}
+
+impl ScriptedResponse {
+    /// 200 SSE response built from an event list.
+    pub fn sse(events: Vec<SseEvent>) -> Self {
+        Self {
+            status: 200,
+            headers: Vec::new(),
+            body: ScriptedBody::Sse(events),
+        }
+    }
+
+    /// JSON body with the given status.
+    pub fn json(status: u16, body: Value) -> Self {
+        Self {
+            status,
+            headers: Vec::new(),
+            body: ScriptedBody::Json(body),
+        }
+    }
+
+    /// Raw text body with the given status.
+    pub fn text(status: u16, body: impl Into<String>) -> Self {
+        Self {
+            status,
+            headers: Vec::new(),
+            body: ScriptedBody::Raw(body.into()),
+        }
+    }
+
+    /// Validate status and headers eagerly so a bad script panics at the
+    /// enqueue call site rather than far away at serve time.
+    pub(crate) fn validate(&self) {
+        StatusCode::from_u16(self.status).expect("invalid scripted status code");
+        for (name, value) in &self.headers {
+            HeaderName::from_bytes(name.as_bytes()).expect("invalid scripted header name");
+            HeaderValue::from_str(value).expect("invalid scripted header value");
+        }
+    }
+
+    /// Render to HTTP with SSE events paced by `delay` (sleep before each
+    /// event, mirroring the fixed/echo `paced_events` pacing) so
+    /// `set_chunk_delay` also holds scripted turns open. `None` streams
+    /// instantly. Non-SSE bodies ignore the delay.
+    pub(crate) fn into_response_paced(self, delay: Option<std::time::Duration>) -> Response {
+        use futures_util::StreamExt as _;
+        let mut resp = match self.body {
+            ScriptedBody::Json(v) => Json(v).into_response(),
+            ScriptedBody::Raw(s) => s.into_response(),
+            ScriptedBody::Sse(events) => {
+                let events: Vec<axum::response::sse::Event> = events
+                    .into_iter()
+                    .map(|e| {
+                        let ev = axum::response::sse::Event::default().data(e.data);
+                        match e.event {
+                            Some(name) => ev.event(name),
+                            None => ev,
+                        }
+                    })
+                    .collect();
+                let stream = stream::iter(events.into_iter().map(Ok::<_, Infallible>)).then(
+                    move |event| async move {
+                        if let Some(d) = delay {
+                            tokio::time::sleep(d).await;
+                        }
+                        event
+                    },
+                );
+                Sse::new(stream)
+                    .keep_alive(KeepAlive::default())
+                    .into_response()
+            }
+        };
+        *resp.status_mut() = StatusCode::from_u16(self.status).expect("valid scripted status code");
+        for (k, v) in self.headers {
+            resp.headers_mut().insert(
+                HeaderName::from_bytes(k.as_bytes()).expect("valid scripted header name"),
+                HeaderValue::from_str(&v).expect("valid scripted header value"),
+            );
+        }
+        resp
+    }
+}

crates/codegen/xai-grok-test-support/src/uds_proxy.rs

@@ -0,0 +1,512 @@
+//! Frame-aware fault-injection proxy for unix-domain-socket IPC.
+//!
+//! Sits between a client and a real listener (`proxy.sock` → `real.sock`),
+//! parsing the leader IPC framing (4-byte big-endian length prefix + body) so
+//! faults land on exact frame boundaries: drop exactly the Nth frame, sever
+//! after a half-written length prefix, delay or duplicate one frame. Everything
+//! is path-addressed, so no production changes are needed — point
+//! `LeaderClient::connect` / `GROK_LEADER_SOCKET` at the proxy path.
+//!
+//! Frame numbering is 1-based and **per proxied connection, per direction**;
+//! reconnects restart the count. Unix-only (the leader transport on Windows is
+//! a named pipe, which cannot be interposed this way); gated in `lib.rs`.
+
+use std::io;
+use std::path::{Path, PathBuf};
+use std::sync::Arc;
+use std::sync::atomic::{AtomicU64, Ordering};
+use std::time::Duration;
+
+use tokio::io::{AsyncReadExt, AsyncWriteExt, ReadHalf, WriteHalf};
+use tokio::net::{UnixListener, UnixStream};
+use tokio_util::sync::CancellationToken;
+
+/// Which pump direction a [`FaultPlan`] applies to.
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
+pub enum FaultDirection {
+    #[default]
+    ClientToLeader,
+    LeaderToClient,
+}
+
+/// Frame-indexed fault schedule (1-based, per connection, per direction).
+///
+/// The default plan is a transparent pass-through.
+#[derive(Debug, Clone, Default)]
+pub struct FaultPlan {
+    /// Direction the frame-indexed faults below apply to; the other direction
+    /// always passes through untouched.
+    pub direction: FaultDirection,
+    /// Silently drop the Nth frame (never forwarded).
+    pub drop_frame: Option<u64>,
+    /// On the Nth frame, forward only 2 bytes of its 4-byte length prefix,
+    /// then hard-close both sides of the connection.
+    pub sever_mid_frame: Option<u64>,
+    /// Hold the Nth frame for the given duration before forwarding it.
+    pub delay: Option<(u64, Duration)>,
+    /// Forward the Nth frame twice.
+    pub duplicate_frame: Option<u64>,
+}
+
+#[derive(Default)]
+struct FaultState {
+    /// Current sever scope: cancelled + swapped for a fresh token on every
+    /// [`FaultHandle::sever_now`], so only connections active at sever time die.
+    sever_now: std::sync::Mutex<CancellationToken>,
+    /// Frames fully forwarded client→leader across all connections.
+    forwarded_c2l: AtomicU64,
+    /// Frames fully forwarded leader→client across all connections.
+    forwarded_l2c: AtomicU64,
+}
+
+/// Runtime control over a running [`UdsProxy`].
+#[derive(Clone, Default)]
+pub struct FaultHandle {
+    state: Arc<FaultState>,
+}
+
+impl FaultHandle {
+    /// Hard-close every active proxied connection immediately (mid-stream
+    /// sever, independent of the frame-indexed plan). Later connections
+    /// through the same proxy are unaffected.
+    pub fn sever_now(&self) {
+        let mut guard = self.state.sever_now.lock().unwrap();
+        guard.cancel();
+        *guard = CancellationToken::new();
+    }
+
+    fn connection_scope(&self) -> CancellationToken {
+        self.state.sever_now.lock().unwrap().child_token()
+    }
+
+    /// Frames fully forwarded so far in the given direction. Relaxed:
+    /// independent counters, no cross-variable ordering to protect.
+    pub fn forwarded(&self, direction: FaultDirection) -> u64 {
+        match direction {
+            FaultDirection::ClientToLeader => self.state.forwarded_c2l.load(Ordering::Relaxed),
+            FaultDirection::LeaderToClient => self.state.forwarded_l2c.load(Ordering::Relaxed),
+        }
+    }
+}
+
+/// A running proxy: listener on [`Self::proxy_path`], forwarding to the
+/// upstream path it was spawned with. Dropping the struct stops the listener
+/// and severs active connections.
+pub struct UdsProxy {
+    pub proxy_path: PathBuf,
+    handle: FaultHandle,
+    cancel: CancellationToken,
+}
+
+impl UdsProxy {
+    /// Bind `proxy_path` and forward each accepted connection to
+    /// `upstream_path`, applying `plan` per connection.
+    pub async fn spawn(
+        proxy_path: impl Into<PathBuf>,
+        upstream_path: impl AsRef<Path>,
+        plan: FaultPlan,
+    ) -> io::Result<Self> {
+        let proxy_path = proxy_path.into();
+        let upstream_path = upstream_path.as_ref().to_path_buf();
+        let _ = std::fs::remove_file(&proxy_path);
+        let listener = UnixListener::bind(&proxy_path)?;
+
+        let handle = FaultHandle::default();
+        let cancel = CancellationToken::new();
+
+        let accept_handle = handle.clone();
+        let accept_cancel = cancel.clone();
+        tokio::spawn(async move {
+            loop {
+                tokio::select! {
+                    _ = accept_cancel.cancelled() => break,
+                    accepted = listener.accept() => {
+                        let Ok((client, _)) = accepted else { break };
+                        let Ok(upstream) = UnixStream::connect(&upstream_path).await else {
+                            // Upstream gone: dropping `client` models a refused
+                            // connection; the caller's retry logic takes over.
+                            continue;
+                        };
+                        spawn_connection(client, upstream, plan.clone(), accept_handle.clone());
+                    }
+                }
+            }
+        });
+
+        Ok(Self {
+            proxy_path,
+            handle,
+            cancel,
+        })
+    }
+
+    pub fn handle(&self) -> FaultHandle {
+        self.handle.clone()
+    }
+
+    /// Stop accepting and sever active connections.
+    pub fn shutdown(&self) {
+        self.handle.sever_now();
+        self.cancel.cancel();
+    }
+}
+
+impl Drop for UdsProxy {
+    fn drop(&mut self) {
+        self.shutdown();
+        let _ = std::fs::remove_file(&self.proxy_path);
+    }
+}
+
+fn spawn_connection(
+    client: UnixStream,
+    upstream: UnixStream,
+    plan: FaultPlan,
+    handle: FaultHandle,
+) {
+    let (client_read, client_write) = tokio::io::split(client);
+    let (upstream_read, upstream_write) = tokio::io::split(upstream);
+
+    // One sever scope per connection: a mid-frame sever (or `sever_now`)
+    // cancels BOTH pumps so the two half-connections drop together.
+    let conn_cancel = handle.connection_scope();
+
+    let c2l_plan = (plan.direction == FaultDirection::ClientToLeader).then(|| plan.clone());
+    let l2c_plan = (plan.direction == FaultDirection::LeaderToClient).then_some(plan);
+
+    let c2l_counter = handle.clone();
+    let c2l_cancel = conn_cancel.clone();
+    tokio::spawn(async move {
+        pump_frames(
+            client_read,
+            upstream_write,
+            c2l_plan,
+            FaultDirection::ClientToLeader,
+            c2l_counter,
+            c2l_cancel,
+        )
+        .await;
+    });
+
+    let l2c_counter = handle;
+    tokio::spawn(async move {
+        pump_frames(
+            upstream_read,
+            client_write,
+            l2c_plan,
+            FaultDirection::LeaderToClient,
+            l2c_counter,
+            conn_cancel,
+        )
+        .await;
+    });
+}
+
+/// Pump length-prefixed frames from `reader` to `writer`, applying `plan`
+/// (when `Some`) to this direction. Ends on EOF, IO error, or sever.
+async fn pump_frames(
+    mut reader: ReadHalf<UnixStream>,
+    mut writer: WriteHalf<UnixStream>,
+    plan: Option<FaultPlan>,
+    direction: FaultDirection,
+    handle: FaultHandle,
+    cancel: CancellationToken,
+) {
+    let mut frame_index: u64 = 0;
+    loop {
+        let frame = tokio::select! {
+            _ = cancel.cancelled() => break,
+            frame = read_frame(&mut reader) => frame,
+        };
+        let Ok((len_prefix, body)) = frame else {
+            break;
+        };
+        frame_index += 1;
+
+        if let Some(ref plan) = plan {
+            if plan.drop_frame == Some(frame_index) {
+                continue;
+            }
+            if plan.sever_mid_frame == Some(frame_index) {
+                // Half a length prefix, then a hard close of the whole
+                // connection: the reader sees a short read, never a body.
+                let _ = writer.write_all(&len_prefix[..2]).await;
+                let _ = writer.flush().await;
+                cancel.cancel();
+                break;
+            }
+            if let Some((nth, duration)) = plan.delay
+                && nth == frame_index
+            {
+                tokio::select! {
+                    _ = cancel.cancelled() => break,
+                    _ = tokio::time::sleep(duration) => {}
+                }
+            }
+            let copies = if plan.duplicate_frame == Some(frame_index) {
+                2
+            } else {
+                1
+            };
+            for _ in 0..copies {
+                if write_frame(&mut writer, &len_prefix, &body).await.is_err() {
+                    return;
+                }
+                bump_forwarded(&handle, direction);
+            }
+            continue;
+        }
+
+        if write_frame(&mut writer, &len_prefix, &body).await.is_err() {
+            return;
+        }
+        bump_forwarded(&handle, direction);
+    }
+}
+
+fn bump_forwarded(handle: &FaultHandle, direction: FaultDirection) {
+    match direction {
+        FaultDirection::ClientToLeader => {
+            handle.state.forwarded_c2l.fetch_add(1, Ordering::Relaxed);
+        }
+        FaultDirection::LeaderToClient => {
+            handle.state.forwarded_l2c.fetch_add(1, Ordering::Relaxed);
+        }
+    }
+}
+
+/// Max frame body the proxy will buffer — mirrors the leader transport's own
+/// 64 MiB `MAX_MESSAGE_SIZE`, so a corrupt/mis-framed length surfaces as a
+/// readable pump error instead of a multi-GiB allocation.
+const MAX_FRAME_SIZE: usize = 64 * 1024 * 1024;
+
+async fn read_frame(reader: &mut ReadHalf<UnixStream>) -> io::Result<([u8; 4], Vec<u8>)> {
+    let mut len_prefix = [0u8; 4];
+    reader.read_exact(&mut len_prefix).await?;
+    let len = u32::from_be_bytes(len_prefix) as usize;
+    if len > MAX_FRAME_SIZE {
+        return Err(io::Error::new(
+            io::ErrorKind::InvalidData,
+            format!("frame length {len} exceeds MAX_FRAME_SIZE ({MAX_FRAME_SIZE})"),
+        ));
+    }
+    let mut body = vec![0u8; len];
+    reader.read_exact(&mut body).await?;
+    Ok((len_prefix, body))
+}
+
+async fn write_frame(
+    writer: &mut WriteHalf<UnixStream>,
+    len_prefix: &[u8; 4],
+    body: &[u8],
+) -> io::Result<()> {
+    writer.write_all(len_prefix).await?;
+    writer.write_all(body).await?;
+    writer.flush().await
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use tempfile::TempDir;
+
+    async fn client_write_frame(stream: &mut UnixStream, body: &[u8]) {
+        stream
+            .write_all(&(body.len() as u32).to_be_bytes())
+            .await
+            .unwrap();
+        stream.write_all(body).await.unwrap();
+        stream.flush().await.unwrap();
+    }
+
+    async fn client_read_frame(stream: &mut UnixStream) -> io::Result<Vec<u8>> {
+        let mut len = [0u8; 4];
+        stream.read_exact(&mut len).await?;
+        let mut body = vec![0u8; u32::from_be_bytes(len) as usize];
+        stream.read_exact(&mut body).await?;
+        Ok(body)
+    }
+
+    /// Upstream that echoes every frame back to the sender.
+    fn spawn_echo_upstream(path: PathBuf) {
+        let listener = UnixListener::bind(&path).unwrap();
+        tokio::spawn(async move {
+            while let Ok((mut stream, _)) = listener.accept().await {
+                tokio::spawn(async move {
+                    loop {
+                        let mut len = [0u8; 4];
+                        if stream.read_exact(&mut len).await.is_err() {
+                            break;
+                        }
+                        let mut body = vec![0u8; u32::from_be_bytes(len) as usize];
+                        if stream.read_exact(&mut body).await.is_err() {
+                            break;
+                        }
+                        if stream.write_all(&len).await.is_err()
+                            || stream.write_all(&body).await.is_err()
+                        {
+                            break;
+                        }
+                        let _ = stream.flush().await;
+                    }
+                });
+            }
+        });
+    }
+
+    #[tokio::test]
+    async fn passes_frames_through_untouched() {
+        let temp = TempDir::new().unwrap();
+        let upstream_path = temp.path().join("real.sock");
+        spawn_echo_upstream(upstream_path.clone());
+        let proxy = UdsProxy::spawn(
+            temp.path().join("proxy.sock"),
+            &upstream_path,
+            FaultPlan::default(),
+        )
+        .await
+        .unwrap();
+
+        let mut client = UnixStream::connect(&proxy.proxy_path).await.unwrap();
+        for payload in [b"one".as_slice(), b"two", b"three"] {
+            client_write_frame(&mut client, payload).await;
+            assert_eq!(client_read_frame(&mut client).await.unwrap(), payload);
+        }
+        assert_eq!(proxy.handle().forwarded(FaultDirection::ClientToLeader), 3);
+        assert_eq!(proxy.handle().forwarded(FaultDirection::LeaderToClient), 3);
+    }
+
+    #[tokio::test]
+    async fn drops_exactly_the_nth_frame() {
+        let temp = TempDir::new().unwrap();
+        let upstream_path = temp.path().join("real.sock");
+        spawn_echo_upstream(upstream_path.clone());
+        let proxy = UdsProxy::spawn(
+            temp.path().join("proxy.sock"),
+            &upstream_path,
+            FaultPlan {
+                drop_frame: Some(2),
+                ..FaultPlan::default()
+            },
+        )
+        .await
+        .unwrap();
+
+        let mut client = UnixStream::connect(&proxy.proxy_path).await.unwrap();
+        client_write_frame(&mut client, b"first").await;
+        client_write_frame(&mut client, b"second").await;
+        client_write_frame(&mut client, b"third").await;
+
+        // The echo of "second" never arrives; "third" comes straight after "first".
+        assert_eq!(client_read_frame(&mut client).await.unwrap(), b"first");
+        assert_eq!(client_read_frame(&mut client).await.unwrap(), b"third");
+    }
+
+    #[tokio::test]
+    async fn duplicates_exactly_the_nth_frame() {
+        let temp = TempDir::new().unwrap();
+        let upstream_path = temp.path().join("real.sock");
+        spawn_echo_upstream(upstream_path.clone());
+        let proxy = UdsProxy::spawn(
+            temp.path().join("proxy.sock"),
+            &upstream_path,
+            FaultPlan {
+                duplicate_frame: Some(1),
+                ..FaultPlan::default()
+            },
+        )
+        .await
+        .unwrap();
+
+        let mut client = UnixStream::connect(&proxy.proxy_path).await.unwrap();
+        client_write_frame(&mut client, b"once").await;
+        assert_eq!(client_read_frame(&mut client).await.unwrap(), b"once");
+        assert_eq!(client_read_frame(&mut client).await.unwrap(), b"once");
+    }
+
+    #[tokio::test]
+    async fn severs_mid_frame_and_closes_both_sides() {
+        let temp = TempDir::new().unwrap();
+        let upstream_path = temp.path().join("real.sock");
+        spawn_echo_upstream(upstream_path.clone());
+        let proxy = UdsProxy::spawn(
+            temp.path().join("proxy.sock"),
+            &upstream_path,
+            FaultPlan {
+                sever_mid_frame: Some(1),
+                ..FaultPlan::default()
+            },
+        )
+        .await
+        .unwrap();
+
+        let mut client = UnixStream::connect(&proxy.proxy_path).await.unwrap();
+        client_write_frame(&mut client, b"never-delivered").await;
+
+        // The upstream got 2 bytes of a length prefix and then a close, so it
+        // echoes nothing; the client's next read observes the sever.
+        let read = client_read_frame(&mut client).await;
+        assert!(
+            read.is_err(),
+            "sever must close the client side, got {read:?}"
+        );
+    }
+
+    #[tokio::test]
+    async fn delays_exactly_the_nth_frame() {
+        let temp = TempDir::new().unwrap();
+        let upstream_path = temp.path().join("real.sock");
+        spawn_echo_upstream(upstream_path.clone());
+        let delay = Duration::from_millis(300);
+        let proxy = UdsProxy::spawn(
+            temp.path().join("proxy.sock"),
+            &upstream_path,
+            FaultPlan {
+                delay: Some((1, delay)),
+                ..FaultPlan::default()
+            },
+        )
+        .await
+        .unwrap();
+
+        let mut client = UnixStream::connect(&proxy.proxy_path).await.unwrap();
+        let started = std::time::Instant::now();
+        client_write_frame(&mut client, b"held").await;
+        assert_eq!(client_read_frame(&mut client).await.unwrap(), b"held");
+        assert!(
+            started.elapsed() >= delay,
+            "frame must be held for the configured delay"
+        );
+
+        // Only the Nth frame is delayed; the next one is immediate.
+        let started = std::time::Instant::now();
+        client_write_frame(&mut client, b"quick").await;
+        assert_eq!(client_read_frame(&mut client).await.unwrap(), b"quick");
+        assert!(started.elapsed() < delay);
+    }
+
+    #[tokio::test]
+    async fn sever_now_drops_active_connections() {
+        let temp = TempDir::new().unwrap();
+        let upstream_path = temp.path().join("real.sock");
+        spawn_echo_upstream(upstream_path.clone());
+        let proxy = UdsProxy::spawn(
+            temp.path().join("proxy.sock"),
+            &upstream_path,
+            FaultPlan::default(),
+        )
+        .await
+        .unwrap();
+
+        let mut client = UnixStream::connect(&proxy.proxy_path).await.unwrap();
+        client_write_frame(&mut client, b"alive").await;
+        assert_eq!(client_read_frame(&mut client).await.unwrap(), b"alive");
+
+        proxy.handle().sever_now();
+        let read = client_read_frame(&mut client).await;
+        assert!(
+            read.is_err(),
+            "sever_now must close the proxied connection, got {read:?}"
+        );
+    }
+}