Synced from monorepo
Changes: - Non-blocking coding-data sharing upsell banner - Consolidate remediation in Doctor - Auto mode defers fail-closed gate asks to the classifier - Coalesce marketplace list fetches - Allow removing a marketplace source by name - Contain hung git marketplace sources (timeouts, non-blocking refresh, unbrick modal) - Label failed workspace RPCs with error_kind - Drop redundant explicit tonic/prost deps from xai-grok-shell - Report real exit codes for completed background shells - Narrow the date-rollover reminder to date-bearing templates - Wire toolOverrides through the session and agent - Security: Bash(git:*) allowlist matches whole command chain by prefix - Split prompt-trigger telemetry and record classifier provenance - Raise connectors-manager timeout to 60s - Auto classifier honors recorded approvals for repeat actions - Apply doctor fixes in the TUI - Auto-mode classifier timeouts prompt instead of silently denying - Scope subagent completion drains to the owning session - Add the toolOverrides wire types - Set client_identifier=grok-agent-sdk - Accept both spellings of the workspace-teleport kill switch - Persist one-shot occurrence journal - Stop turns that poll the exact same tool call 16x in a row - Copy compaction checkpoint files when forking sessions - Auto-focus permission prompt from scrollback - Esc cancels the running turn in non-vim and minimal modes - List Ctrl+Z undo and redo in keyboard shortcuts - Out-of-process macOS mic capture - Show active auth mode on session-info - Install the npm binary under $GROK_HOME - Remove hover/click dead zones between dashboard items - Route startup warnings to doctor - Document [feedback.user] author identity config - Extend bang command timeout - Close combine-queued edit-hold race - Integrate relocation recovery - Expose privacy notice rollout flag - Break harness discovery ref cycle so connections can idle-evict - Shift/Alt+Enter inserts newline when editing a queued prompt - Gate project Claude permissions on folder trust - Echo response.create.event_id on response.created - Toast when session creation fails from disk full - Add shared test process lifecycle - Enable dynamic workflows by default - Add relocation transaction state machine - Add shared test sandbox - Surface auth failures on model-switch compact - Persist durable scheduler expiry - Confirm before removing extensions-modal items - Re-run compact and prompt after login when compact hit expired auth - Recap sends hosted tools under backend search
This commit is contained in:
grokkybara[bot]
parent
3af4d5d398
commit
a5727c5960
482 changed files with 37627 additions and 13402 deletions
|
|
@ -13,6 +13,7 @@ async-trait = { workspace = true }
|
|||
axum = { workspace = true }
|
||||
clap = { workspace = true }
|
||||
futures-util = { workspace = true }
|
||||
portable-pty = { workspace = true }
|
||||
serde = { workspace = true }
|
||||
serde_json = { workspace = true }
|
||||
tempfile = { workspace = true }
|
||||
|
|
@ -21,7 +22,9 @@ tokio-tungstenite = { workspace = true }
|
|||
tokio-util = { workspace = true, features = ["compat"] }
|
||||
tracing = { workspace = true }
|
||||
tracing-subscriber = { workspace = true, features = ["fmt"] }
|
||||
url = { workspace = true }
|
||||
xai-acp-lib = { workspace = true }
|
||||
xai-tty-utils = { workspace = true }
|
||||
|
||||
[dev-dependencies]
|
||||
reqwest = { workspace = true }
|
||||
|
|
|
|||
|
|
@ -1,10 +1,12 @@
|
|||
# xai-grok-test-support
|
||||
|
||||
Shared test infrastructure for the grok-build crates: mock inference server,
|
||||
SSE wire-format generators, ACP stdio clients, headless
|
||||
runner, and sandboxed process env. Consumed by `xai-grok-shell` integration
|
||||
tests, `xai-grok-pager-pty-harness` (`ContentController`), and `xai-grok-sampler`
|
||||
tests.
|
||||
SSE wire-format generators, ACP stdio clients, headless runner, and the shared
|
||||
`TestSandbox` filesystem/environment plus `TestProcess` subprocess owners. PR3
|
||||
owns test subprocesses only; production spawning, leader protocol, and startup
|
||||
behavior are unchanged. Consumed by `xai-grok-shell`
|
||||
integration tests, `xai-grok-pager-pty-harness` (`ContentController`), and
|
||||
`xai-grok-sampler` tests.
|
||||
|
||||
> **Freshness rule:** update this README in the same PR that changes `src/` —
|
||||
> reviewers should treat a `src/` diff without a README diff as incomplete.
|
||||
|
|
@ -21,20 +23,36 @@ test-support surface.
|
|||
| `mock_server` | `MockInferenceServer` — `/v1/chat/completions`, `/v1/responses`, `/v1/messages`, `/v1/models`, `/v1/settings`, `/v1/user` on `127.0.0.1:0`. `/v1/models` entries are `MockModelEntry` (re-exported as `MockModel` for PTY tests): `new(id)` / `with_agent_type(id, ty)` plus chainable `with_api_backend`, `with_supports_backend_search(bool)` → `supportsBackendSearch`, `with_supports_reasoning_effort(bool)` → `supportsReasoningEffort`, `with_reasoning_effort(&str)` → `reasoningEffort`, `with_reasoning_efforts(Vec<Value>)` → `reasoningEfforts` (raw option tables/bare strings), all emitted top-level as `parse_remote_model_value` reads them. Inference precedence is **matched expectation > compatibility FIFO > required-auth > echo/fixed mode**. Register a uniquely named response with `expect_response(name, InferenceRequestMatcher::{foreground,auxiliary}(InferenceEndpoint::{ChatCompletions,Responses,Messages}), ScriptedResponse)` or `expect_response_blocked`; duplicate names fail at registration and requests atomically claim one matching expectation. Overlapping duplicate requests replay by a deterministic fingerprint of endpoint, request kind, non-empty `x-grok-req-id`, and serialized request body; tool-result follow-ups reuse the turn id but change the body, so they claim the next expectation. Production exposes no explicit HTTP attempt/model-call identity, so completed sequential retries are intentionally not inferred from timing: after the active shared call settles, an identical request claims the next expectation. A foreground request normally carries a non-empty `x-grok-turn-idx`; a non-turn non-empty `x-grok-req-id` is auxiliary even if it uses tools, and empty headers fall through to the 2+-tool compatibility heuristic. The returned `InferenceExpectation` has watch-backed `wait_received`, `wait_blocked`, `release`, `wait_satisfied`, `is_satisfied`, and `assert_satisfied` lifecycle operations. `release` only opens the barrier; response-body/stream-owned RAII publishes `Satisfied` only when the primary crosses terminal and every active overlapping copy settles. Primary cancellation cleans up without satisfaction or replay retention, and dropping a handle safely releases blocked work. Echo (default) streams `Echo: <last user message>` and fixed mode via `set_response(text)` reconstructs bytes exactly. Constructors (`start`, `start_with_models`, `start_with_required_auth`) return `anyhow::Result`. Settings are 404-until-set (`set_settings(impl Serialize)`, `preset_allow_access()` for the `{"allow_access": true}` gate); scripted `/v1/settings` one-shots (`enqueue_response`) take precedence over the steady-state value (stale-snapshot tests). `/v1/user` serves a minimal `UserInfo` whose `subscriptionTier` is controlled by `set_user_subscription_tier(Option<&str>)` (`None` = free); its log entries keep the query string (e.g. `/v1/user?include=subscription`) so subscription-check cadence is countable. Request log: `requests()` (`LogEntry` with body, `authorization`, full POST headers + `header(name)` accessor), `request_bodies()`, `request_count()`, `has_chat_completion_request()` / `has_responses_request()` (exact, per endpoint), `messages_request_count()`, `last_system_prompt()`, `request_log_summary()`. **Storage:** `POST /v1/storage` with flippable 401 (`set_storage_unauthorized`); accepted uploads via `storage_uploads()` → `StorageUpload { path, size, body, authorization }` (`body` retained up to 256 KiB, empty above; `authorization` is the raw header). Runtime knobs: `set_models`, `set_messages_stop_reason`. Shuts down on drop. |
|
||||
| `scripted` | Data-only response bodies (no axum types in the public surface): `SseEvent { event, data }` (`::data`, `::with_event`), `ScriptedBody::{Json, Sse, Raw}` (`Raw` = byte-controllable malformed SSE), `ScriptedResponse { status, headers, body }` (`::sse`, `::json`, `::text`). Prefer request-matched expectations for inference calls; `enqueue_response(path, response)` remains a compatibility FIFO per path and is still used for non-inference one-shots such as `/v1/settings`. Scripted SSE honors `set_chunk_delay`; matched JSON, raw, SSE, and even empty SSE bodies all honor per-expectation completion barriers. The compatibility `hold_agent_completions` gate also covers foreground scripted SSE on all three inference endpoints. Validation is eager — bad status/header panics at registration. |
|
||||
| `sse` | The three wire formats as event-list builders: `chat_completion_events` / `responses_api_events` / `messages_api_events(text, model, stop_reason)` (echo-style, whitespace-collapsing) plus byte-exact axum variants `chat_completion_events_exact` / `responses_api_events_exact` and matching public scripted variants `chat_completion_script_exact` / `responses_api_script_exact` (messages is single-delta, byte-exact by construction). The exact/echo split is load-bearing — see the in-module byte-exactness tests. Also the scripted-scenario builders returning `SseEvent`s (for `ScriptedResponse::sse`): `responses_api_reasoning_only_events(reasoning, model)` — reasoning summary deltas completing with a `reasoning` item but no message/output-text, so the shell collector classifies the turn `EmptyReason::ReasoningOnly` (the model-doomloop trigger); `responses_api_reasoning_and_text_events(reasoning, text, model)` — reasoning deltas then a normal text answer (the ordinary reasoning-model turn); `responses_api_reasoning_then_tool_call_events(reasoning, call_id, name, arguments, model)` + its Chat Completions twin `chat_completions_reasoning_then_tool_call_events(...)` — reasoning deltas then one tool call (the think-then-call turn whose tool call finishes the thought and keeps the turn non-empty); the doom-loop check trio: `responses_api_doom_loop_check_events(triggers, reasoning, model)` — a doomed reasoning-only turn with NAMED `response.doom_loop_check` frames re-sent per cumulative prefix of `triggers` plus the terminal `doom_loop_check.triggers` copy on `response.completed`, `responses_api_doom_loop_terminal_only_events(triggers, reasoning, text, model)` — a normal answer whose terminal response alone carries the field, and `responses_api_with_doom_loop_frame(check_frame_data, reasoning, text, model)` — splices one named check frame with a caller-supplied payload (byte-exact `xai_grok_sampling_types::doom_loop::SAMPLE_CHECK_EVENT_DATA{,_CUMULATIVE}` fixtures or malformed variants) into an ordinary turn. |
|
||||
| `acp_client` | `GrokStdioClient` — drives `grok agent stdio` over real pipes through `agent-client-protocol`: spawn variants (`spawn`, `spawn_with_home`, `spawn_with_home_and_env`, `spawn_with_home_env_and_args`), initialize/authenticate, session create/load, prompt, `*_with_timeout` wrappers, captured text + stderr. `RawStdioClient` — raw-wire sibling for bytes the typed `ClientSideConnection` can never produce (escaped-slash methods `"session\/prompt"`, string UUID ids — the Xcode/Foundation shape): `send_line` writes a line verbatim; `response_for_id` matches the response by exact string id (the match IS the id-echo assertion), skips notifications, auto-refuses agent→client requests with `-32601`, and panics on timeout with skipped-traffic diagnostics (count + last lines; `0 other messages` = true silence). Both spawn through one hermetic `spawn_agent_process` (sandbox env + debug-log kill-list exists once) atop `process::spawn_piped_with_stderr_capture` (crate-internal `process` module: pipes, `kill_on_drop`, stderr drain — also used by `leader::LeaderStdioClient`). |
|
||||
| `headless` | `run_headless(server, args, cwd)` / `run_headless_with_env(server, args, cwd, env)` (extra env applied after the defaults, so it overrides them) / `run_headless_with_cmd(cmd)` → `HeadlessResult { status, stdout, stderr, timed_out }` (60s cap), `assert_headless_success`, `assert_no_crashes` (panic/SIGSEGV/linker patterns), `stderr_tail`. |
|
||||
| `env` | `grok_binary()` (`GROK_BINARY` env → `CARGO_BIN_EXE` → local debug build of `xai-grok-pager`), `git_workdir()` (temp git repo, forces full libgit2 init), `test_env_cmd_tokio(cmd, mock_url, home)` (sandboxed HOME **and GROK_HOME** — Windows resolves `~` via USERPROFILE, so HOME alone doesn't sandbox — + mock endpoints + telemetry kill-switches). |
|
||||
| `leader` | Unix-only `LeaderStdioClient` (`grok agent --leader stdio`, `env_clear`-hermetic, sandboxed `GROK_LEADER_SOCKET`; `spawn_with_binary` runs an explicit binary for version-skew lanes, per-role resolution via `leader_binary()` / `client_binary()` honoring `GROK_BINARY_LEADER` / `GROK_BINARY_CLIENT`) + lock-file helpers: `leader_lock_path`, `read_leader_pid`, `pid_alive`, `wait_for_live_leader`, `wait_for_new_leader`, `wait_for_replay_notifications`, `leader_log`. |
|
||||
| `sandbox` | `TestSandbox` — one owner for a temp root, isolated `HOME`/`USERPROFILE`, explicit `GROK_HOME`, workspace, and `TMPDIR`/`TMP`/`TEMP`. Child commands use `env_clear()` plus a minimal platform allowlist, loopback `NO_PROXY`, interactive-git suppression, telemetry/feedback/trace/instrumentation/updater kill switches, and no ambient leader socket or proxy variables. Unix preserves the host `SHELL` when set and falls back to `/bin/sh`; explicit overrides still win. `TestSandbox::builder().mock_url(url)` wires grok API/models/auxiliary endpoints plus a fake CI key; `.git()` initializes and commits the owned workspace. Bazel test targets that execute Git directly provide `@git_hermetic` runfiles and `GIT_BIN_PATH`; at construction, `TestSandbox` resolves that path against the parent cwd while it is still the Bazel execroot, stores absolute `GIT_BIN_PATH`/`GIT_EXEC_PATH`, and prepends the binary parent to its baseline `PATH`. `TestSandbox::git_command()` applies that cleared environment plus detached, non-interactive Git settings. Without `GIT_BIN_PATH`, ordinary baseline `PATH` is preserved and no special binary/exec vars are added. `set_env`/`extend_env` and `remove_env` are the narrow post-baseline override seam. `diagnostic_summary()` redacts credential-key segments/suffixes and all malformed/non-loopback endpoints; loopback URLs are parsed and stripped of userinfo/query/fragment. |
|
||||
| `process` | `TestProcess` — canonical Tokio child owner stacked over `TestSandbox`: clears/reapplies the sandbox env, applies `pager_env`, enforces null/piped stdin policy, TTY-detaches, owns the pre-PR3 `xai_tty_utils::ProcessGroup`, and captures bounded stdout/stderr tails. Unix detachment establishes the child session/process group before exec; Windows preserves `CREATE_NO_WINDOW` and uses the existing best-effort post-spawn Job attachment without claiming atomic descendant containment. Private Unix `waitid(WNOWAIT)` observes exit so descendants are cleaned before PID/PGID reuse. `wait_with_deadline` is non-destructive; Unix `close` sends SIGTERM then escalates, while Windows uses immediate Job hard-kill policy; Drop synchronously kills and performs a bounded best-effort reap. PID becomes unavailable after reap; status/reason, truncation counters, read/lifecycle errors, and secret-sanitized tails remain cached. `TestProcessTree` is the process-tree adapter for dependencies that retain their concrete child. All lifecycle policy is test-only; production utility behavior and APIs are unchanged. |
|
||||
| `acp_client` | `GrokStdioClient` drives `grok agent stdio` over real pipes through `agent-client-protocol`: `spawn` creates a sandbox, `spawn_with_sandbox` reuses one across restarts, and `spawn_with_sandbox_env_and_args` adds explicit env/global-argument overrides. It exposes initialize/authenticate, session create/load, prompt, `*_with_timeout` wrappers, child PID, captured text/stderr, process diagnostics, explicit close/kill signalling, and `take_sandbox`. `RawStdioClient` is the raw-wire sibling for escaped-slash methods and string UUID ids: exact-id response matching skips notifications, auto-refuses agent→client requests with `-32601`, and reports skipped traffic on timeout. Both keep the sandbox alive while `TestProcess` owns the child tree and pipe-tail diagnostics. |
|
||||
| `headless` | `run_headless[_with_env]` runs grok with an owned canonical `TestSandbox`; `run_headless_in_sandbox[_with_env]` owns a supplied sandbox, while `run_headless_in_sandbox_borrowed[_with_env]` keeps it available for artifact inspection. `_with_env` variants apply explicit last-wins overrides after the hermetic baseline. `TestProcess` owns lifecycle and timeout tree-kill; the scaled 60s process deadline is followed by a separate bounded 2s pipe-drain budget, with retained-pipe or read-task failures returning the bounded partial tail. All variants return `HeadlessResult { status, stdout, stderr, timed_out, elapsed }`. Assertion helpers are `assert_headless_success`, `assert_no_crashes`, and `stderr_tail`. |
|
||||
| `env` | Binary resolution (`grok_binary()`: `GROK_BINARY` → `CARGO_BIN_EXE` → local debug build) and `git_workdir()`, which returns a git-initialized `TestSandbox`; use `.workspace()` for the cwd. |
|
||||
| `leader` | Unix-only `LeaderFixture` is mandatory for every `LeaderStdioClient`. It owns exactly one concrete initial leader and the client objects it directly spawns. Callers close/drop clients first; `LeaderFixture::close` rejects active clients and performs bounded TERM→KILL→reap only on the initial owned child/group. If both graceful and hard client cleanup fail, the test-only unwind containment path requests hard kills and intentionally leaks the retained client/leader owners after signaling; this preserves ownership through panic unwind and is bounded by the test-process lifetime. Lock-file PIDs are observations only: detached replacement generations are never adopted or signaled. Death/re-election and version-skew cases that produce detached replacements remain `leader-acceptance` ignored/manual with tracking language until OS containment or a test-only leader binary can own the whole generation chain. No production marker/protocol/bootstrap behavior is required. |
|
||||
| `uds_proxy` | Unix-only `UdsProxy` — frame-aware (4-byte BE length prefix) man-in-the-middle for leader IPC sockets. `UdsProxy::spawn(proxy_path, upstream_path, FaultPlan)`; `FaultPlan { direction, drop_frame, sever_mid_frame, delay, duplicate_frame }` (1-based frame index, per connection per direction); runtime `FaultHandle::sever_now()` + `forwarded(direction)` counters; frame bodies capped at 64 MiB (leader-transport parity — corrupt lengths error instead of allocating). Zero production changes: point `LeaderClient::connect` / `GROK_LEADER_SOCKET` at the proxy path. |
|
||||
|
||||
## Consumer matrix
|
||||
|
||||
| Consumer | Uses | Notes |
|
||||
|----------|------|-------|
|
||||
| `xai-grok-shell` `tests/*.rs` | Everything | Direct imports (`use xai_grok_test_support::*` or module paths); no local shim. |
|
||||
| `xai-grok-pager-pty-harness` `src/content.rs` | `MockInferenceServer`, `MockModelEntry` (re-exported as `MockModel`) | `ContentController` wraps the server and **keeps the HOME-sandbox `TempDir` + `env_for_pager()` harness-side**; presets `allow_access` + a fixed default response at construction. |
|
||||
| `xai-grok-shell` `tests/*.rs` | `TestSandbox`, `TestProcess` through ACP/leader/headless wrappers, mock server | Binary-driving tests share the same path/env owner; multi-process restart and leader fixtures retain one sandbox across clients. Raw Tokio child ownership is centralized in the wrappers. |
|
||||
| `xai-grok-pager-pty-harness` | `TestSandbox`, `TestProcessTree`, `MockInferenceServer`, `MockModelEntry` | `ContentController` owns the sandbox and server. `spawn_with_content[_env][_in_dir]` applies that sandbox followed by explicit last-wins overrides. OAuth tests use `EnvOp::Remove` for `XAI_API_KEY`; ordinary overrides use `EnvOp::Set`. `portable-pty` remains the concrete child/wait/signal owner; `TestProcessTree` attaches by PID. Unix gets process-group teardown; Windows attachment is best effort, non-atomic, and reported in diagnostics. PTY exit status is cached so every wait is idempotent; PID/signals disappear after reap; Drop uses a bounded direct-child reap wait. |
|
||||
| `xai-grok-sampler` `tests/test_actor.rs` | `sse` generators | Happy-path payloads only; the actor keeps its own router for stall/conditional fixtures. |
|
||||
|
||||
## Sandbox contract
|
||||
|
||||
- Keep the `TestSandbox` alive at least as long as every child using its paths.
|
||||
- Use the builder only for construction-time endpoint/git choices. Use
|
||||
`set_env`/`extend_env` for test-specific flags and terminal brands; the last
|
||||
explicit override wins. Use `remove_env` to test absence.
|
||||
- Do not add process-global env mutation. Keep filesystem/environment ownership
|
||||
in `TestSandbox`; process groups, jobs, output tails, and kill-tree ownership
|
||||
stay in the separate `TestProcess`/`TestProcessTree` harness.
|
||||
- Diagnostics may name sandbox paths and sanitized HTTP(S)/WS(S) loopback URLs.
|
||||
URL parsing fails closed: userinfo/query/fragment are stripped, while malformed
|
||||
or non-loopback values are redacted. Credential-like key segments/suffixes are
|
||||
always redacted.
|
||||
|
||||
## Adding a capability
|
||||
|
||||
**A response mode** (`mock_server.rs`): extend the private `ResponseMode` enum
|
||||
|
|
|
|||
|
|
@ -2,8 +2,8 @@
|
|||
//! [`GrokStdioClient`] (`agent-client-protocol::ClientSideConnection` —
|
||||
//! authentication, session lifecycle, permissions, notification streaming) and
|
||||
//! the raw-wire [`RawStdioClient`] (verbatim JSON-RPC lines for shapes the
|
||||
//! typed client can't produce), plus the shared subprocess spawn/stderr-capture
|
||||
//! plumbing used by every harness in this crate.
|
||||
//! typed client can't produce), all backed by the shared [`TestProcess`]
|
||||
//! lifecycle owner.
|
||||
|
||||
use std::path::Path;
|
||||
use std::sync::Arc;
|
||||
|
|
@ -13,51 +13,50 @@ use std::time::Duration;
|
|||
use crate::scaled;
|
||||
|
||||
use agent_client_protocol::{self as acp, Agent as _};
|
||||
use tempfile::TempDir;
|
||||
use tokio_util::compat::{TokioAsyncReadCompatExt, TokioAsyncWriteCompatExt};
|
||||
use xai_acp_lib::LineBufferedRead;
|
||||
|
||||
use crate::env::{grok_binary, test_env_cmd_tokio};
|
||||
use crate::env::grok_binary;
|
||||
use crate::headless::stderr_tail;
|
||||
use crate::mock_server::MockInferenceServer;
|
||||
use crate::process::spawn_piped_with_stderr_capture;
|
||||
use crate::process::{TestOutput, TestProcess, TestProcessConfig, TestStdin};
|
||||
use crate::sandbox::TestSandbox;
|
||||
|
||||
/// Spawn `grok agent stdio` with the canonical hermetic test env: the sandbox
|
||||
/// from [`test_env_cmd_tokio`] plus the debug-logging kill-list, so the
|
||||
/// hermeticity setup exists exactly once for the typed ([`GrokStdioClient`])
|
||||
/// and raw ([`RawStdioClient`]) harnesses. `leading_args` go before the
|
||||
/// `agent stdio` subcommand (global flags); `extra_env` is applied after the
|
||||
/// kill-list so a test can still set e.g. `GROK_DEBUG_LOG=1` explicitly.
|
||||
/// Spawn `grok agent stdio` with the sandbox's canonical hermetic environment.
|
||||
/// `leading_args` go before the `agent stdio` subcommand (global flags).
|
||||
fn spawn_agent_process(
|
||||
sandbox: &mut TestSandbox,
|
||||
server: &MockInferenceServer,
|
||||
cwd: &Path,
|
||||
home: &Path,
|
||||
extra_env: &[(&str, &str)],
|
||||
leading_args: &[&str],
|
||||
) -> (tokio::process::Child, Arc<std::sync::Mutex<Vec<u8>>>) {
|
||||
let binary = grok_binary();
|
||||
) -> TestProcess {
|
||||
sandbox.set_mock_url(server.url());
|
||||
for (key, value) in extra_env {
|
||||
sandbox.set_env(*key, *value);
|
||||
}
|
||||
|
||||
let binary = grok_binary();
|
||||
let mut cmd = tokio::process::Command::new(&binary);
|
||||
cmd.args(leading_args)
|
||||
.args(["agent", "stdio"])
|
||||
.current_dir(cwd);
|
||||
test_env_cmd_tokio(&mut cmd, &server.url(), home);
|
||||
// Hermetic firehose env: clear inherited debug-logging knobs so a test
|
||||
// controls logging only via `extra_env` / `leading_args` (mirrors the
|
||||
// headless `debug_cmd`).
|
||||
for k in [
|
||||
"GROK_DEBUG_LOG",
|
||||
"GROK_LOG_FILE",
|
||||
"GROK_LOG_SAMPLING",
|
||||
"GROK_HOOKS_LOG",
|
||||
] {
|
||||
cmd.env_remove(k);
|
||||
}
|
||||
for (k, v) in extra_env {
|
||||
cmd.env(k, v);
|
||||
}
|
||||
|
||||
spawn_piped_with_stderr_capture(cmd)
|
||||
TestProcess::spawn(
|
||||
cmd,
|
||||
sandbox,
|
||||
TestProcessConfig::new()
|
||||
.label("grok agent stdio")
|
||||
.stdin(TestStdin::Piped)
|
||||
.stdout(TestOutput::Piped),
|
||||
)
|
||||
.unwrap_or_else(|error| {
|
||||
panic!(
|
||||
"failed to spawn ACP test client at {}: {error}\n{}",
|
||||
binary.display(),
|
||||
sandbox.diagnostic_summary(),
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
|
|
@ -115,49 +114,41 @@ impl acp::Client for TestAcpClient {
|
|||
/// Child process is killed on drop.
|
||||
pub struct GrokStdioClient {
|
||||
conn: acp::ClientSideConnection,
|
||||
_child: tokio::process::Child,
|
||||
home: Option<TempDir>,
|
||||
process: TestProcess,
|
||||
sandbox: Option<TestSandbox>,
|
||||
capture: Arc<TextCapture>,
|
||||
stderr: Arc<std::sync::Mutex<Vec<u8>>>,
|
||||
}
|
||||
|
||||
impl GrokStdioClient {
|
||||
pub async fn spawn(server: &MockInferenceServer, cwd: &Path) -> Self {
|
||||
let home = TempDir::new().expect("create temp home");
|
||||
Self::spawn_with_home(server, cwd, home).await
|
||||
Self::spawn_with_sandbox(server, cwd, TestSandbox::new()).await
|
||||
}
|
||||
|
||||
pub async fn spawn_with_home(server: &MockInferenceServer, cwd: &Path, home: TempDir) -> Self {
|
||||
Self::spawn_with_home_and_env(server, cwd, home, &[]).await
|
||||
}
|
||||
|
||||
/// Like [`spawn_with_home`] but applies extra environment variables to the
|
||||
/// child process (after the standard test env). Used by tests that toggle
|
||||
/// behavior via env vars (e.g. the vendor-compat suite).
|
||||
pub async fn spawn_with_home_and_env(
|
||||
pub async fn spawn_with_sandbox(
|
||||
server: &MockInferenceServer,
|
||||
cwd: &Path,
|
||||
home: TempDir,
|
||||
extra_env: &[(&str, &str)],
|
||||
sandbox: TestSandbox,
|
||||
) -> Self {
|
||||
Self::spawn_with_home_env_and_args(server, cwd, home, extra_env, &[]).await
|
||||
Self::spawn_with_sandbox_env_and_args(server, cwd, sandbox, &[], &[]).await
|
||||
}
|
||||
|
||||
/// Like [`spawn_with_home_and_env`] but also prepends `leading_args` before
|
||||
/// the `agent stdio` subcommand. Used to drive top-level global flags (e.g.
|
||||
/// `--debug`) so a test can exercise the flag's master switch, not just env.
|
||||
pub async fn spawn_with_home_env_and_args(
|
||||
pub async fn spawn_with_sandbox_env_and_args(
|
||||
server: &MockInferenceServer,
|
||||
cwd: &Path,
|
||||
home: TempDir,
|
||||
mut sandbox: TestSandbox,
|
||||
extra_env: &[(&str, &str)],
|
||||
leading_args: &[&str],
|
||||
) -> Self {
|
||||
let (mut child, stderr) =
|
||||
spawn_agent_process(server, cwd, home.path(), extra_env, leading_args);
|
||||
let mut process = spawn_agent_process(&mut sandbox, server, cwd, extra_env, leading_args);
|
||||
|
||||
let outgoing = child.stdin.take().unwrap().compat_write();
|
||||
let incoming = child.stdout.take().unwrap().compat();
|
||||
let outgoing = process
|
||||
.take_stdin()
|
||||
.expect("child stdin missing")
|
||||
.compat_write();
|
||||
let incoming = process
|
||||
.take_stdout()
|
||||
.expect("child stdout missing")
|
||||
.compat();
|
||||
|
||||
let capture = Arc::new(TextCapture::default());
|
||||
let client = TestAcpClient {
|
||||
|
|
@ -172,10 +163,9 @@ impl GrokStdioClient {
|
|||
|
||||
Self {
|
||||
conn,
|
||||
_child: child,
|
||||
home: Some(home),
|
||||
process,
|
||||
sandbox: Some(sandbox),
|
||||
capture,
|
||||
stderr,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -296,16 +286,35 @@ impl GrokStdioClient {
|
|||
}
|
||||
|
||||
pub fn stderr(&self) -> String {
|
||||
String::from_utf8_lossy(&self.stderr.lock().unwrap()).into_owned()
|
||||
self.process.stderr_tail().text
|
||||
}
|
||||
|
||||
pub fn take_home(&mut self) -> TempDir {
|
||||
self.home.take().expect("test home already taken")
|
||||
pub fn child_pid(&self) -> Option<u32> {
|
||||
self.process.pid()
|
||||
}
|
||||
|
||||
/// Return the home directory path (for cache invalidation between phases).
|
||||
pub fn home_path(&self) -> &std::path::Path {
|
||||
self.home.as_ref().expect("test home already taken").path()
|
||||
pub fn process_diagnostics(&self) -> String {
|
||||
self.process.diagnostic_summary()
|
||||
}
|
||||
|
||||
pub fn start_terminate(&mut self) -> std::io::Result<()> {
|
||||
self.process.start_terminate()
|
||||
}
|
||||
|
||||
pub fn start_kill(&mut self) {
|
||||
self.process.start_kill();
|
||||
}
|
||||
|
||||
pub async fn close(&mut self) -> std::io::Result<std::process::ExitStatus> {
|
||||
self.process.close().await
|
||||
}
|
||||
|
||||
pub fn take_sandbox(&mut self) -> TestSandbox {
|
||||
self.sandbox.take().expect("test sandbox already taken")
|
||||
}
|
||||
|
||||
pub fn sandbox(&self) -> &TestSandbox {
|
||||
self.sandbox.as_ref().expect("test sandbox already taken")
|
||||
}
|
||||
|
||||
/// Timing breadcrumb for tuning CI timeout budgets (visible with --nocapture).
|
||||
|
|
@ -423,31 +432,49 @@ impl GrokStdioClient {
|
|||
/// ids. Child process is killed on drop.
|
||||
pub struct RawStdioClient {
|
||||
stdin: tokio::process::ChildStdin,
|
||||
stdout: tokio::io::BufReader<tokio::process::ChildStdout>,
|
||||
stderr: Arc<std::sync::Mutex<Vec<u8>>>,
|
||||
_child: tokio::process::Child,
|
||||
_home: TempDir,
|
||||
stdout: tokio::io::BufReader<crate::process::TestProcessStdout>,
|
||||
process: TestProcess,
|
||||
_sandbox: TestSandbox,
|
||||
}
|
||||
|
||||
impl RawStdioClient {
|
||||
pub async fn spawn(server: &MockInferenceServer, cwd: &Path) -> Self {
|
||||
let home = TempDir::new().expect("create temp home");
|
||||
let (mut child, stderr) = spawn_agent_process(server, cwd, home.path(), &[], &[]);
|
||||
let mut sandbox = TestSandbox::new();
|
||||
let mut process = spawn_agent_process(&mut sandbox, server, cwd, &[], &[]);
|
||||
|
||||
let stdin = child.stdin.take().expect("child stdin missing");
|
||||
let child_stdout = child.stdout.take().expect("child stdout missing");
|
||||
let stdin = process.take_stdin().expect("child stdin missing");
|
||||
let child_stdout = process.take_stdout().expect("child stdout missing");
|
||||
|
||||
Self {
|
||||
stdin,
|
||||
stdout: tokio::io::BufReader::new(child_stdout),
|
||||
stderr,
|
||||
_child: child,
|
||||
_home: home,
|
||||
process,
|
||||
_sandbox: sandbox,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn stderr(&self) -> String {
|
||||
String::from_utf8_lossy(&self.stderr.lock().unwrap()).into_owned()
|
||||
self.process.stderr_tail().text
|
||||
}
|
||||
|
||||
pub fn child_pid(&self) -> Option<u32> {
|
||||
self.process.pid()
|
||||
}
|
||||
|
||||
pub fn process_diagnostics(&self) -> String {
|
||||
self.process.diagnostic_summary()
|
||||
}
|
||||
|
||||
pub fn start_terminate(&mut self) -> std::io::Result<()> {
|
||||
self.process.start_terminate()
|
||||
}
|
||||
|
||||
pub fn start_kill(&mut self) {
|
||||
self.process.start_kill();
|
||||
}
|
||||
|
||||
pub async fn close(&mut self) -> std::io::Result<std::process::ExitStatus> {
|
||||
self.process.close().await
|
||||
}
|
||||
|
||||
/// Write `line` verbatim followed by `\n`, and flush.
|
||||
|
|
|
|||
|
|
@ -1,10 +1,10 @@
|
|||
//! Shared environment helpers: binary resolution, git workdirs, env var setup.
|
||||
//! Binary resolution, serial env guards, and git sandbox creation.
|
||||
|
||||
use std::ffi::{OsStr, OsString};
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::Command;
|
||||
|
||||
use tempfile::TempDir;
|
||||
use crate::sandbox::TestSandbox;
|
||||
|
||||
/// RAII guard for a single environment variable in `#[serial]` tests: snapshots
|
||||
/// the prior value on construction, applies the change, then restores the prior
|
||||
|
|
@ -75,8 +75,8 @@ fn ensure_local_grok_binary(binary: &Path) {
|
|||
}
|
||||
|
||||
let cargo = std::env::var("CARGO").unwrap_or_else(|_| "cargo".to_string());
|
||||
let output = Command::new(&cargo)
|
||||
.current_dir(workspace_root())
|
||||
let mut cmd = Command::new(&cargo);
|
||||
cmd.current_dir(workspace_root())
|
||||
.args([
|
||||
"build",
|
||||
"-p",
|
||||
|
|
@ -84,6 +84,10 @@ fn ensure_local_grok_binary(binary: &Path) {
|
|||
"--bin",
|
||||
"xai-grok-pager",
|
||||
])
|
||||
.stdin(std::process::Stdio::null())
|
||||
.envs(xai_tty_utils::pager_env());
|
||||
xai_tty_utils::detach_std_command(&mut cmd);
|
||||
let output = cmd
|
||||
.output()
|
||||
.unwrap_or_else(|e| panic!("failed to spawn {cargo} to build xai-grok-pager: {e}"));
|
||||
|
||||
|
|
@ -121,63 +125,7 @@ pub fn grok_binary() -> PathBuf {
|
|||
binary
|
||||
}
|
||||
|
||||
/// Temp dir with a git repo + one committed file.
|
||||
/// Forces libgit2 to fully init (the codepath that breaks with bad OpenSSL linking).
|
||||
pub fn git_workdir() -> TempDir {
|
||||
let dir = TempDir::new().expect("create temp dir");
|
||||
let path = dir.path();
|
||||
|
||||
fn run_git(args: &[&str], dir: &Path) {
|
||||
let output = Command::new("git")
|
||||
.args(args)
|
||||
.current_dir(dir)
|
||||
.output()
|
||||
.unwrap_or_else(|e| panic!("failed to spawn git {}: {e}", args.join(" ")));
|
||||
assert!(
|
||||
output.status.success(),
|
||||
"git {} failed (exit {:?}):\n{}",
|
||||
args.join(" "),
|
||||
output.status.code(),
|
||||
String::from_utf8_lossy(&output.stderr),
|
||||
);
|
||||
}
|
||||
|
||||
run_git(&["init"], path);
|
||||
// Configure git user for commits (required in CI where no global config exists)
|
||||
run_git(&["config", "user.email", "test@test.com"], path);
|
||||
run_git(&["config", "user.name", "Test"], path);
|
||||
|
||||
std::fs::write(path.join("README.md"), "test file\n").expect("write test file");
|
||||
|
||||
run_git(&["add", "-A"], path);
|
||||
run_git(&["commit", "-m", "init", "--no-gpg-sign"], path);
|
||||
|
||||
dir
|
||||
}
|
||||
|
||||
/// Point grok at the mock server with a fake API key and telemetry disabled.
|
||||
pub fn test_env_cmd_tokio(
|
||||
cmd: &mut tokio::process::Command,
|
||||
mock_url: &str,
|
||||
home: &std::path::Path,
|
||||
) {
|
||||
cmd.env("HOME", home)
|
||||
// HOME alone does not sandbox grok on Windows: the product resolves
|
||||
// `~` via `USERPROFILE`/Known Folders (`std::env::home_dir()`), so
|
||||
// without an explicit GROK_HOME every spawned child shares the real
|
||||
// `%USERPROFILE%\.grok` — test 1's models_cache.json (which embeds
|
||||
// its per-test mock-server URL) then poisons every later test's
|
||||
// prompt (the windows-x86_64 lifecycle "prompt timed out" failure).
|
||||
// Mirrors `leader.rs` and the pty-harness `env_for_pager`.
|
||||
.env("GROK_HOME", home.join(".grok"))
|
||||
.env("GROK_CLI_CHAT_PROXY_BASE_URL", mock_url)
|
||||
.env("GROK_XAI_API_BASE_URL", mock_url)
|
||||
.env("XAI_API_KEY", "test-key-for-ci")
|
||||
.env("GROK_TELEMETRY_ENABLED", "false")
|
||||
.env("GROK_FEEDBACK_ENABLED", "false")
|
||||
.env("GROK_TRACE_UPLOAD", "false")
|
||||
.env("GROK_INSTRUMENTATION", "disabled")
|
||||
// Release binaries (CI lifecycle tests) otherwise spawn a background
|
||||
// update check that hits the network and can add latency under Rosetta.
|
||||
.env("GROK_DISABLE_AUTOUPDATER", "1");
|
||||
/// Create an owned, git-initialized [`TestSandbox`].
|
||||
pub fn git_workdir() -> TestSandbox {
|
||||
TestSandbox::builder().git().build()
|
||||
}
|
||||
|
|
|
|||
|
|
@ -2,23 +2,24 @@
|
|||
//!
|
||||
//! Runs the grok binary as a subprocess with the mock server, captures output.
|
||||
|
||||
use std::path::Path;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::ExitStatus;
|
||||
use std::time::Duration;
|
||||
|
||||
use tempfile::TempDir;
|
||||
use tokio::io::AsyncReadExt as _;
|
||||
|
||||
use crate::env::{grok_binary, test_env_cmd_tokio};
|
||||
use crate::env::grok_binary;
|
||||
use crate::mock_server::MockInferenceServer;
|
||||
use crate::process::{TestOutput, TestProcess, TestProcessConfig};
|
||||
use crate::sandbox::TestSandbox;
|
||||
|
||||
pub struct HeadlessResult {
|
||||
pub status: ExitStatus,
|
||||
pub stdout: String,
|
||||
pub stderr: String,
|
||||
pub timed_out: bool,
|
||||
/// Wall time of the grok invocation; logged so CI timeout budgets can be
|
||||
/// tuned against observed durations.
|
||||
/// Wall time of the headless command invocation; logged so CI timeout
|
||||
/// budgets can be tuned against observed durations.
|
||||
pub elapsed: Duration,
|
||||
}
|
||||
|
||||
|
|
@ -28,8 +29,10 @@ fn headless_timeout() -> Duration {
|
|||
crate::scaled(Duration::from_secs(60))
|
||||
}
|
||||
|
||||
/// Run `grok` with the given args against the mock server, bounded by
|
||||
/// [`headless_timeout_secs`]. Uses an isolated HOME and disables telemetry.
|
||||
const HEADLESS_DRAIN_TIMEOUT: Duration = Duration::from_secs(2);
|
||||
|
||||
/// Run `grok` with the given args against the mock server, bounded by the
|
||||
/// scaled headless timeout. Uses an isolated HOME and disables telemetry.
|
||||
pub async fn run_headless(
|
||||
server: &MockInferenceServer,
|
||||
args: &[&str],
|
||||
|
|
@ -39,94 +42,188 @@ pub async fn run_headless(
|
|||
}
|
||||
|
||||
/// Like [`run_headless`], but with extra environment variables applied after the
|
||||
/// shared defaults so they take precedence — e.g. to re-enable a feature the
|
||||
/// defaults turn off.
|
||||
/// sandbox baseline so they take precedence — e.g. to re-enable a feature the
|
||||
/// baseline turns off.
|
||||
pub async fn run_headless_with_env(
|
||||
server: &MockInferenceServer,
|
||||
args: &[&str],
|
||||
cwd: &Path,
|
||||
env: &[(&str, &str)],
|
||||
) -> HeadlessResult {
|
||||
let home = TempDir::new().expect("create temp home");
|
||||
let sandbox = TestSandbox::builder().mock_url(server.url()).build();
|
||||
let mut cmd = tokio::process::Command::new(grok_binary());
|
||||
cmd.args(args)
|
||||
.current_dir(cwd)
|
||||
.stdin(std::process::Stdio::null())
|
||||
.stdout(std::process::Stdio::piped())
|
||||
.stderr(std::process::Stdio::piped())
|
||||
.kill_on_drop(true);
|
||||
test_env_cmd_tokio(&mut cmd, &server.url(), home.path());
|
||||
cmd.envs(env.iter().copied());
|
||||
run_headless_with_cmd(cmd).await
|
||||
cmd.args(args).current_dir(cwd);
|
||||
run_headless_with_cmd_and_sandbox(cmd, &sandbox, env).await
|
||||
}
|
||||
|
||||
pub async fn run_headless_with_cmd(mut cmd: tokio::process::Command) -> HeadlessResult {
|
||||
let binary = grok_binary();
|
||||
/// Apply and retain one [`TestSandbox`] while running a custom headless command.
|
||||
pub async fn run_headless_in_sandbox(
|
||||
cmd: tokio::process::Command,
|
||||
sandbox: TestSandbox,
|
||||
) -> HeadlessResult {
|
||||
run_headless_in_sandbox_with_env(cmd, sandbox, &[]).await
|
||||
}
|
||||
|
||||
pub async fn run_headless_in_sandbox_with_env(
|
||||
cmd: tokio::process::Command,
|
||||
sandbox: TestSandbox,
|
||||
overrides: &[(&str, &str)],
|
||||
) -> HeadlessResult {
|
||||
run_headless_in_sandbox_borrowed_with_env(cmd, &sandbox, overrides).await
|
||||
}
|
||||
|
||||
/// Run a custom headless command while leaving the caller's sandbox available
|
||||
/// for post-run artifact inspection.
|
||||
pub async fn run_headless_in_sandbox_borrowed(
|
||||
cmd: tokio::process::Command,
|
||||
sandbox: &TestSandbox,
|
||||
) -> HeadlessResult {
|
||||
run_headless_in_sandbox_borrowed_with_env(cmd, sandbox, &[]).await
|
||||
}
|
||||
|
||||
pub async fn run_headless_in_sandbox_borrowed_with_env(
|
||||
cmd: tokio::process::Command,
|
||||
sandbox: &TestSandbox,
|
||||
overrides: &[(&str, &str)],
|
||||
) -> HeadlessResult {
|
||||
run_headless_with_cmd_and_sandbox(cmd, sandbox, overrides).await
|
||||
}
|
||||
|
||||
async fn run_headless_with_cmd_and_sandbox(
|
||||
cmd: tokio::process::Command,
|
||||
sandbox: &TestSandbox,
|
||||
overrides: &[(&str, &str)],
|
||||
) -> HeadlessResult {
|
||||
let program = PathBuf::from(cmd.as_std().get_program());
|
||||
let started = std::time::Instant::now();
|
||||
let mut child = cmd
|
||||
.spawn()
|
||||
.unwrap_or_else(|e| panic!("failed to spawn grok binary at {}: {e}", binary.display()));
|
||||
|
||||
let stdout = child.stdout.take().expect("child stdout missing");
|
||||
let stderr = child.stderr.take().expect("child stderr missing");
|
||||
let mut process = TestProcess::spawn(
|
||||
cmd,
|
||||
sandbox,
|
||||
TestProcessConfig::new()
|
||||
.label(format!("headless command {}", program.display()))
|
||||
.stdout(TestOutput::Piped)
|
||||
.stderr(TestOutput::Piped)
|
||||
.envs(overrides.iter().copied()),
|
||||
)
|
||||
.unwrap_or_else(|error| {
|
||||
panic!(
|
||||
"failed to spawn headless command at {}: {error}\n{}",
|
||||
program.display(),
|
||||
sandbox.diagnostic_summary(),
|
||||
)
|
||||
});
|
||||
|
||||
let mut stdout = process.take_stdout().expect("child stdout missing");
|
||||
let stdout_handle = tokio::spawn(async move {
|
||||
let mut stdout = stdout;
|
||||
let mut stdout_buf = Vec::new();
|
||||
stdout.read_to_end(&mut stdout_buf).await?;
|
||||
Ok::<Vec<u8>, std::io::Error>(stdout_buf)
|
||||
let mut bytes = Vec::new();
|
||||
stdout.read_to_end(&mut bytes).await?;
|
||||
Ok::<Vec<u8>, std::io::Error>(bytes)
|
||||
});
|
||||
let mut stderr = process.take_stderr().expect("child stderr missing");
|
||||
let stderr_handle = tokio::spawn(async move {
|
||||
let mut stderr = stderr;
|
||||
let mut stderr_buf = Vec::new();
|
||||
stderr.read_to_end(&mut stderr_buf).await?;
|
||||
Ok::<Vec<u8>, std::io::Error>(stderr_buf)
|
||||
let mut bytes = Vec::new();
|
||||
stderr.read_to_end(&mut bytes).await?;
|
||||
Ok::<Vec<u8>, std::io::Error>(bytes)
|
||||
});
|
||||
|
||||
let (status, timed_out) = match tokio::time::timeout(headless_timeout(), child.wait()).await {
|
||||
Ok(result) => (
|
||||
result.unwrap_or_else(|e| {
|
||||
panic!("failed to wait for grok binary {}: {e}", binary.display())
|
||||
}),
|
||||
false,
|
||||
),
|
||||
Err(_) => {
|
||||
let _ = child.kill().await;
|
||||
let status = child.wait().await.unwrap_or_else(|e| {
|
||||
let (status, timed_out) = match process
|
||||
.wait_with_deadline(headless_timeout())
|
||||
.await
|
||||
.unwrap_or_else(|error| {
|
||||
panic!(
|
||||
"failed to wait for headless command {}: {error}\n{}",
|
||||
program.display(),
|
||||
process.diagnostic_summary(),
|
||||
)
|
||||
}) {
|
||||
Some(status) => (status, false),
|
||||
None => {
|
||||
let status = process.kill().await.unwrap_or_else(|error| {
|
||||
panic!(
|
||||
"failed to kill timed out grok binary {}: {e}",
|
||||
binary.display()
|
||||
"failed to kill timed out headless command {}: {error}\n{}",
|
||||
program.display(),
|
||||
process.diagnostic_summary(),
|
||||
)
|
||||
});
|
||||
(status, true)
|
||||
}
|
||||
};
|
||||
|
||||
let stdout_bytes = match stdout_handle.await {
|
||||
Ok(Ok(bytes)) => bytes,
|
||||
Ok(Err(err)) => panic!("failed to read stdout from {}: {err}", binary.display()),
|
||||
Err(err) => panic!("stdout task join failed for {}: {err}", binary.display()),
|
||||
};
|
||||
let stderr_bytes = match stderr_handle.await {
|
||||
Ok(Ok(bytes)) => bytes,
|
||||
Ok(Err(err)) => panic!("failed to read stderr from {}: {err}", binary.display()),
|
||||
Err(err) => panic!("stderr task join failed for {}: {err}", binary.display()),
|
||||
};
|
||||
let stdout = finish_output_drain(
|
||||
stdout_handle,
|
||||
process.stdout_tail().text,
|
||||
"stdout",
|
||||
&program,
|
||||
&process,
|
||||
)
|
||||
.await;
|
||||
let stderr = finish_output_drain(
|
||||
stderr_handle,
|
||||
process.stderr_tail().text,
|
||||
"stderr",
|
||||
&program,
|
||||
&process,
|
||||
)
|
||||
.await;
|
||||
|
||||
let elapsed = started.elapsed();
|
||||
// Timing breadcrumb for tuning CI timeout budgets against observed
|
||||
// durations (visible with --nocapture).
|
||||
eprintln!("[harness-timing] headless grok run: {elapsed:?} (timed_out={timed_out})");
|
||||
eprintln!(
|
||||
"[harness-timing] headless command {}: {elapsed:?} (timed_out={timed_out})",
|
||||
program.display()
|
||||
);
|
||||
|
||||
HeadlessResult {
|
||||
status,
|
||||
stdout: String::from_utf8_lossy(&stdout_bytes).into_owned(),
|
||||
stderr: String::from_utf8_lossy(&stderr_bytes).into_owned(),
|
||||
stdout,
|
||||
stderr,
|
||||
timed_out,
|
||||
elapsed,
|
||||
}
|
||||
}
|
||||
|
||||
async fn finish_output_drain(
|
||||
mut handle: tokio::task::JoinHandle<std::io::Result<Vec<u8>>>,
|
||||
partial_tail: String,
|
||||
stream: &str,
|
||||
program: &Path,
|
||||
process: &TestProcess,
|
||||
) -> String {
|
||||
match tokio::time::timeout(HEADLESS_DRAIN_TIMEOUT, &mut handle).await {
|
||||
Ok(Ok(Ok(bytes))) => String::from_utf8_lossy(&bytes).into_owned(),
|
||||
Ok(Ok(Err(error))) => {
|
||||
tracing::warn!(
|
||||
%stream,
|
||||
program = %program.display(),
|
||||
%error,
|
||||
"headless output drain failed; returning captured partial tail"
|
||||
);
|
||||
partial_tail
|
||||
}
|
||||
Ok(Err(error)) => {
|
||||
tracing::warn!(
|
||||
%stream,
|
||||
program = %program.display(),
|
||||
%error,
|
||||
"headless output task failed; returning captured partial tail"
|
||||
);
|
||||
partial_tail
|
||||
}
|
||||
Err(_) => {
|
||||
handle.abort();
|
||||
let _ = handle.await;
|
||||
tracing::warn!(
|
||||
%stream,
|
||||
program = %program.display(),
|
||||
diagnostics = %process.diagnostic_summary(),
|
||||
"headless output drain timed out; returning captured partial tail"
|
||||
);
|
||||
partial_tail
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const CRASH_PATTERNS: &[&str] = &[
|
||||
"panicked at",
|
||||
"SIGSEGV",
|
||||
|
|
@ -175,3 +272,90 @@ pub fn assert_no_crashes(stderr: &str) {
|
|||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[cfg(unix)]
|
||||
#[tokio::test]
|
||||
async fn borrowed_runner_keeps_sandbox_artifacts_available() {
|
||||
let sandbox = TestSandbox::new();
|
||||
let artifact = sandbox.temp_dir().join("borrowed-headless.txt");
|
||||
let script = format!("printf kept > '{}'", artifact.display());
|
||||
let mut cmd = tokio::process::Command::new("/bin/sh");
|
||||
cmd.args(["-c", &script]);
|
||||
|
||||
let result = run_headless_in_sandbox_borrowed(cmd, &sandbox).await;
|
||||
|
||||
assert!(result.status.success(), "stderr: {}", result.stderr);
|
||||
assert_eq!(std::fs::read_to_string(artifact).unwrap(), "kept");
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
#[tokio::test]
|
||||
async fn output_drain_timeout_returns_partial_capture() {
|
||||
let (tx, rx) = tokio::sync::oneshot::channel::<()>();
|
||||
let handle = tokio::spawn(async move {
|
||||
let _keep_open = tx;
|
||||
let _ = rx.await;
|
||||
Ok::<Vec<u8>, std::io::Error>(b"complete".to_vec())
|
||||
});
|
||||
let sandbox = TestSandbox::new();
|
||||
let mut command = tokio::process::Command::new("/bin/sh");
|
||||
command.args(["-c", "exit 0"]);
|
||||
let mut process = TestProcess::spawn(
|
||||
command,
|
||||
&sandbox,
|
||||
TestProcessConfig::new().label("headless-drain-test"),
|
||||
)
|
||||
.expect("spawn drain fixture");
|
||||
process
|
||||
.wait_with_deadline(Duration::from_secs(2))
|
||||
.await
|
||||
.expect("wait fixture")
|
||||
.expect("fixture exits");
|
||||
|
||||
let output = finish_output_drain(
|
||||
handle,
|
||||
"partial".to_owned(),
|
||||
"stdout",
|
||||
Path::new("fixture"),
|
||||
&process,
|
||||
)
|
||||
.await;
|
||||
assert_eq!(output, "partial");
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
#[tokio::test]
|
||||
async fn custom_headless_env_is_explicit_and_wins_after_sandbox_baseline() {
|
||||
let sandbox = TestSandbox::new();
|
||||
let mut cmd = tokio::process::Command::new("/bin/sh");
|
||||
cmd.args([
|
||||
"-c",
|
||||
"printf '%s|%s|%s|%s' \"${AMBIENT_ONLY-unset}\" \"$GROK_PROMPT_SUGGESTIONS\" \"$FEATURE_TEST_VAR\" \"$HOME\"",
|
||||
])
|
||||
.env("AMBIENT_ONLY", "discarded")
|
||||
.env("GROK_PROMPT_SUGGESTIONS", "command-level-discarded");
|
||||
|
||||
let result = run_headless_in_sandbox_borrowed_with_env(
|
||||
cmd,
|
||||
&sandbox,
|
||||
&[
|
||||
("GROK_PROMPT_SUGGESTIONS", "explicit-override"),
|
||||
("FEATURE_TEST_VAR", "enabled"),
|
||||
],
|
||||
)
|
||||
.await;
|
||||
|
||||
assert!(result.status.success(), "stderr: {}", result.stderr);
|
||||
assert_eq!(
|
||||
result.stdout,
|
||||
format!(
|
||||
"unset|explicit-override|enabled|{}",
|
||||
sandbox.home().display()
|
||||
)
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,13 +1,13 @@
|
|||
//! Leader-mode (`grok agent --leader stdio`) test harness.
|
||||
//!
|
||||
//! Spawns the real binary as a stdio client whose bridge elects a leader
|
||||
//! subprocess hosting the actual sessions, speaks ACP over pipes, and
|
||||
//! exposes lock-file helpers for leader-lifecycle assertions. Unix-only:
|
||||
//! the leader transport is a unix socket.
|
||||
//! The fixture owns only subprocess handles it created: one initial persistent
|
||||
//! leader and each returned stdio client. Lock-file PIDs are observations only;
|
||||
//! detached replacement generations are never adopted or signaled.
|
||||
|
||||
use std::io::{self, ErrorKind};
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::Arc;
|
||||
use std::sync::atomic::{AtomicU32, Ordering};
|
||||
use std::sync::{Arc, Mutex, Weak};
|
||||
use std::time::Duration;
|
||||
|
||||
use agent_client_protocol::{self as acp, Agent as _};
|
||||
|
|
@ -16,7 +16,8 @@ use xai_acp_lib::LineBufferedRead;
|
|||
|
||||
use crate::env::grok_binary;
|
||||
use crate::mock_server::MockInferenceServer;
|
||||
use crate::process::spawn_piped_with_stderr_capture;
|
||||
use crate::process::{TestOutput, TestProcess, TestProcessConfig, TestProcessTree, TestStdin};
|
||||
use crate::sandbox::TestSandbox;
|
||||
|
||||
/// Env var naming the binary that elects/hosts the leader in a two-binary
|
||||
/// (version-skew) test. Falls back to [`grok_binary`]'s resolution.
|
||||
|
|
@ -28,26 +29,28 @@ pub const CLIENT_BINARY_ENV: &str = "GROK_BINARY_CLIENT";
|
|||
|
||||
fn role_binary(env_key: &str) -> PathBuf {
|
||||
if let Ok(path) = std::env::var(env_key) {
|
||||
let p = PathBuf::from(path);
|
||||
assert!(p.exists(), "{env_key} does not exist: {}", p.display());
|
||||
return p;
|
||||
let path = PathBuf::from(path);
|
||||
assert!(
|
||||
path.exists(),
|
||||
"{env_key} does not exist: {}",
|
||||
path.display()
|
||||
);
|
||||
return path;
|
||||
}
|
||||
grok_binary()
|
||||
}
|
||||
|
||||
/// Binary for the leader-electing side of a version-skew test
|
||||
/// (`GROK_BINARY_LEADER`, else the shared [`grok_binary`] resolution).
|
||||
/// Binary for the leader-electing side of a version-skew test.
|
||||
pub fn leader_binary() -> PathBuf {
|
||||
role_binary(LEADER_BINARY_ENV)
|
||||
}
|
||||
|
||||
/// Binary for the client side of a version-skew test (`GROK_BINARY_CLIENT`,
|
||||
/// else the shared [`grok_binary`] resolution).
|
||||
/// Binary for the client side of a version-skew test.
|
||||
pub fn client_binary() -> PathBuf {
|
||||
role_binary(CLIENT_BINARY_ENV)
|
||||
}
|
||||
|
||||
/// Capture for notifications + reconnect signals.
|
||||
/// Capture for notifications and reconnect signals.
|
||||
#[derive(Default)]
|
||||
pub struct Capture {
|
||||
chunks: std::sync::Mutex<Vec<String>>,
|
||||
|
|
@ -68,11 +71,11 @@ impl acp::Client for LeaderAcpClient {
|
|||
let outcome = args
|
||||
.options
|
||||
.iter()
|
||||
.find(|o| o.kind == acp::PermissionOptionKind::AllowOnce)
|
||||
.find(|option| option.kind == acp::PermissionOptionKind::AllowOnce)
|
||||
.or(args.options.first())
|
||||
.map(|o| {
|
||||
.map(|option| {
|
||||
acp::RequestPermissionOutcome::Selected(acp::SelectedPermissionOutcome::new(
|
||||
o.option_id.clone(),
|
||||
option.option_id.clone(),
|
||||
))
|
||||
})
|
||||
.unwrap_or(acp::RequestPermissionOutcome::Cancelled);
|
||||
|
|
@ -85,9 +88,9 @@ impl acp::Client for LeaderAcpClient {
|
|||
.fetch_add(1, Ordering::SeqCst);
|
||||
if let acp::SessionUpdate::AgentMessageChunk(acp::ContentChunk { content, .. }) =
|
||||
args.update
|
||||
&& let acp::ContentBlock::Text(t) = content
|
||||
&& let acp::ContentBlock::Text(text) = content
|
||||
{
|
||||
self.capture.chunks.lock().unwrap().push(t.text);
|
||||
self.capture.chunks.lock().unwrap().push(text.text);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
|
@ -102,81 +105,550 @@ impl acp::Client for LeaderAcpClient {
|
|||
}
|
||||
}
|
||||
|
||||
/// A `grok agent --leader stdio` client subprocess speaking ACP over pipes.
|
||||
/// The leader subprocess it elects hosts the actual sessions.
|
||||
pub struct LeaderStdioClient {
|
||||
pub conn: acp::ClientSideConnection,
|
||||
// Exposed for PID assertions.
|
||||
pub child: tokio::process::Child,
|
||||
capture: Arc<Capture>,
|
||||
stderr: Arc<std::sync::Mutex<Vec<u8>>>,
|
||||
/// Owns the concrete initial persistent leader shared by a test's clients.
|
||||
///
|
||||
/// Production-created replacement generations are outside this fixture's
|
||||
/// ownership. [`Self::wait_for_new_leader`] may observe one for assertions but
|
||||
/// never turns its lock-file PID into signal authority.
|
||||
pub struct LeaderFixture {
|
||||
inner: Arc<Mutex<LeaderFixtureState>>,
|
||||
}
|
||||
|
||||
impl LeaderStdioClient {
|
||||
pub async fn spawn(server: &MockInferenceServer, cwd: &Path, home: &Path) -> Self {
|
||||
Self::spawn_with_binary(&grok_binary(), server, cwd, home).await
|
||||
struct LeaderFixtureState {
|
||||
binary: PathBuf,
|
||||
socket: PathBuf,
|
||||
lock: PathBuf,
|
||||
active_clients: usize,
|
||||
leader: Option<PersistentLeader>,
|
||||
}
|
||||
|
||||
struct PersistentLeader {
|
||||
child: std::process::Child,
|
||||
tree: TestProcessTree,
|
||||
pid: u32,
|
||||
}
|
||||
|
||||
struct FixtureClientRegistration {
|
||||
fixture: Weak<Mutex<LeaderFixtureState>>,
|
||||
}
|
||||
|
||||
impl FixtureClientRegistration {
|
||||
fn new(fixture: &Arc<Mutex<LeaderFixtureState>>) -> Self {
|
||||
fixture
|
||||
.lock()
|
||||
.unwrap_or_else(|error| error.into_inner())
|
||||
.active_clients += 1;
|
||||
Self {
|
||||
fixture: Arc::downgrade(fixture),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for FixtureClientRegistration {
|
||||
fn drop(&mut self) {
|
||||
if let Some(fixture) = self.fixture.upgrade() {
|
||||
let mut fixture = fixture.lock().unwrap_or_else(|error| error.into_inner());
|
||||
fixture.active_clients = fixture.active_clients.saturating_sub(1);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A `grok agent --leader stdio` client subprocess speaking ACP over pipes.
|
||||
pub struct LeaderStdioClient {
|
||||
pub conn: acp::ClientSideConnection,
|
||||
process: TestProcess,
|
||||
capture: Arc<Capture>,
|
||||
registration: Option<FixtureClientRegistration>,
|
||||
}
|
||||
|
||||
impl LeaderFixture {
|
||||
/// Start one concrete persistent leader under the shared sandbox.
|
||||
pub async fn start(
|
||||
server: &MockInferenceServer,
|
||||
cwd: &Path,
|
||||
sandbox: &TestSandbox,
|
||||
) -> io::Result<Self> {
|
||||
Self::start_with_binary(&grok_binary(), server, cwd, sandbox).await
|
||||
}
|
||||
|
||||
/// [`Self::spawn`] with an explicit binary, for two-binary version-skew
|
||||
/// tests (pair with [`leader_binary`] / [`client_binary`]).
|
||||
pub async fn spawn_with_binary(
|
||||
pub async fn start_with_binary(
|
||||
binary: &Path,
|
||||
server: &MockInferenceServer,
|
||||
cwd: &Path,
|
||||
home: &Path,
|
||||
) -> Self {
|
||||
let mut cmd = tokio::process::Command::new(binary);
|
||||
cmd.args(["agent", "--leader", "stdio"])
|
||||
.current_dir(cwd)
|
||||
// Hermetic env: the developer's shell may export GROK_* vars
|
||||
// (e.g. GROK_LEADER_SOCKET pointing at a REAL leader on this
|
||||
// machine). env_clear + explicit allowlist guarantees the test
|
||||
// can never touch a leader outside its sandbox home.
|
||||
.env_clear()
|
||||
.env("PATH", std::env::var("PATH").unwrap_or_default())
|
||||
.env("HOME", home)
|
||||
.env("GROK_HOME", home.join(".grok"))
|
||||
// Pin the socket inside the sandbox. The lock file is the
|
||||
// sibling `.lock` (leader.sock -> leader.lock), and the spawned
|
||||
// leader subprocess inherits/forwards this env var, so every
|
||||
// (re-)elected leader binds the same sandboxed path.
|
||||
.env("GROK_LEADER_SOCKET", home.join(".grok").join("leader.sock"))
|
||||
sandbox: &TestSandbox,
|
||||
) -> io::Result<Self> {
|
||||
Self::start_with_binary_timeout(binary, server, cwd, sandbox, Duration::from_secs(30)).await
|
||||
}
|
||||
|
||||
async fn start_with_binary_timeout(
|
||||
binary: &Path,
|
||||
server: &MockInferenceServer,
|
||||
cwd: &Path,
|
||||
sandbox: &TestSandbox,
|
||||
readiness_timeout: Duration,
|
||||
) -> io::Result<Self> {
|
||||
let socket = sandbox.grok_home().join("leader.sock");
|
||||
let lock = sandbox.grok_home().join("leader.lock");
|
||||
let mut cmd = std::process::Command::new(binary);
|
||||
cmd.args([
|
||||
"agent",
|
||||
"leader",
|
||||
"--no-exit-on-disconnect",
|
||||
"--relay-on-demand",
|
||||
"--no-auto-update",
|
||||
])
|
||||
.current_dir(cwd)
|
||||
.stdin(std::process::Stdio::null())
|
||||
.stdout(std::process::Stdio::null());
|
||||
sandbox.apply_to_std_command(&mut cmd);
|
||||
cmd.envs(xai_tty_utils::pager_env())
|
||||
.env("GROK_CLI_CHAT_PROXY_BASE_URL", server.url())
|
||||
.env("GROK_XAI_API_BASE_URL", server.url())
|
||||
.env("GROK_MODELS_BASE_URL", server.url())
|
||||
.env("GROK_FEEDBACK_BASE_URL", server.url())
|
||||
.env("GROK_TRACE_UPLOAD_URL", server.url())
|
||||
.env("XAI_API_KEY", "test-key-for-ci")
|
||||
.env("GROK_TELEMETRY_ENABLED", "false")
|
||||
.env("GROK_FEEDBACK_ENABLED", "false")
|
||||
.env("GROK_TRACE_UPLOAD", "false")
|
||||
.env("GROK_INSTRUMENTATION", "disabled")
|
||||
// Inherited by the spawned leader, whose stderr goes to
|
||||
// ~/.grok/leader.log — keep it chatty for diagnosis.
|
||||
.env("GROK_LEADER_SOCKET", &socket)
|
||||
.env("RUST_LOG", "xai_grok_shell=debug");
|
||||
let log_path = sandbox.grok_home().join("leader.log");
|
||||
match std::fs::File::create(&log_path) {
|
||||
Ok(log) => {
|
||||
cmd.stderr(log);
|
||||
}
|
||||
Err(_) => {
|
||||
cmd.stderr(std::process::Stdio::null());
|
||||
}
|
||||
}
|
||||
xai_tty_utils::detach_std_command(&mut cmd);
|
||||
#[allow(clippy::disallowed_methods)]
|
||||
let mut child = cmd.spawn()?;
|
||||
let pid = child.id();
|
||||
let tree = match TestProcessTree::try_attach(pid, "persistent grok test leader") {
|
||||
Ok(tree) => tree,
|
||||
Err(error) => {
|
||||
let _ = child.kill();
|
||||
let _ = wait_std_child_bounded(&mut child, Duration::from_secs(1));
|
||||
return Err(error);
|
||||
}
|
||||
};
|
||||
let fixture = Self {
|
||||
inner: Arc::new(Mutex::new(LeaderFixtureState {
|
||||
binary: binary.to_path_buf(),
|
||||
socket,
|
||||
lock,
|
||||
active_clients: 0,
|
||||
leader: Some(PersistentLeader { child, tree, pid }),
|
||||
})),
|
||||
};
|
||||
fixture.finish_start(readiness_timeout).await
|
||||
}
|
||||
|
||||
let (mut child, stderr) = spawn_piped_with_stderr_capture(cmd);
|
||||
async fn finish_start(self, timeout: Duration) -> io::Result<Self> {
|
||||
if let Err(error) = self.wait_ready(timeout).await {
|
||||
let cleanup = self.close().await;
|
||||
return Err(match cleanup {
|
||||
Ok(()) => error,
|
||||
Err(cleanup) => io::Error::new(
|
||||
error.kind(),
|
||||
format!("{error}; readiness cleanup also failed: {cleanup}"),
|
||||
),
|
||||
});
|
||||
}
|
||||
Ok(self)
|
||||
}
|
||||
|
||||
let outgoing = child.stdin.take().unwrap().compat_write();
|
||||
let incoming = child.stdout.take().unwrap().compat();
|
||||
pub async fn spawn_client(
|
||||
&self,
|
||||
server: &MockInferenceServer,
|
||||
cwd: &Path,
|
||||
sandbox: &TestSandbox,
|
||||
) -> io::Result<LeaderStdioClient> {
|
||||
let binary = self
|
||||
.inner
|
||||
.lock()
|
||||
.unwrap_or_else(|error| error.into_inner())
|
||||
.binary
|
||||
.clone();
|
||||
self.spawn_client_with_binary(&binary, server, cwd, sandbox)
|
||||
.await
|
||||
}
|
||||
|
||||
pub async fn spawn_client_with_binary(
|
||||
&self,
|
||||
binary: &Path,
|
||||
server: &MockInferenceServer,
|
||||
cwd: &Path,
|
||||
sandbox: &TestSandbox,
|
||||
) -> io::Result<LeaderStdioClient> {
|
||||
let socket = self
|
||||
.inner
|
||||
.lock()
|
||||
.unwrap_or_else(|error| error.into_inner())
|
||||
.socket
|
||||
.clone();
|
||||
let registration = FixtureClientRegistration::new(&self.inner);
|
||||
LeaderStdioClient::spawn_with_binary_and_socket(
|
||||
binary,
|
||||
server,
|
||||
cwd,
|
||||
sandbox,
|
||||
socket,
|
||||
registration,
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
/// PID of the concrete initial leader while the fixture still owns it.
|
||||
pub fn leader_pid(&self) -> Option<u32> {
|
||||
self.inner
|
||||
.lock()
|
||||
.unwrap_or_else(|error| error.into_inner())
|
||||
.leader
|
||||
.as_ref()
|
||||
.map(|leader| leader.pid)
|
||||
}
|
||||
|
||||
/// Observe a replacement PID from the lock file without adopting it.
|
||||
pub async fn wait_for_new_leader(&self, old_pid: u32, timeout: Duration) -> io::Result<u32> {
|
||||
let lock = self
|
||||
.inner
|
||||
.lock()
|
||||
.unwrap_or_else(|error| error.into_inner())
|
||||
.lock
|
||||
.clone();
|
||||
let deadline = tokio::time::Instant::now() + timeout;
|
||||
loop {
|
||||
if let Some(pid) = read_pid_path(&lock)
|
||||
&& pid != old_pid
|
||||
&& pid_alive(pid)
|
||||
{
|
||||
return Ok(pid);
|
||||
}
|
||||
if tokio::time::Instant::now() >= deadline {
|
||||
return Err(io::Error::new(
|
||||
ErrorKind::TimedOut,
|
||||
format!("no replacement leader appeared after pid {old_pid}"),
|
||||
));
|
||||
}
|
||||
tokio::time::sleep(Duration::from_millis(50)).await;
|
||||
}
|
||||
}
|
||||
|
||||
/// Hard-kill only the concrete initial leader spawned by this fixture.
|
||||
pub fn kill_current_concrete_leader(&self) -> io::Result<u32> {
|
||||
let mut state = self.inner.lock().unwrap_or_else(|error| error.into_inner());
|
||||
let leader = state
|
||||
.leader
|
||||
.as_mut()
|
||||
.ok_or_else(|| io::Error::other("leader fixture no longer owns an initial leader"))?;
|
||||
let tree_result = leader.tree.kill();
|
||||
let child_result = leader.child.kill();
|
||||
if let Err(error) = tree_result
|
||||
&& !is_missing_process_error(&error)
|
||||
{
|
||||
return Err(error);
|
||||
}
|
||||
if let Err(error) = child_result
|
||||
&& !is_missing_process_error(&error)
|
||||
{
|
||||
return Err(error);
|
||||
}
|
||||
Ok(leader.pid)
|
||||
}
|
||||
|
||||
/// Reap the concrete initial leader after a crash test killed it.
|
||||
pub async fn reap_exited_concrete_leaders(&self) -> io::Result<()> {
|
||||
let state = self.inner.clone();
|
||||
tokio::task::spawn_blocking(move || {
|
||||
let mut state = state.lock().unwrap_or_else(|error| error.into_inner());
|
||||
let Some(leader) = state.leader.as_mut() else {
|
||||
return Ok(());
|
||||
};
|
||||
reap_exited_persistent_leader(leader, Duration::from_secs(2))?;
|
||||
state.leader = None;
|
||||
Ok(())
|
||||
})
|
||||
.await
|
||||
.map_err(|error| io::Error::other(format!("leader reap task: {error}")))?
|
||||
}
|
||||
|
||||
/// On failed test cleanup, hard-kill the concrete initial leader and leak
|
||||
/// its already-signaled owner so unwind cannot run blocking Drop cleanup.
|
||||
/// Lock-file and detached replacement PIDs are never consulted or signaled.
|
||||
pub fn contain_failed_cleanup_for_unwind(&self) {
|
||||
let leader = self
|
||||
.inner
|
||||
.lock()
|
||||
.unwrap_or_else(|error| error.into_inner())
|
||||
.leader
|
||||
.take();
|
||||
if let Some(mut leader) = leader {
|
||||
let _ = leader.tree.kill();
|
||||
let _ = leader.child.kill();
|
||||
std::mem::forget(leader);
|
||||
}
|
||||
}
|
||||
|
||||
/// Close the directly-owned clients first, then shut down the concrete
|
||||
/// initial leader. Detached replacements are intentionally untouched.
|
||||
pub async fn close(&self) -> io::Result<()> {
|
||||
let state = self.inner.clone();
|
||||
tokio::task::spawn_blocking(move || {
|
||||
let mut state = state.lock().unwrap_or_else(|error| error.into_inner());
|
||||
if state.active_clients != 0 {
|
||||
return Err(io::Error::other(format!(
|
||||
"cannot close leader fixture while {} directly-owned client(s) remain; close/drop clients first",
|
||||
state.active_clients
|
||||
)));
|
||||
}
|
||||
let Some(leader) = state.leader.as_mut() else {
|
||||
return Ok(());
|
||||
};
|
||||
shutdown_persistent_leader(leader)?;
|
||||
state.leader = None;
|
||||
Ok(())
|
||||
})
|
||||
.await
|
||||
.map_err(|error| io::Error::other(format!("leader cleanup task: {error}")))?
|
||||
}
|
||||
|
||||
async fn wait_ready(&self, timeout: Duration) -> io::Result<()> {
|
||||
let (socket, pid) = {
|
||||
let state = self.inner.lock().unwrap_or_else(|error| error.into_inner());
|
||||
let leader = state
|
||||
.leader
|
||||
.as_ref()
|
||||
.expect("leader fixture missing concrete owner");
|
||||
(state.socket.clone(), leader.pid)
|
||||
};
|
||||
let deadline = tokio::time::Instant::now() + timeout;
|
||||
loop {
|
||||
if socket.exists() && pid_alive(pid) {
|
||||
return Ok(());
|
||||
}
|
||||
if tokio::time::Instant::now() >= deadline {
|
||||
return Err(io::Error::new(
|
||||
ErrorKind::TimedOut,
|
||||
format!(
|
||||
"persistent leader pid {pid} did not become ready at {}",
|
||||
socket.display()
|
||||
),
|
||||
));
|
||||
}
|
||||
tokio::time::sleep(Duration::from_millis(50)).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for LeaderFixture {
|
||||
fn drop(&mut self) {
|
||||
let leader = self
|
||||
.inner
|
||||
.lock()
|
||||
.unwrap_or_else(|error| error.into_inner())
|
||||
.leader
|
||||
.take();
|
||||
if let Some(mut leader) = leader {
|
||||
let _ = shutdown_persistent_leader(&mut leader);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn shutdown_persistent_leader(leader: &mut PersistentLeader) -> io::Result<()> {
|
||||
const GRACE: Duration = Duration::from_secs(2);
|
||||
const HARD_WAIT: Duration = Duration::from_secs(2);
|
||||
|
||||
if crate::process::process_has_exited_without_reap(leader.pid, "persistent leader")? {
|
||||
return reap_exited_persistent_leader(leader, HARD_WAIT);
|
||||
}
|
||||
|
||||
if let Err(error) = leader.tree.terminate()
|
||||
&& !is_missing_process_error(&error)
|
||||
{
|
||||
return Err(error);
|
||||
}
|
||||
if !wait_std_child_exit_without_reap(&leader.child, GRACE)? {
|
||||
if let Err(error) = leader.tree.kill()
|
||||
&& !is_missing_process_error(&error)
|
||||
{
|
||||
return Err(error);
|
||||
}
|
||||
if let Err(error) = leader.child.kill()
|
||||
&& !is_missing_process_error(&error)
|
||||
{
|
||||
return Err(error);
|
||||
}
|
||||
if !wait_std_child_exit_without_reap(&leader.child, HARD_WAIT)? {
|
||||
return Err(io::Error::new(
|
||||
ErrorKind::TimedOut,
|
||||
format!("persistent leader pid {} did not exit", leader.pid),
|
||||
));
|
||||
}
|
||||
}
|
||||
reap_exited_persistent_leader(leader, HARD_WAIT)
|
||||
}
|
||||
|
||||
fn reap_exited_persistent_leader(
|
||||
leader: &mut PersistentLeader,
|
||||
timeout: Duration,
|
||||
) -> io::Result<()> {
|
||||
if !wait_std_child_exit_without_reap(&leader.child, timeout)? {
|
||||
return Err(io::Error::new(
|
||||
ErrorKind::TimedOut,
|
||||
format!("persistent leader pid {} did not exit", leader.pid),
|
||||
));
|
||||
}
|
||||
// macOS may report EPERM when the group contains only the unreaped zombie
|
||||
// leader. The direct child is already known exited; attempt descendant
|
||||
// cleanup while its PGID is reserved, then revoke before consuming status.
|
||||
// Focused tests separately prove a live descendant is removed.
|
||||
let _ = leader.tree.kill();
|
||||
leader.tree.release();
|
||||
leader.child.wait().map(|_| ())
|
||||
}
|
||||
|
||||
fn wait_std_child_exit_without_reap(
|
||||
child: &std::process::Child,
|
||||
timeout: Duration,
|
||||
) -> io::Result<bool> {
|
||||
let deadline = std::time::Instant::now() + timeout;
|
||||
loop {
|
||||
if crate::process::process_has_exited_without_reap(child.id(), "persistent leader")? {
|
||||
return Ok(true);
|
||||
}
|
||||
if std::time::Instant::now() >= deadline {
|
||||
return Ok(false);
|
||||
}
|
||||
std::thread::sleep(Duration::from_millis(10));
|
||||
}
|
||||
}
|
||||
|
||||
fn is_missing_process_error(error: &io::Error) -> bool {
|
||||
matches!(error.raw_os_error(), Some(code) if code == libc::ESRCH || code == libc::ECHILD)
|
||||
}
|
||||
|
||||
fn read_pid_path(path: &Path) -> Option<u32> {
|
||||
std::fs::read_to_string(path).ok()?.trim().parse().ok()
|
||||
}
|
||||
|
||||
fn wait_std_child_bounded(
|
||||
child: &mut std::process::Child,
|
||||
timeout: Duration,
|
||||
) -> io::Result<Option<std::process::ExitStatus>> {
|
||||
let deadline = std::time::Instant::now() + timeout;
|
||||
loop {
|
||||
if let Some(status) = child.try_wait()? {
|
||||
return Ok(Some(status));
|
||||
}
|
||||
if std::time::Instant::now() >= deadline {
|
||||
return Ok(None);
|
||||
}
|
||||
std::thread::sleep(Duration::from_millis(10));
|
||||
}
|
||||
}
|
||||
|
||||
impl LeaderStdioClient {
|
||||
async fn spawn_with_binary_and_socket(
|
||||
binary: &Path,
|
||||
server: &MockInferenceServer,
|
||||
cwd: &Path,
|
||||
sandbox: &TestSandbox,
|
||||
leader_socket: PathBuf,
|
||||
registration: FixtureClientRegistration,
|
||||
) -> io::Result<Self> {
|
||||
let mut cmd = tokio::process::Command::new(binary);
|
||||
cmd.args(["agent", "--leader", "stdio"]).current_dir(cwd);
|
||||
let mut process = TestProcess::spawn(
|
||||
cmd,
|
||||
sandbox,
|
||||
TestProcessConfig::new()
|
||||
.label("grok leader stdio client")
|
||||
.stdin(TestStdin::Piped)
|
||||
.stdout(TestOutput::Piped)
|
||||
.env("GROK_CLI_CHAT_PROXY_BASE_URL", server.url())
|
||||
.env("GROK_XAI_API_BASE_URL", server.url())
|
||||
.env("GROK_MODELS_BASE_URL", server.url())
|
||||
.env("GROK_FEEDBACK_BASE_URL", server.url())
|
||||
.env("GROK_TRACE_UPLOAD_URL", server.url())
|
||||
.env("XAI_API_KEY", "test-key-for-ci")
|
||||
.env("GROK_LEADER_SOCKET", leader_socket)
|
||||
.env("RUST_LOG", "xai_grok_shell=debug"),
|
||||
)
|
||||
.map_err(|error| {
|
||||
io::Error::new(
|
||||
error.kind(),
|
||||
format!(
|
||||
"failed to spawn leader stdio client at {}: {error}\n{}",
|
||||
binary.display(),
|
||||
sandbox.diagnostic_summary(),
|
||||
),
|
||||
)
|
||||
})?;
|
||||
|
||||
let outgoing = process
|
||||
.take_stdin()
|
||||
.ok_or_else(|| io::Error::other("leader stdio client stdin pipe missing"))?
|
||||
.compat_write();
|
||||
let incoming = process
|
||||
.take_stdout()
|
||||
.ok_or_else(|| io::Error::other("leader stdio client stdout pipe missing"))?
|
||||
.compat();
|
||||
|
||||
let capture = Arc::new(Capture::default());
|
||||
let client = LeaderAcpClient {
|
||||
capture: capture.clone(),
|
||||
};
|
||||
let incoming = LineBufferedRead::spawn_local(incoming);
|
||||
let (conn, handle_io) = acp::ClientSideConnection::new(client, outgoing, incoming, |fut| {
|
||||
tokio::task::spawn_local(fut);
|
||||
});
|
||||
let (conn, handle_io) =
|
||||
acp::ClientSideConnection::new(client, outgoing, incoming, |future| {
|
||||
tokio::task::spawn_local(future);
|
||||
});
|
||||
tokio::task::spawn_local(handle_io);
|
||||
|
||||
Self {
|
||||
Ok(Self {
|
||||
conn,
|
||||
child,
|
||||
process,
|
||||
capture,
|
||||
stderr,
|
||||
}
|
||||
registration: Some(registration),
|
||||
})
|
||||
}
|
||||
|
||||
pub fn child_pid(&self) -> Option<u32> {
|
||||
self.process.pid()
|
||||
}
|
||||
|
||||
pub fn stderr_text(&self) -> String {
|
||||
String::from_utf8_lossy(&self.stderr.lock().unwrap()).into_owned()
|
||||
self.process.stderr_tail().text
|
||||
}
|
||||
|
||||
pub fn process_diagnostics(&self) -> String {
|
||||
self.process.diagnostic_summary()
|
||||
}
|
||||
|
||||
pub fn start_terminate(&mut self) -> io::Result<()> {
|
||||
self.process.start_terminate()
|
||||
}
|
||||
|
||||
pub fn start_kill(&mut self) {
|
||||
self.process.start_kill();
|
||||
}
|
||||
|
||||
/// Request a nonblocking hard kill while retaining concrete process and
|
||||
/// fixture-registration ownership for unwind containment.
|
||||
pub fn contain_failed_cleanup_for_unwind(&mut self) {
|
||||
self.process.start_kill();
|
||||
}
|
||||
|
||||
pub async fn close(&mut self) -> io::Result<std::process::ExitStatus> {
|
||||
let status = self.process.close().await?;
|
||||
self.registration.take();
|
||||
Ok(status)
|
||||
}
|
||||
|
||||
pub async fn kill_and_close(&mut self) -> io::Result<std::process::ExitStatus> {
|
||||
let status = self.process.kill().await?;
|
||||
self.registration.take();
|
||||
Ok(status)
|
||||
}
|
||||
|
||||
pub fn captured_text(&self) -> String {
|
||||
|
|
@ -215,7 +687,7 @@ impl LeaderStdioClient {
|
|||
let api_key_method = init
|
||||
.auth_methods
|
||||
.iter()
|
||||
.find(|m| &*m.id().0 == "xai.api_key")
|
||||
.find(|method| &*method.id().0 == "xai.api_key")
|
||||
.expect("xai.api_key auth method");
|
||||
self.conn
|
||||
.authenticate(
|
||||
|
|
@ -296,10 +768,12 @@ pub fn read_leader_pid(home: &Path) -> Option<u32> {
|
|||
}
|
||||
|
||||
pub fn pid_alive(pid: u32) -> bool {
|
||||
unsafe { libc::kill(pid as i32, 0) == 0 }
|
||||
// SAFETY: signal 0 performs an existence/permission check only.
|
||||
let result = unsafe { libc::kill(pid as libc::pid_t, 0) };
|
||||
result == 0 || io::Error::last_os_error().raw_os_error() == Some(libc::EPERM)
|
||||
}
|
||||
|
||||
/// Wait until the leader lock file contains a live PID, return it.
|
||||
/// Wait until the leader lock file contains a live PID.
|
||||
pub async fn wait_for_live_leader(home: &Path, timeout: Duration) -> Option<u32> {
|
||||
let deadline = tokio::time::Instant::now() + timeout;
|
||||
while tokio::time::Instant::now() < deadline {
|
||||
|
|
@ -313,27 +787,7 @@ pub async fn wait_for_live_leader(home: &Path, timeout: Duration) -> Option<u32>
|
|||
None
|
||||
}
|
||||
|
||||
/// Wait until the leader lock file contains a live PID *different* from `old_pid`.
|
||||
pub async fn wait_for_new_leader(home: &Path, old_pid: u32, timeout: Duration) -> Option<u32> {
|
||||
let deadline = tokio::time::Instant::now() + timeout;
|
||||
while tokio::time::Instant::now() < deadline {
|
||||
if let Some(pid) = read_leader_pid(home)
|
||||
&& pid != old_pid
|
||||
&& pid_alive(pid)
|
||||
{
|
||||
return Some(pid);
|
||||
}
|
||||
tokio::time::sleep(Duration::from_millis(100)).await;
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// Wait for evidence that the bridge finished its reconnect replay.
|
||||
///
|
||||
/// The `x.ai/leader_reconnected` ext notification is dropped by the typed
|
||||
/// `ClientSideConnection` (bare `x.ai/*` methods are rejected by the ACP
|
||||
/// decoder), so we wait for the replayed `session/load` to emit session
|
||||
/// notifications instead: the notification count rises above `baseline`.
|
||||
pub async fn wait_for_replay_notifications(
|
||||
client: &LeaderStdioClient,
|
||||
baseline: u32,
|
||||
|
|
@ -352,3 +806,115 @@ pub async fn wait_for_replay_notifications(
|
|||
pub fn leader_log(home: &Path) -> String {
|
||||
std::fs::read_to_string(home.join(".grok").join("leader.log")).unwrap_or_default()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn fake_leader(script: &str) -> PersistentLeader {
|
||||
let mut cmd = std::process::Command::new("/bin/sh");
|
||||
cmd.args(["-c", script])
|
||||
.stdin(std::process::Stdio::null())
|
||||
.stdout(std::process::Stdio::null())
|
||||
.stderr(std::process::Stdio::null())
|
||||
.envs(xai_tty_utils::pager_env());
|
||||
xai_tty_utils::detach_std_command(&mut cmd);
|
||||
let child = cmd.spawn().expect("spawn fake persistent leader");
|
||||
let pid = child.id();
|
||||
let tree = TestProcessTree::try_attach(pid, "fake persistent leader")
|
||||
.expect("attach fake persistent leader");
|
||||
PersistentLeader { child, tree, pid }
|
||||
}
|
||||
|
||||
fn fixture(root: &Path, leader: PersistentLeader) -> LeaderFixture {
|
||||
LeaderFixture {
|
||||
inner: Arc::new(Mutex::new(LeaderFixtureState {
|
||||
binary: PathBuf::from("fixture"),
|
||||
socket: root.join("leader.sock"),
|
||||
lock: root.join("leader.lock"),
|
||||
active_clients: 0,
|
||||
leader: Some(leader),
|
||||
})),
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn close_terminates_and_reaps_directly_owned_leader() {
|
||||
let temp = tempfile::tempdir().expect("tempdir");
|
||||
let leader = fake_leader("trap 'exit 0' TERM; while :; do sleep 1; done");
|
||||
let pid = leader.pid;
|
||||
let fixture = fixture(temp.path(), leader);
|
||||
|
||||
fixture.close().await.expect("close fixture");
|
||||
|
||||
assert!(!pid_alive(pid));
|
||||
assert!(fixture.inner.lock().unwrap().leader.is_none());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn active_direct_client_registration_blocks_fixture_close() {
|
||||
let temp = tempfile::tempdir().expect("tempdir");
|
||||
let fixture = fixture(
|
||||
temp.path(),
|
||||
fake_leader("trap 'exit 0' TERM; while :; do sleep 1; done"),
|
||||
);
|
||||
let registration = FixtureClientRegistration::new(&fixture.inner);
|
||||
|
||||
let error = fixture
|
||||
.close()
|
||||
.await
|
||||
.expect_err("active client must block close");
|
||||
assert!(error.to_string().contains("close/drop clients first"));
|
||||
|
||||
drop(registration);
|
||||
fixture.close().await.expect("close after client drop");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn lock_file_replacement_pid_is_never_adopted_or_signaled() {
|
||||
let temp = tempfile::tempdir().expect("tempdir");
|
||||
let initial = fake_leader("trap 'exit 0' TERM; while :; do sleep 1; done");
|
||||
let initial_pid = initial.pid;
|
||||
let mut replacement = fake_leader("trap 'exit 0' TERM; while :; do sleep 1; done");
|
||||
let replacement_pid = replacement.pid;
|
||||
std::fs::write(temp.path().join("leader.lock"), replacement_pid.to_string())
|
||||
.expect("replacement lock");
|
||||
let fixture = fixture(temp.path(), initial);
|
||||
|
||||
drop(fixture);
|
||||
|
||||
assert!(!pid_alive(initial_pid));
|
||||
assert!(
|
||||
pid_alive(replacement_pid),
|
||||
"observed replacement must remain untouched"
|
||||
);
|
||||
shutdown_persistent_leader(&mut replacement).expect("clean replacement test owner");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn close_hard_kills_term_ignoring_descendant() {
|
||||
let temp = tempfile::tempdir().expect("tempdir");
|
||||
let pid_file = temp.path().join("descendant.pid");
|
||||
let script = format!(
|
||||
"trap 'exit 0' TERM; sh -c 'trap \"\" TERM; echo $$ > {}; while :; do sleep 1; done' & while :; do sleep 1; done",
|
||||
pid_file.display()
|
||||
);
|
||||
let fixture = fixture(temp.path(), fake_leader(&script));
|
||||
let deadline = tokio::time::Instant::now() + Duration::from_secs(2);
|
||||
while !pid_file.exists() && tokio::time::Instant::now() < deadline {
|
||||
tokio::time::sleep(Duration::from_millis(10)).await;
|
||||
}
|
||||
let descendant: u32 = std::fs::read_to_string(&pid_file)
|
||||
.expect("descendant pid")
|
||||
.trim()
|
||||
.parse()
|
||||
.expect("parse descendant pid");
|
||||
|
||||
fixture.close().await.expect("close fixture");
|
||||
let deadline = tokio::time::Instant::now() + Duration::from_secs(2);
|
||||
while pid_alive(descendant) && tokio::time::Instant::now() < deadline {
|
||||
tokio::time::sleep(Duration::from_millis(10)).await;
|
||||
}
|
||||
assert!(!pid_alive(descendant), "descendant {descendant} leaked");
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
dead_code
|
||||
)]
|
||||
//! Shared test utilities for grok-build crates: mock inference server, SSE
|
||||
//! generators, ACP stdio client, headless runner, env sandbox.
|
||||
//! generators, ACP stdio client, headless runner, env/process sandbox.
|
||||
//!
|
||||
//! Provides:
|
||||
//! - [`MockInferenceServer`] — Mock /v1/chat/completions + /v1/responses with request logging
|
||||
|
|
@ -14,8 +14,10 @@
|
|||
//! - [`RawStdioClient`] — raw-wire ACP driver for bytes the typed client can't
|
||||
//! produce (Foundation `\/` methods, string UUID ids)
|
||||
//! - [`leader::LeaderStdioClient`] — ACP client that drives `grok agent --leader stdio` (unix)
|
||||
//! - [`TestSandbox`] — Own isolated paths, hermetic child env, optional git setup, diagnostics
|
||||
//! - [`TestProcess`] — Own detached child lifecycle, process-tree teardown, bounded output tails
|
||||
//! - [`run_headless`] — Run `grok -p` against the mock server and capture output
|
||||
//! - [`git_workdir`] — Create a temp directory with git repo (forces libgit2 init)
|
||||
//! - [`git_workdir`] — Create a git-initialized [`TestSandbox`]
|
||||
//! - [`grok_binary`] — Resolve the grok binary path (GROK_BINARY env or cargo_bin)
|
||||
//! - [`spawn_counting_server`] — Connection-counting HTTP/1.1 server for wire/pooling tests
|
||||
//! - [`uds_proxy::UdsProxy`] — Frame-aware fault-injection proxy for leader IPC sockets (unix)
|
||||
|
|
@ -38,7 +40,8 @@ mod inference_override;
|
|||
#[cfg(unix)]
|
||||
pub mod leader;
|
||||
pub mod mock_server;
|
||||
mod process;
|
||||
pub mod process;
|
||||
pub mod sandbox;
|
||||
pub mod scripted;
|
||||
pub mod sse;
|
||||
#[cfg(unix)]
|
||||
|
|
@ -48,9 +51,20 @@ pub use counting_server::spawn_counting_server;
|
|||
pub use env::{EnvGuard, git_workdir, grok_binary};
|
||||
pub use headless::{
|
||||
HeadlessResult, assert_headless_success, assert_no_crashes, run_headless,
|
||||
run_headless_with_cmd, run_headless_with_env, stderr_tail,
|
||||
run_headless_in_sandbox, run_headless_in_sandbox_borrowed,
|
||||
run_headless_in_sandbox_borrowed_with_env, run_headless_in_sandbox_with_env,
|
||||
run_headless_with_env, stderr_tail,
|
||||
};
|
||||
pub use inference_override::{InferenceEndpoint, InferenceExpectation, InferenceRequestMatcher};
|
||||
#[cfg(unix)]
|
||||
pub use leader::LeaderFixture;
|
||||
pub use mock_server::{
|
||||
MockInferenceServer, MockModelEntry, ScriptedResponse, SseEvent, StorageUpload,
|
||||
};
|
||||
#[cfg(unix)]
|
||||
pub use process::process_has_exited_without_reap;
|
||||
pub use process::{
|
||||
TestOutput, TestOutputSnapshot, TestProcess, TestProcessConfig, TestProcessState,
|
||||
TestProcessStderr, TestProcessStdout, TestProcessTermination, TestProcessTree, TestStdin,
|
||||
};
|
||||
pub use sandbox::{TestSandbox, TestSandboxBuilder};
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
919
crates/codegen/xai-grok-test-support/src/sandbox.rs
Normal file
919
crates/codegen/xai-grok-test-support/src/sandbox.rs
Normal file
|
|
@ -0,0 +1,919 @@
|
|||
//! Hermetic filesystem and child-environment owner for grok integration tests.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::ffi::{OsStr, OsString};
|
||||
use std::fmt::Write as _;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::{Command, Stdio};
|
||||
|
||||
use tempfile::TempDir;
|
||||
|
||||
const TEST_API_KEY: &str = "test-key-for-ci";
|
||||
const REDACTED: &str = "<redacted>";
|
||||
|
||||
/// One test's isolated filesystem tree and canonical child environment.
|
||||
///
|
||||
/// Construction never mutates the process environment. Child commands start
|
||||
/// from `env_clear()` and receive only platform essentials, sandbox paths,
|
||||
/// grok network kill switches, and explicit overrides.
|
||||
pub struct TestSandbox {
|
||||
root: TempDir,
|
||||
home: PathBuf,
|
||||
grok_home: PathBuf,
|
||||
workspace: PathBuf,
|
||||
temp: PathBuf,
|
||||
env: BTreeMap<OsString, OsString>,
|
||||
}
|
||||
|
||||
impl TestSandbox {
|
||||
/// Create an isolated non-git workspace with no mock endpoint configured.
|
||||
pub fn new() -> Self {
|
||||
Self::builder().build()
|
||||
}
|
||||
|
||||
/// Configure construction-time sandbox options.
|
||||
pub fn builder() -> TestSandboxBuilder {
|
||||
TestSandboxBuilder::default()
|
||||
}
|
||||
|
||||
/// Temp root owning every sandbox path.
|
||||
pub fn root(&self) -> &Path {
|
||||
self.root.path()
|
||||
}
|
||||
|
||||
/// Isolated `HOME` / `USERPROFILE`.
|
||||
pub fn home(&self) -> &Path {
|
||||
&self.home
|
||||
}
|
||||
|
||||
/// Explicit grok state root.
|
||||
pub fn grok_home(&self) -> &Path {
|
||||
&self.grok_home
|
||||
}
|
||||
|
||||
/// Isolated working directory. When built with [`TestSandboxBuilder::git`],
|
||||
/// this contains a repository with one committed `README.md`.
|
||||
pub fn workspace(&self) -> &Path {
|
||||
&self.workspace
|
||||
}
|
||||
|
||||
/// Isolated `TMPDIR` / `TMP` / `TEMP`.
|
||||
pub fn temp_dir(&self) -> &Path {
|
||||
&self.temp
|
||||
}
|
||||
|
||||
/// Override one child variable after the hermetic baseline. This is the
|
||||
/// supported seam for feature flags and simulated terminal brands.
|
||||
pub fn set_env(&mut self, key: impl AsRef<OsStr>, value: impl AsRef<OsStr>) -> &mut Self {
|
||||
self.env
|
||||
.insert(key.as_ref().to_owned(), value.as_ref().to_owned());
|
||||
self
|
||||
}
|
||||
|
||||
/// Apply several explicit child overrides in order; later duplicate keys win.
|
||||
pub fn extend_env<I, K, V>(&mut self, overrides: I) -> &mut Self
|
||||
where
|
||||
I: IntoIterator<Item = (K, V)>,
|
||||
K: AsRef<OsStr>,
|
||||
V: AsRef<OsStr>,
|
||||
{
|
||||
self.env.extend(
|
||||
overrides
|
||||
.into_iter()
|
||||
.map(|(key, value)| (key.as_ref().to_owned(), value.as_ref().to_owned())),
|
||||
);
|
||||
self
|
||||
}
|
||||
|
||||
/// Remove one child variable from the baseline or prior overrides.
|
||||
pub fn remove_env(&mut self, key: impl AsRef<OsStr>) -> &mut Self {
|
||||
self.env.remove(key.as_ref());
|
||||
self
|
||||
}
|
||||
|
||||
/// Wire the mock endpoint onto an already-built sandbox.
|
||||
pub fn set_mock_url(&mut self, url: impl Into<String>) -> &mut Self {
|
||||
apply_mock_url(&mut self.env, url.into());
|
||||
self
|
||||
}
|
||||
|
||||
/// Return the effective child environment in stable key order.
|
||||
pub fn env(&self) -> Vec<(OsString, OsString)> {
|
||||
self.env
|
||||
.iter()
|
||||
.map(|(key, value)| (key.to_owned(), value.to_owned()))
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Apply the effective environment to a Tokio child command. Explicit
|
||||
/// command-level `.env(...)` calls made afterward have final precedence.
|
||||
pub fn apply_to_tokio_command(&self, cmd: &mut tokio::process::Command) {
|
||||
cmd.env_clear().envs(self.env());
|
||||
}
|
||||
|
||||
/// Merge the effective environment into a portable PTY command builder.
|
||||
/// The caller is responsible for calling `env_clear()` first.
|
||||
pub fn apply_to_command_builder(&self, cmd: &mut portable_pty::CommandBuilder) {
|
||||
for (key, value) in &self.env {
|
||||
cmd.env(key.as_os_str(), value.as_os_str());
|
||||
}
|
||||
}
|
||||
|
||||
/// Apply the effective environment to a standard child command. Explicit
|
||||
/// command-level `.env(...)` calls made afterward have final precedence.
|
||||
pub fn apply_to_std_command(&self, cmd: &mut Command) {
|
||||
cmd.env_clear().envs(self.env());
|
||||
}
|
||||
|
||||
/// Build a detached, non-interactive Git command using this sandbox's
|
||||
/// selected binary and cleared child environment.
|
||||
pub fn git_command(&self) -> Command {
|
||||
let git = self
|
||||
.env
|
||||
.get(OsStr::new("GIT_BIN_PATH"))
|
||||
.map_or_else(|| OsString::from("git"), OsString::to_owned);
|
||||
let mut cmd = Command::new(git);
|
||||
self.apply_to_std_command(&mut cmd);
|
||||
xai_tty_utils::detach_std_command(&mut cmd);
|
||||
cmd.stdin(Stdio::null()).envs(xai_tty_utils::pager_env());
|
||||
for &(key, value) in &xai_tty_utils::GIT_AUTH_SUPPRESSION_ENVS {
|
||||
cmd.env(key, value);
|
||||
}
|
||||
cmd.arg("--no-optional-locks");
|
||||
cmd
|
||||
}
|
||||
|
||||
/// Values that must be removed from captured child-output diagnostics.
|
||||
///
|
||||
/// This intentionally returns values only, never keys. Endpoint URLs,
|
||||
/// credentials, and sandbox-owned private paths can be echoed by a failing
|
||||
/// child even though process diagnostics never print its environment.
|
||||
pub(crate) fn diagnostic_redactions(&self) -> Vec<String> {
|
||||
self.env
|
||||
.iter()
|
||||
.filter(|(key, _)| diagnostic_value_is_sensitive(key))
|
||||
.map(|(_, value)| value.to_string_lossy().into_owned())
|
||||
.filter(|value| !value.is_empty())
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Sanitized, deterministic summary for assertion and spawn diagnostics.
|
||||
/// Secret-bearing values are never included.
|
||||
pub fn diagnostic_summary(&self) -> String {
|
||||
let mut summary = format!(
|
||||
"root={} home={} grok_home={} workspace={} temp={}",
|
||||
self.root().display(),
|
||||
self.home.display(),
|
||||
self.grok_home.display(),
|
||||
self.workspace.display(),
|
||||
self.temp.display(),
|
||||
);
|
||||
for (key, value) in &self.env {
|
||||
let key = key.to_string_lossy();
|
||||
let display = if is_secret_key(&key) {
|
||||
REDACTED.to_owned()
|
||||
} else if is_endpoint_key(&key) {
|
||||
sanitize_endpoint(value)
|
||||
} else {
|
||||
value.to_string_lossy().into_owned()
|
||||
};
|
||||
let _ = write!(summary, " {key}={display}");
|
||||
}
|
||||
summary
|
||||
}
|
||||
}
|
||||
|
||||
impl Default for TestSandbox {
|
||||
fn default() -> Self {
|
||||
Self::new()
|
||||
}
|
||||
}
|
||||
|
||||
/// Minimal construction-time choices for [`TestSandbox`]. Runtime feature
|
||||
/// variables belong on [`TestSandbox::set_env`] instead of a growing config.
|
||||
#[derive(Default)]
|
||||
pub struct TestSandboxBuilder {
|
||||
mock_url: Option<String>,
|
||||
git: bool,
|
||||
}
|
||||
|
||||
impl TestSandboxBuilder {
|
||||
/// Wire grok API, models, feedback, trace, conversation, and web traffic to
|
||||
/// a loopback mock endpoint and install the fake CI API key.
|
||||
pub fn mock_url(mut self, url: impl Into<String>) -> Self {
|
||||
self.mock_url = Some(url.into());
|
||||
self
|
||||
}
|
||||
|
||||
/// Initialize the workspace as a git repository with one committed file.
|
||||
pub fn git(mut self) -> Self {
|
||||
self.git = true;
|
||||
self
|
||||
}
|
||||
|
||||
/// Materialize the filesystem tree and canonical child environment.
|
||||
pub fn build(self) -> TestSandbox {
|
||||
let root = TempDir::new().expect("create test sandbox root");
|
||||
let home = root.path().join("home");
|
||||
let grok_home = home.join(".grok");
|
||||
let workspace = root.path().join("workspace");
|
||||
let temp = root.path().join("tmp");
|
||||
for path in [&home, &grok_home, &workspace, &temp] {
|
||||
std::fs::create_dir_all(path)
|
||||
.unwrap_or_else(|e| panic!("create sandbox path {}: {e}", path.display()));
|
||||
}
|
||||
|
||||
let parent_cwd = std::env::current_dir().expect("read parent cwd for test sandbox");
|
||||
let mut env = baseline_env(&home, &grok_home, &temp, &parent_cwd);
|
||||
if let Some(url) = self.mock_url {
|
||||
apply_mock_url(&mut env, url);
|
||||
}
|
||||
|
||||
let sandbox = TestSandbox {
|
||||
root,
|
||||
home,
|
||||
grok_home,
|
||||
workspace,
|
||||
temp,
|
||||
env,
|
||||
};
|
||||
if self.git {
|
||||
sandbox.init_git_workspace();
|
||||
}
|
||||
sandbox
|
||||
}
|
||||
}
|
||||
|
||||
impl TestSandbox {
|
||||
fn init_git_workspace(&self) {
|
||||
run_git(self, &["init"]);
|
||||
run_git(self, &["config", "user.email", "test@test.invalid"]);
|
||||
run_git(self, &["config", "user.name", "Grok Test"]);
|
||||
std::fs::write(self.workspace.join("README.md"), "test file\n")
|
||||
.expect("write sandbox git fixture");
|
||||
run_git(self, &["add", "-A"]);
|
||||
run_git(self, &["commit", "-m", "init", "--no-gpg-sign"]);
|
||||
}
|
||||
}
|
||||
|
||||
fn run_git(sandbox: &TestSandbox, args: &[&str]) {
|
||||
let mut cmd = sandbox.git_command();
|
||||
let git = cmd.get_program().to_owned();
|
||||
cmd.args(args).current_dir(sandbox.workspace());
|
||||
let output = cmd.output().unwrap_or_else(|e| {
|
||||
panic!(
|
||||
"failed to spawn git at {} for `git {}`: {e}\n{}",
|
||||
Path::new(&git).display(),
|
||||
args.join(" "),
|
||||
sandbox.diagnostic_summary(),
|
||||
)
|
||||
});
|
||||
assert!(
|
||||
output.status.success(),
|
||||
"git {} failed (exit {:?}):\n{}\n{}",
|
||||
args.join(" "),
|
||||
output.status.code(),
|
||||
String::from_utf8_lossy(&output.stderr),
|
||||
sandbox.diagnostic_summary(),
|
||||
);
|
||||
}
|
||||
|
||||
fn apply_mock_url(env: &mut BTreeMap<OsString, OsString>, url: String) {
|
||||
for key in [
|
||||
"GROK_CLI_CHAT_PROXY_BASE_URL",
|
||||
"GROK_XAI_API_BASE_URL",
|
||||
"GROK_MODELS_BASE_URL",
|
||||
"GROK_FEEDBACK_BASE_URL",
|
||||
"GROK_TRACE_UPLOAD_URL",
|
||||
"GROK_MANAGED_CONFIG_URL",
|
||||
"GROK_CODE_WEB_URL",
|
||||
"GROK_CONVERSATIONS_BASE_URL",
|
||||
] {
|
||||
env.insert(key.into(), url.clone().into());
|
||||
}
|
||||
env.insert("XAI_API_KEY".into(), TEST_API_KEY.into());
|
||||
}
|
||||
|
||||
fn baseline_env(
|
||||
home: &Path,
|
||||
grok_home: &Path,
|
||||
temp: &Path,
|
||||
parent_cwd: &Path,
|
||||
) -> BTreeMap<OsString, OsString> {
|
||||
let parent_env = std::env::vars_os().collect();
|
||||
baseline_env_from_parent(home, grok_home, temp, parent_cwd, &parent_env)
|
||||
}
|
||||
|
||||
fn baseline_env_from_parent(
|
||||
home: &Path,
|
||||
grok_home: &Path,
|
||||
temp: &Path,
|
||||
parent_cwd: &Path,
|
||||
parent_env: &BTreeMap<OsString, OsString>,
|
||||
) -> BTreeMap<OsString, OsString> {
|
||||
let mut env = BTreeMap::new();
|
||||
for key in platform_allowlist() {
|
||||
if let Some(value) = parent_env.get(OsStr::new(key)) {
|
||||
env.insert((*key).into(), value.to_owned());
|
||||
}
|
||||
}
|
||||
apply_hermetic_git_env(&mut env, parent_cwd, parent_env);
|
||||
#[cfg(unix)]
|
||||
env.entry("SHELL".into())
|
||||
.or_insert_with(|| OsString::from("/bin/sh"));
|
||||
|
||||
for (key, value) in [
|
||||
("HOME", home),
|
||||
("USERPROFILE", home),
|
||||
("GROK_HOME", grok_home),
|
||||
("TMPDIR", temp),
|
||||
("TMP", temp),
|
||||
("TEMP", temp),
|
||||
] {
|
||||
env.insert(key.into(), value.as_os_str().to_owned());
|
||||
}
|
||||
for (key, value) in [
|
||||
("GROK_TELEMETRY_ENABLED", "false"),
|
||||
("GROK_TELEMETRY_TRACE_UPLOAD", "false"),
|
||||
("GROK_FEEDBACK_ENABLED", "false"),
|
||||
("GROK_TRACE_UPLOAD", "false"),
|
||||
("GROK_INSTRUMENTATION", "disabled"),
|
||||
("OTEL_SDK_DISABLED", "true"),
|
||||
("DISABLE_TELEMETRY", "1"),
|
||||
("DISABLE_FEEDBACK_COMMAND", "1"),
|
||||
("GROK_DISABLE_AUTOUPDATER", "1"),
|
||||
("GROK_PROMPT_SUGGESTIONS", "false"),
|
||||
("NO_PROXY", "127.0.0.1,localhost,::1"),
|
||||
("no_proxy", "127.0.0.1,localhost,::1"),
|
||||
("GIT_CONFIG_NOSYSTEM", "1"),
|
||||
("GIT_TERMINAL_PROMPT", "0"),
|
||||
("GIT_ASKPASS", ""),
|
||||
("GIT_LFS_SKIP_SMUDGE", "1"),
|
||||
("PAGER", platform_pager()),
|
||||
("GIT_PAGER", platform_pager()),
|
||||
] {
|
||||
env.insert(key.into(), value.into());
|
||||
}
|
||||
env.insert(
|
||||
"GIT_CONFIG_GLOBAL".into(),
|
||||
grok_home.join("gitconfig").into_os_string(),
|
||||
);
|
||||
env
|
||||
}
|
||||
|
||||
fn apply_hermetic_git_env(
|
||||
env: &mut BTreeMap<OsString, OsString>,
|
||||
parent_cwd: &Path,
|
||||
parent_env: &BTreeMap<OsString, OsString>,
|
||||
) {
|
||||
let Some(git_bin) = parent_env.get(OsStr::new("GIT_BIN_PATH")) else {
|
||||
return;
|
||||
};
|
||||
let git_bin = PathBuf::from(git_bin);
|
||||
let git_bin = if git_bin.is_absolute() {
|
||||
git_bin
|
||||
} else {
|
||||
parent_cwd.join(git_bin)
|
||||
};
|
||||
let Some(parent) = git_bin.parent().map(Path::to_owned) else {
|
||||
return;
|
||||
};
|
||||
|
||||
let mut paths = vec![parent.to_owned()];
|
||||
if let Some(path) = parent_env.get(OsStr::new("PATH")) {
|
||||
paths.extend(std::env::split_paths(path));
|
||||
}
|
||||
let path = std::env::join_paths(paths).unwrap_or_else(|_| parent.as_os_str().to_owned());
|
||||
env.insert("GIT_BIN_PATH".into(), git_bin.into_os_string());
|
||||
env.insert("GIT_EXEC_PATH".into(), parent.into_os_string());
|
||||
env.insert("PATH".into(), path);
|
||||
}
|
||||
|
||||
fn platform_allowlist() -> &'static [&'static str] {
|
||||
#[cfg(windows)]
|
||||
{
|
||||
&[
|
||||
"PATH",
|
||||
"PATHEXT",
|
||||
"SystemRoot",
|
||||
"WINDIR",
|
||||
"ComSpec",
|
||||
"NUMBER_OF_PROCESSORS",
|
||||
"GIT_BIN_PATH",
|
||||
]
|
||||
}
|
||||
#[cfg(not(windows))]
|
||||
{
|
||||
&[
|
||||
"PATH",
|
||||
"LANG",
|
||||
"LC_ALL",
|
||||
"DYLD_LIBRARY_PATH",
|
||||
"LD_LIBRARY_PATH",
|
||||
"GIT_BIN_PATH",
|
||||
"SHELL",
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
fn platform_pager() -> &'static str {
|
||||
#[cfg(unix)]
|
||||
{
|
||||
"cat"
|
||||
}
|
||||
#[cfg(not(unix))]
|
||||
{
|
||||
""
|
||||
}
|
||||
}
|
||||
|
||||
fn is_secret_key(key: &str) -> bool {
|
||||
let upper = key.to_ascii_uppercase();
|
||||
let segments: Vec<_> = upper
|
||||
.split(|c: char| !c.is_ascii_alphanumeric())
|
||||
.filter(|segment| !segment.is_empty())
|
||||
.collect();
|
||||
segments.iter().any(|segment| {
|
||||
matches!(
|
||||
*segment,
|
||||
"TOKEN"
|
||||
| "SECRET"
|
||||
| "PASSWORD"
|
||||
| "PASSWD"
|
||||
| "PASS"
|
||||
| "KEY"
|
||||
| "AUTH"
|
||||
| "AUTHORIZATION"
|
||||
| "CREDENTIAL"
|
||||
| "CREDENTIALS"
|
||||
| "COOKIE"
|
||||
| "SESSION"
|
||||
) || segment.ends_with("TOKEN")
|
||||
|| segment.ends_with("SECRET")
|
||||
|| segment.ends_with("PASSWORD")
|
||||
|| segment.ends_with("CREDENTIAL")
|
||||
|| segment.ends_with("CREDENTIALS")
|
||||
|| segment.ends_with("APIKEY")
|
||||
})
|
||||
}
|
||||
|
||||
fn is_endpoint_key(key: &str) -> bool {
|
||||
let upper = key.to_ascii_uppercase();
|
||||
upper.contains("URL") || upper.contains("ENDPOINT") || upper.contains("PROXY")
|
||||
}
|
||||
|
||||
fn diagnostic_value_is_sensitive(key: &OsStr) -> bool {
|
||||
let key = key.to_string_lossy();
|
||||
is_secret_key(&key)
|
||||
|| is_endpoint_key(&key)
|
||||
|| matches!(
|
||||
key.to_ascii_uppercase().as_str(),
|
||||
"HOME" | "USERPROFILE" | "GROK_HOME" | "TMPDIR" | "TMP" | "TEMP" | "GIT_CONFIG_GLOBAL"
|
||||
)
|
||||
}
|
||||
|
||||
fn sanitize_endpoint(value: &OsStr) -> String {
|
||||
let Ok(mut url) = url::Url::parse(value.to_string_lossy().as_ref()) else {
|
||||
return REDACTED.to_owned();
|
||||
};
|
||||
let Some(host) = url.host() else {
|
||||
return REDACTED.to_owned();
|
||||
};
|
||||
let loopback = match host {
|
||||
url::Host::Domain(domain) => domain.eq_ignore_ascii_case("localhost"),
|
||||
url::Host::Ipv4(address) => address.is_loopback(),
|
||||
url::Host::Ipv6(address) => address.is_loopback(),
|
||||
};
|
||||
if !loopback || !matches!(url.scheme(), "http" | "https" | "ws" | "wss") {
|
||||
return REDACTED.to_owned();
|
||||
}
|
||||
|
||||
let _ = url.set_username("");
|
||||
let _ = url.set_password(None);
|
||||
url.set_query(None);
|
||||
url.set_fragment(None);
|
||||
url.to_string()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn env_value(sandbox: &TestSandbox, key: &str) -> Option<OsString> {
|
||||
sandbox
|
||||
.env()
|
||||
.into_iter()
|
||||
.find(|(candidate, _)| candidate == key)
|
||||
.map(|(_, value)| value)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn owns_distinct_isolated_paths() {
|
||||
let sandbox = TestSandbox::new();
|
||||
for path in [
|
||||
sandbox.home(),
|
||||
sandbox.grok_home(),
|
||||
sandbox.workspace(),
|
||||
sandbox.temp_dir(),
|
||||
] {
|
||||
assert!(path.starts_with(sandbox.root()), "{}", path.display());
|
||||
assert!(path.is_dir(), "{}", path.display());
|
||||
}
|
||||
assert_ne!(sandbox.home(), sandbox.workspace());
|
||||
assert_ne!(sandbox.home(), sandbox.temp_dir());
|
||||
assert_eq!(sandbox.grok_home(), sandbox.home().join(".grok"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn separate_instances_do_not_share_paths() {
|
||||
let first = TestSandbox::new();
|
||||
let second = TestSandbox::new();
|
||||
assert_ne!(first.root(), second.root());
|
||||
assert_ne!(first.home(), second.home());
|
||||
assert_ne!(first.workspace(), second.workspace());
|
||||
assert_ne!(first.temp_dir(), second.temp_dir());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn git_workspace_smoke_uses_committed_fixture() {
|
||||
let sandbox = TestSandbox::builder().git().build();
|
||||
assert!(sandbox.workspace().join(".git").is_dir());
|
||||
assert_eq!(
|
||||
std::fs::read_to_string(sandbox.workspace().join("README.md")).unwrap(),
|
||||
"test file\n"
|
||||
);
|
||||
let mut cmd = sandbox.git_command();
|
||||
cmd.args(["status", "--porcelain"])
|
||||
.current_dir(sandbox.workspace());
|
||||
let output = cmd.output().expect("run git status in sandbox");
|
||||
assert!(
|
||||
output.status.success(),
|
||||
"git status failed: {}",
|
||||
String::from_utf8_lossy(&output.stderr)
|
||||
);
|
||||
assert!(output.stdout.is_empty(), "workspace must start clean");
|
||||
}
|
||||
|
||||
fn resolved_baseline_env(
|
||||
parent_cwd: &Path,
|
||||
parent_env: BTreeMap<OsString, OsString>,
|
||||
) -> BTreeMap<OsString, OsString> {
|
||||
let root = tempfile::tempdir().expect("create baseline fixture");
|
||||
baseline_env_from_parent(
|
||||
&root.path().join("home"),
|
||||
&root.path().join("home/.grok"),
|
||||
&root.path().join("tmp"),
|
||||
parent_cwd,
|
||||
&parent_env,
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn relative_git_bin_path_resolves_against_parent_cwd() {
|
||||
let parent = tempfile::tempdir().expect("create parent cwd fixture");
|
||||
let parent_cwd = parent.path();
|
||||
let relative_git = Path::new("external/git_hermetic/bin/git");
|
||||
let env = resolved_baseline_env(
|
||||
parent_cwd,
|
||||
BTreeMap::from([
|
||||
(OsString::from("GIT_BIN_PATH"), relative_git.into()),
|
||||
(OsString::from("PATH"), OsString::from("/usr/bin")),
|
||||
]),
|
||||
);
|
||||
let git_bin = parent_cwd.join(relative_git);
|
||||
let parent = git_bin.parent().expect("git binary parent");
|
||||
assert_eq!(
|
||||
env.get(OsStr::new("GIT_BIN_PATH")).map(OsString::as_os_str),
|
||||
Some(git_bin.as_os_str())
|
||||
);
|
||||
assert_eq!(
|
||||
env.get(OsStr::new("GIT_EXEC_PATH"))
|
||||
.map(OsString::as_os_str),
|
||||
Some(parent.as_os_str())
|
||||
);
|
||||
assert_eq!(
|
||||
std::env::split_paths(env.get(OsStr::new("PATH")).expect("git PATH"))
|
||||
.next()
|
||||
.as_deref(),
|
||||
Some(parent)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn absent_git_bin_path_preserves_baseline_path_without_git_vars() {
|
||||
let path = OsString::from("/ordinary/bin");
|
||||
let env = resolved_baseline_env(
|
||||
Path::new("/bazel/execroot/workspace"),
|
||||
BTreeMap::from([(OsString::from("PATH"), path.to_owned())]),
|
||||
);
|
||||
assert_eq!(env.get(OsStr::new("PATH")), Some(&path));
|
||||
assert!(!env.contains_key(OsStr::new("GIT_BIN_PATH")));
|
||||
assert!(!env.contains_key(OsStr::new("GIT_EXEC_PATH")));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn git_command_uses_sandbox_state_without_process_global_mutation() {
|
||||
let root = TempDir::new().expect("create git command fixture");
|
||||
let git = root.path().join("git-dist/bin/git");
|
||||
let git_parent = git.parent().expect("git binary parent");
|
||||
let env = resolved_baseline_env(
|
||||
root.path(),
|
||||
BTreeMap::from([
|
||||
(OsString::from("GIT_BIN_PATH"), git.as_os_str().to_owned()),
|
||||
(OsString::from("PATH"), OsString::from("/ordinary/bin")),
|
||||
]),
|
||||
);
|
||||
let sandbox = TestSandbox {
|
||||
home: root.path().join("home"),
|
||||
grok_home: root.path().join("home/.grok"),
|
||||
workspace: root.path().join("workspace"),
|
||||
temp: root.path().join("tmp"),
|
||||
root,
|
||||
env,
|
||||
};
|
||||
|
||||
let process_git_env =
|
||||
["GIT_BIN_PATH", "GIT_EXEC_PATH", "PATH"].map(|key| (key, std::env::var_os(key)));
|
||||
let cmd = sandbox.git_command();
|
||||
assert_eq!(
|
||||
["GIT_BIN_PATH", "GIT_EXEC_PATH", "PATH"].map(|key| (key, std::env::var_os(key))),
|
||||
process_git_env
|
||||
);
|
||||
let command_env: BTreeMap<_, _> = cmd
|
||||
.get_envs()
|
||||
.map(|(key, value)| (key.to_owned(), value.map(OsStr::to_owned)))
|
||||
.collect();
|
||||
assert_eq!(cmd.get_program(), git);
|
||||
assert_eq!(
|
||||
command_env
|
||||
.get(OsStr::new("GIT_BIN_PATH"))
|
||||
.and_then(Option::as_deref),
|
||||
Some(git.as_os_str())
|
||||
);
|
||||
assert_eq!(
|
||||
command_env
|
||||
.get(OsStr::new("GIT_EXEC_PATH"))
|
||||
.and_then(Option::as_deref),
|
||||
Some(git_parent.as_os_str())
|
||||
);
|
||||
assert_eq!(
|
||||
std::env::split_paths(
|
||||
command_env
|
||||
.get(OsStr::new("PATH"))
|
||||
.and_then(Option::as_deref)
|
||||
.expect("git PATH"),
|
||||
)
|
||||
.next()
|
||||
.as_deref(),
|
||||
Some(git_parent)
|
||||
);
|
||||
assert_eq!(
|
||||
command_env
|
||||
.get(OsStr::new("GIT_TERMINAL_PROMPT"))
|
||||
.and_then(Option::as_deref),
|
||||
Some(OsStr::new("0"))
|
||||
);
|
||||
assert_eq!(
|
||||
command_env
|
||||
.get(OsStr::new("GIT_SSH_COMMAND"))
|
||||
.and_then(Option::as_deref),
|
||||
Some(OsStr::new("ssh -o BatchMode=yes"))
|
||||
);
|
||||
assert_eq!(
|
||||
cmd.get_args().next(),
|
||||
Some(OsStr::new("--no-optional-locks"))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn baseline_is_hermetic_and_network_quiet() {
|
||||
let sandbox = TestSandbox::builder()
|
||||
.mock_url("http://127.0.0.1:43123/v1")
|
||||
.build();
|
||||
assert_eq!(env_value(&sandbox, "HOME"), Some(sandbox.home().into()));
|
||||
assert_eq!(
|
||||
env_value(&sandbox, "GROK_HOME"),
|
||||
Some(sandbox.grok_home().into())
|
||||
);
|
||||
assert_eq!(
|
||||
env_value(&sandbox, "TMPDIR"),
|
||||
Some(sandbox.temp_dir().into())
|
||||
);
|
||||
assert_eq!(
|
||||
env_value(&sandbox, "XAI_API_KEY").as_deref(),
|
||||
Some(OsStr::new(TEST_API_KEY))
|
||||
);
|
||||
assert_eq!(
|
||||
env_value(&sandbox, "GROK_DISABLE_AUTOUPDATER").as_deref(),
|
||||
Some(OsStr::new("1"))
|
||||
);
|
||||
assert_eq!(
|
||||
env_value(&sandbox, "GROK_TELEMETRY_TRACE_UPLOAD").as_deref(),
|
||||
Some(OsStr::new("false"))
|
||||
);
|
||||
assert_eq!(
|
||||
env_value(&sandbox, "NO_PROXY").as_deref(),
|
||||
Some(OsStr::new("127.0.0.1,localhost,::1"))
|
||||
);
|
||||
for proxy in [
|
||||
"HTTP_PROXY",
|
||||
"HTTPS_PROXY",
|
||||
"ALL_PROXY",
|
||||
"http_proxy",
|
||||
"https_proxy",
|
||||
"all_proxy",
|
||||
] {
|
||||
assert_eq!(env_value(&sandbox, proxy), None, "{proxy} must not leak");
|
||||
}
|
||||
assert_eq!(env_value(&sandbox, "GROK_LEADER_SOCKET"), None);
|
||||
assert_eq!(env_value(&sandbox, "GROK_DISABLE_WEB_FETCH"), None);
|
||||
assert_eq!(env_value(&sandbox, "GROK_WEB_FETCH"), None);
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
#[test]
|
||||
fn unix_shell_policy_preserves_host_or_falls_back_and_can_be_overridden() {
|
||||
let mut sandbox = TestSandbox::new();
|
||||
let expected = std::env::var_os("SHELL").unwrap_or_else(|| OsString::from("/bin/sh"));
|
||||
assert_eq!(env_value(&sandbox, "SHELL"), Some(expected));
|
||||
|
||||
sandbox.set_env("SHELL", "/bin/bash");
|
||||
assert_eq!(
|
||||
env_value(&sandbox, "SHELL").as_deref(),
|
||||
Some(OsStr::new("/bin/bash"))
|
||||
);
|
||||
|
||||
let mut cmd = Command::new("unused");
|
||||
sandbox.apply_to_std_command(&mut cmd);
|
||||
assert_eq!(
|
||||
cmd.get_envs()
|
||||
.find(|(key, _)| *key == OsStr::new("SHELL"))
|
||||
.and_then(|(_, value)| value),
|
||||
Some(OsStr::new("/bin/bash"))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn command_application_clears_ambient_env_and_command_override_wins() {
|
||||
let sandbox = TestSandbox::new();
|
||||
let mut cmd = Command::new("unused");
|
||||
cmd.env("AMBIENT_SECRET", "must-disappear")
|
||||
.env("GROK_PROMPT_SUGGESTIONS", "ambient");
|
||||
sandbox.apply_to_std_command(&mut cmd);
|
||||
cmd.env("GROK_PROMPT_SUGGESTIONS", "command");
|
||||
let env: BTreeMap<_, _> = cmd
|
||||
.get_envs()
|
||||
.filter_map(|(key, value)| value.map(|value| (key.to_owned(), value.to_owned())))
|
||||
.collect();
|
||||
assert!(!env.contains_key(OsStr::new("AMBIENT_SECRET")));
|
||||
assert_eq!(
|
||||
env.get(OsStr::new("GROK_PROMPT_SUGGESTIONS"))
|
||||
.map(OsString::as_os_str),
|
||||
Some(OsStr::new("command"))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn explicit_overrides_win_and_can_remove_baseline_entries() {
|
||||
let mut sandbox = TestSandbox::new();
|
||||
sandbox
|
||||
.set_env("TERM_PROGRAM", "vscode")
|
||||
.set_env("GROK_PROMPT_SUGGESTIONS", "true")
|
||||
.set_env("NO_PROXY", "override.invalid")
|
||||
.remove_env("GROK_DISABLE_AUTOUPDATER");
|
||||
assert_eq!(
|
||||
env_value(&sandbox, "TERM_PROGRAM").as_deref(),
|
||||
Some(OsStr::new("vscode"))
|
||||
);
|
||||
assert_eq!(
|
||||
env_value(&sandbox, "GROK_PROMPT_SUGGESTIONS").as_deref(),
|
||||
Some(OsStr::new("true"))
|
||||
);
|
||||
assert_eq!(
|
||||
env_value(&sandbox, "NO_PROXY").as_deref(),
|
||||
Some(OsStr::new("override.invalid"))
|
||||
);
|
||||
assert_eq!(env_value(&sandbox, "GROK_DISABLE_AUTOUPDATER"), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cross_platform_home_and_temp_names_are_present() {
|
||||
let sandbox = TestSandbox::new();
|
||||
assert_eq!(
|
||||
env_value(&sandbox, "USERPROFILE"),
|
||||
Some(sandbox.home().into())
|
||||
);
|
||||
assert_eq!(env_value(&sandbox, "TEMP"), Some(sandbox.temp_dir().into()));
|
||||
assert_eq!(env_value(&sandbox, "TMP"), Some(sandbox.temp_dir().into()));
|
||||
}
|
||||
|
||||
#[cfg(windows)]
|
||||
#[test]
|
||||
fn windows_platform_essentials_are_allowlisted() {
|
||||
let sandbox = TestSandbox::new();
|
||||
for essential in ["PATH", "PATHEXT", "SystemRoot", "ComSpec"] {
|
||||
if std::env::var_os(essential).is_some() {
|
||||
assert!(env_value(&sandbox, essential).is_some(), "{essential}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn diagnostics_fail_closed_for_credential_keys() {
|
||||
let mut sandbox = TestSandbox::new();
|
||||
for (key, value) in [
|
||||
("CUSTOM_TOKEN", "token-do-not-print"),
|
||||
("SERVICE_API_KEY", "api-key-do-not-print"),
|
||||
("clientSecret", "secret-do-not-print"),
|
||||
("DB_PASSWORD_FILE", "/secret/password-file"),
|
||||
("AWS_CREDENTIALS", "credentials-do-not-print"),
|
||||
("SESSION_COOKIE", "cookie-do-not-print"),
|
||||
("GROK_DEPLOYMENT_KEY", "deployment-key-do-not-print"),
|
||||
("GROK_EXTRA_AUTH_KEY", "alpha-test-key-do-not-print"),
|
||||
("AWS_ACCESS_KEY_ID", "aws-access-key-do-not-print"),
|
||||
("PRIVATE_KEY", "private-key-do-not-print"),
|
||||
] {
|
||||
sandbox.set_env(key, value);
|
||||
}
|
||||
sandbox.set_env("SAFE_FEATURE", "enabled");
|
||||
let summary = sandbox.diagnostic_summary();
|
||||
for key in [
|
||||
"CUSTOM_TOKEN",
|
||||
"SERVICE_API_KEY",
|
||||
"clientSecret",
|
||||
"DB_PASSWORD_FILE",
|
||||
"AWS_CREDENTIALS",
|
||||
"SESSION_COOKIE",
|
||||
"GROK_DEPLOYMENT_KEY",
|
||||
"GROK_EXTRA_AUTH_KEY",
|
||||
"AWS_ACCESS_KEY_ID",
|
||||
"PRIVATE_KEY",
|
||||
] {
|
||||
assert!(summary.contains(&format!("{key}=<redacted>")), "{summary}");
|
||||
}
|
||||
assert!(summary.contains("SAFE_FEATURE=enabled"), "{summary}");
|
||||
for secret in [
|
||||
"token-do-not-print",
|
||||
"api-key-do-not-print",
|
||||
"secret-do-not-print",
|
||||
"/secret/password-file",
|
||||
"credentials-do-not-print",
|
||||
"cookie-do-not-print",
|
||||
"deployment-key-do-not-print",
|
||||
"alpha-test-key-do-not-print",
|
||||
"aws-access-key-do-not-print",
|
||||
"private-key-do-not-print",
|
||||
] {
|
||||
assert!(!summary.contains(secret), "{summary}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn diagnostics_show_only_sanitized_loopback_urls() {
|
||||
let cases = [
|
||||
(
|
||||
"HTTP_URL",
|
||||
"http://user:password@127.0.0.1:43123/v1?token=secret#fragment",
|
||||
"http://127.0.0.1:43123/v1",
|
||||
),
|
||||
(
|
||||
"HTTPS_URL",
|
||||
"https://localhost:43124/path?api_key=secret",
|
||||
"https://localhost:43124/path",
|
||||
),
|
||||
(
|
||||
"IPV6_URL",
|
||||
"http://user:password@[::1]:43125/v1#secret",
|
||||
"http://[::1]:43125/v1",
|
||||
),
|
||||
(
|
||||
"IPV4_OTHER_LOOPBACK_URL",
|
||||
"http://127.0.0.2:43126/v1?secret=yes",
|
||||
"http://127.0.0.2:43126/v1",
|
||||
),
|
||||
];
|
||||
let mut sandbox = TestSandbox::new();
|
||||
for (key, raw, _) in cases {
|
||||
sandbox.set_env(key, raw);
|
||||
}
|
||||
sandbox
|
||||
.set_env(
|
||||
"REMOTE_URL",
|
||||
"https://user:password@example.test/v1?token=secret",
|
||||
)
|
||||
.set_env("MALFORMED_URL", "not a url password=secret")
|
||||
.set_env("HTTPS_PROXY", "https://user:pass@proxy.example.test");
|
||||
|
||||
let summary = sandbox.diagnostic_summary();
|
||||
for (key, _, expected) in cases {
|
||||
assert!(summary.contains(&format!("{key}={expected}")), "{summary}");
|
||||
}
|
||||
for key in ["REMOTE_URL", "MALFORMED_URL", "HTTPS_PROXY"] {
|
||||
assert!(summary.contains(&format!("{key}=<redacted>")), "{summary}");
|
||||
}
|
||||
for secret in ["user", "password", "token=secret", "fragment", "pass@"] {
|
||||
assert!(!summary.contains(secret), "{summary}");
|
||||
}
|
||||
assert!(!summary.contains(TEST_API_KEY), "{summary}");
|
||||
}
|
||||
}
|
||||
Loading…
Reference in a new issue