bingshuo/grok-build-upstream-mirror
Watch
1
0
Fork
You've already forked grok-build-upstream-mirror
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Synced from monorepo

Browse source 
Synced from monorepo

Changes:
- Cache growing transcripts on the messages backend
- Tell the model when a wait was clamped instead of re-inviting it
- Stop the stationarity nudge from claiming results are identical
- Deliver the stationarity nudge after the tool result
- Run auth provider commands through the platform shell (fixes Windows)
- Keep monitor tool stdout short and prescriptive
- Use UUIDs for analytics event insert IDs
- Stop crashing at startup when the host runs out of threads
- Delete the current session from within the session
- Add project forking-settings toggle (backend and deploy-time control)
- Reap a session’s bash and background commands when it closes
- Reap a session’s hook child processes when it closes
- Track coding-data consent decisions
- Fail open the access gate to stop false CLI paywalls
- Ship Agent Dashboard user guide
- Enable doom-loop recovery by default
- Kill agent children and the idle inhibitor when the parent process dies
- Fix multi-process credential wipe and orphaned session log writers

Source-Revision: 6372e41d828b8a6ee82c29e01a69e27ec895cca9
This commit is contained in:
grokkybara[bot] 2026-07-29 17:17:54 +00:00
commit 500129c714
89 changed files with 3841 additions and 771 deletions
Download patch file Download diff file Expand all files Collapse all files

98
crates/codegen/xai-grok-shell/tests/test_auth_provider_command_e2e.rs Normal file
View file

@ -0,0 +1,98 @@
//! End-to-end guard for `auth_provider_command`: a configured external auth
//! provider must actually mint the session credential on the host platform.
//!
//! Regression cover. The provider used to be spawned through a hardcoded
//! `sh -c`. On Windows that either fails to spawn (no `sh` in a default
//! install) or, where Git Bash is present, silently eats the backslashes in a
//! native path — `C:\Windows\System32\whoami.exe` reaches the shell as
//! `C:WindowsSystem32whoami.exe` and exits 127. Either way the auth flow fell
//! through to the built-in browser login, so a configured provider looked like
//! it had been ignored.
//!
//! The test drives the public entry point (`try_ensure_fresh_auth` →
//! `AuthManager::auth` → external refresher → platform shell) and is hermetic:
//! a throwaway `GROK_HOME`, no network, and a provider command that needs no
//! binary beyond what the platform shell already provides.
use std::collections::BTreeMap;
use std::path::Path;
use chrono::Utc;
use xai_grok_shell::auth::{AuthMode, GrokAuth, GrokComConfig, try_ensure_fresh_auth};
const SEED_TOKEN: &str = "stale-token-that-must-be-replaced";
/// Point the process at a throwaway grok home. `grok_home()` memoizes into a
/// `OnceLock`, so every phase below shares this one directory — which is why
/// they live in a single test rather than racing each other as separate ones.
fn use_temp_grok_home(dir: &Path) {
// SAFETY: single-threaded test entry, before any thread that reads the
// environment is spawned.
unsafe {
std::env::set_var("GROK_HOME", dir);
}
}
/// Seed an expired credential so `auth()` takes the refresh path; a cold home
/// returns `NotLoggedIn` without ever consulting the provider.
fn seed_expired_credential(home: &Path, scope: &str) {
let expired = GrokAuth {
key: SEED_TOKEN.to_owned(),
auth_mode: AuthMode::External,
expires_at: Some(Utc::now() - chrono::Duration::hours(1)),
..GrokAuth::default()
};
let store: BTreeMap<String, GrokAuth> = [(scope.to_owned(), expired)].into_iter().collect();
std::fs::write(
home.join("auth.json"),
serde_json::to_string(&store).expect("serialize auth store"),
)
.expect("write auth.json");
}
/// Run one provider command through the real auth path and return the token.
async fn mint_with_provider(home: &Path, command: &str) -> String {
let config = GrokComConfig {
auth_provider_command: Some(command.to_owned()),
..GrokComConfig::default()
};
seed_expired_credential(home, &config.auth_scope());
let auth = try_ensure_fresh_auth(&config).await.unwrap_or_else(|| {
panic!("auth_provider_command `{command}` was configured but no credential was minted")
});
assert_eq!(
auth.auth_mode,
AuthMode::External,
"credential must come from the provider, not a cached or built-in path"
);
assert_ne!(
auth.key, SEED_TOKEN,
"the expired seed must have been replaced by the provider's output"
);
auth.key
}
#[tokio::test]
async fn auth_provider_command_mints_the_session_credential() {
let home = tempfile::tempdir().expect("tempdir");
use_temp_grok_home(home.path());
// `echo <token>` is valid in both `sh -c` and `cmd /C`, so this phase needs
// no external binary and runs identically on every platform.
let token = mint_with_provider(home.path(), "echo grok-ext-token").await;
assert_eq!(token, "grok-ext-token");
// Windows only: an absolute native path, the form an operator actually
// writes in config.toml, and the exact shape a POSIX shell mangles. Run
// after the portable phase so a failure here is unambiguously about
// backslash handling rather than the provider path in general.
#[cfg(windows)]
{
let token = mint_with_provider(home.path(), r"C:\Windows\System32\whoami.exe").await;
assert!(
!token.trim().is_empty(),
"a native Windows path must reach the provider intact"
);
}
}

87
crates/codegen/xai-grok-shell/tests/test_built_binary_e2e.rs
View file

@ -1237,6 +1237,93 @@ async fn test_stdio_xcode_escaped_slash_methods_get_responses() {
);
}
/// `grok agent stdio` must initiate shutdown and exit when its client closes
/// stdin (EOF) — a dead parent means closed pipes, so this is the primary
/// orphan guard on every platform (the Linux `PR_SET_PDEATHSIG` binding in
/// `run_stdio_agent` additionally covers an agent wedged mid-turn that never
/// reads stdin again). Guards the `spawn_stdin_line_reader` → stdin_closed →
/// simplex-shutdown → `handle_io` completion chain end to end.
#[tokio::test]
#[ignore] // requires pre-built binary; run with --ignored
async fn test_stdio_agent_exits_on_stdin_eof() {
use tokio::io::{AsyncBufReadExt as _, AsyncWriteExt as _};
let server = MockInferenceServer::start()
.await
.expect("start mock server");
let mut sandbox = TestSandbox::builder().git().build();
sandbox.set_mock_url(server.url());
let mut cmd = tokio::process::Command::new(grok_binary());
cmd.args(["agent", "stdio"])
.current_dir(sandbox.workspace());
let mut process = TestProcess::spawn(
cmd,
&sandbox,
TestProcessConfig::new()
.label("grok agent stdio (eof)")
.stdin(TestStdin::Piped)
.stdout(TestOutput::Piped),
)
.expect("spawn grok agent stdio");
// Prove the agent is up and serving before the EOF (an exit during
// startup would trivially pass the wait below).
let mut stdin = process.take_stdin().expect("child stdin missing");
let stdout = process.take_stdout().expect("child stdout missing");
let mut reader = tokio::io::BufReader::new(stdout);
stdin
.write_all(
concat!(
r#"{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":1,"#,
r#""clientCapabilities":{"fs":{"readTextFile":false,"writeTextFile":false},"terminal":false},"#,
r#""_meta":{"startupHints":{"nonInteractive":true,"skipGitStatus":true,"skipProjectLayout":true},"#,
r#""clientType":"eof-test","clientVersion":"0.0.0"}}}"#,
"\n"
)
.as_bytes(),
)
.await
.expect("write initialize");
stdin.flush().await.expect("flush initialize");
let mut line = String::new();
tokio::time::timeout(scaled(Duration::from_secs(20)), reader.read_line(&mut line))
.await
.unwrap_or_else(|_| {
panic!(
"no initialize response before EOF\nstderr:\n{}",
stderr_tail(&process.stderr_tail().text, 1200)
)
})
.expect("read initialize response");
assert!(
line.contains("\"result\""),
"initialize must respond with a result, got: {line}"
);
// Close the write end: the agent sees stdin EOF, exactly as when its
// parent dies and the inherited pipe closes.
drop(stdin);
// Exit path includes a bounded teardown (100ms simplex flush + 2s upload
// queue grace), so allow comfortably more than that.
let status = process
.wait_with_deadline(scaled(Duration::from_secs(30)))
.await
.expect("wait for agent exit")
.unwrap_or_else(|| {
panic!(
"grok agent stdio did not exit after stdin EOF\n{}",
process.diagnostic_summary()
)
});
assert!(
status.success(),
"agent should exit cleanly on stdin EOF, got {status:?}\nstderr:\n{}",
stderr_tail(&process.stderr_tail().text, 1200)
);
}
// ── Config test harness ─────────────────────────────────────────────────────
/// Isolated headless run with a custom `~/.grok/`. Clean env (no leaked

172
crates/codegen/xai-grok-shell/tests/test_leader_soak.rs
View file

@ -1,15 +1,10 @@
//! Leader soak: an in-process leader server fronting a REAL `MvpAgent`, hammered
//! by churning `LeaderClient`s until a time budget expires. Asserts the leader
//! neither leaks memory nor accumulates zombie clients, and that no response is
//! ever dropped on a live-client send (`leader.response.send_failed`).
//!
//! Duration is bounded by `LEADER_SOAK_SECS` (default 10s so an ad-hoc
//! `--ignored` run stays quick). RSS growth is bounded by
//! `LEADER_SOAK_MAX_RSS_GROWTH_MB` (default 1024). On-demand today — no CI
//! lane runs it; a real soak is the long form:
//! Leader soak: a real `MvpAgent` behind an in-process leader, churned by
//! clients until `LEADER_SOAK_SECS` expires. Each cycle closes its sessions,
//! so the bounds measure what teardown reclaims.
//!
//! ```bash
//! LEADER_SOAK_SECS=1200 cargo test -p xai-grok-shell --test test_leader_soak -- --ignored --nocapture
//! LEADER_SOAK_SECS=1200 cargo test -p xai-grok-shell --features test-support \
//! --test test_leader_soak -- --ignored --nocapture
//! ```
#![cfg(unix)]
@ -17,25 +12,14 @@
use std::sync::Arc;
use std::time::Duration;
use agent_client_protocol as acp;
use tempfile::TempDir;
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
use tokio_util::compat::{TokioAsyncReadCompatExt, TokioAsyncWriteCompatExt};
use tokio_util::sync::CancellationToken;
use xai_acp_lib::{
AcpAgentGatewayReceiver as GatewayReceiver, AcpAgentGatewaySender as GatewaySender,
LineBufferedRead,
};
use xai_grok_shell::agent::config::Config as AgentConfig;
use xai_grok_shell::agent::mvp_agent::MvpAgent;
use xai_grok_shell::leader::{
ClientCapabilities, ClientMode, LeaderClient, LeaderServerControlState, LeaderServerMetadata,
run_leader_server,
};
use xai_grok_test_support::resources::ResourceSnapshot;
const SIMPLEX_BUF: usize = 8 * 1024 * 1024;
fn env_u64(key: &str, default: u64) -> u64 {
std::env::var(key)
.ok()
@ -113,13 +97,13 @@ async fn leader_soak_churning_clients_no_leaks_no_zombies() {
let sock_path = grok_home.path().join("leader-soak.sock");
let soak_secs = env_u64("LEADER_SOAK_SECS", 10);
let max_growth_mb = env_u64("LEADER_SOAK_MAX_RSS_GROWTH_MB", 1024);
let max_thread_growth = env_u64("LEADER_SOAK_MAX_THREAD_GROWTH", 64) as usize;
let send_failed_before = send_failed_count();
let local = tokio::task::LocalSet::new();
local
.run_until(async {
// ── Leader server (survives client churn) ────────────────────
let (acp_tx, mut acp_rx) = tokio::sync::mpsc::unbounded_channel::<String>();
let (acp_tx, acp_rx) = tokio::sync::mpsc::unbounded_channel::<String>();
let (response_tx, response_rx) = tokio::sync::mpsc::unbounded_channel::<String>();
let cancel = CancellationToken::new();
let client_count = Arc::new(std::sync::atomic::AtomicUsize::new(0));
@ -152,70 +136,9 @@ async fn leader_soak_churning_clients_no_leaks_no_zombies() {
.await;
});
// ── Real agent behind it ──────────────────────────────────────
// Copied from `run_leader`'s agent-spawn + IPC/stdout bridge
// blocks in src/agent/app.rs (inside its LocalSet body); kept as
// a deliberate copy so production stays untouched. Second copy of
// the same wiring: xai-grok-pager/src/app/leader_cluster/mod.rs
// (`spawn_leader_generation`) — keep the two copies behaviorally
// identical.
let (agent_in_read, agent_in_write) = tokio::io::simplex(SIMPLEX_BUF);
let (agent_out_read, agent_out_write) = tokio::io::simplex(SIMPLEX_BUF);
tokio::task::spawn_local(async move {
let agent_config = AgentConfig::default();
let auth_manager = Arc::new(agent_config.create_auth_manager());
let (gw_tx, gw_rx) = tokio::sync::mpsc::unbounded_channel();
let gateway = GatewaySender::new(gw_tx);
let agent = MvpAgent::new(gateway, &agent_config, auth_manager, None)
.expect("valid agent config");
let incoming = LineBufferedRead::spawn_local(agent_in_read.compat());
let (conn, handle_io) = acp::AgentSideConnection::new(
agent,
agent_out_write.compat_write(),
incoming,
|fut| {
tokio::task::spawn_local(fut);
},
);
tokio::task::spawn_local(
GatewayReceiver::new(gw_rx, conn)
.with_on_meta(xai_file_utils::trace_context::span_from_meta_traceparent)
.run(),
);
let _ = handle_io.await;
});
// Leader → agent stdin.
tokio::task::spawn_local(async move {
let mut agent_in_write = agent_in_write;
while let Some(msg) = acp_rx.recv().await {
if agent_in_write.write_all(msg.as_bytes()).await.is_err()
|| agent_in_write.write_all(b"\n").await.is_err()
{
break;
}
}
});
// Agent stdout → leader responses.
let response_tx_for_agent = response_tx.clone();
tokio::task::spawn_local(async move {
let mut reader = BufReader::new(agent_out_read);
let mut line = String::new();
loop {
line.clear();
match reader.read_line(&mut line).await {
Ok(0) => break,
Ok(_) => {
let msg = line.trim_end_matches(['\r', '\n']).to_string();
if !msg.is_empty() {
let _ = response_tx_for_agent.send(msg);
}
}
Err(_) => break,
}
}
});
// Hold a sender for the whole soak: the leader's response channel
// must not close when the agent's output ends.
xai_grok_shell::leader::in_process::spawn_agent(acp_rx, response_tx.clone());
let deadline = tokio::time::Instant::now() + Duration::from_secs(10);
while !sock_path.exists() && tokio::time::Instant::now() < deadline {
@ -223,7 +146,6 @@ async fn leader_soak_churning_clients_no_leaks_no_zombies() {
}
assert!(sock_path.exists(), "leader socket never bound");
// ── One-time initialize + authenticate through the leader ────
let mut bootstrap = LeaderClient::connect(
sock_path.clone(),
"soak-bootstrap",
@ -247,14 +169,17 @@ async fn leader_soak_churning_clients_no_leaks_no_zombies() {
)
.await;
eprintln!(
"[soak] budgets: {soak_secs}s, rss {max_growth_mb} MB, threads {max_thread_growth}"
);
let rss_before = ResourceSnapshot::capture();
let soak_deadline = tokio::time::Instant::now() + Duration::from_secs(soak_secs);
let workdir_str = workdir.path().to_string_lossy().to_string();
let mut cycles: u64 = 0;
let mut turns: u64 = 0;
// ── Churn: 10 fresh clients per cycle, 2 sessions each, one
// scripted turn per session, then all disconnect ───────────────
// Each cycle: 10 fresh clients, 2 sessions each, one scripted
// turn per session, then all disconnect.
while tokio::time::Instant::now() < soak_deadline {
cycles += 1;
let mut clients = Vec::new();
@ -298,10 +223,22 @@ async fn leader_soak_churning_clients_no_leaks_no_zombies() {
)
.await;
turns += 1;
// Disconnecting leaves sessions resident for a
// reconnect; `_` is the wire form for a custom method.
let close_id = 300 + s;
rpc(
client,
format!(
r#"{{"jsonrpc":"2.0","id":{close_id},"method":"_x.ai/session/close","params":{{"sessionId":"{sid}"}}}}"#
),
close_id,
"x.ai/session/close",
)
.await;
}
}
// Churn: everyone disconnects; the roster must drain fully.
for client in clients {
client.cancel();
}
@ -314,13 +251,39 @@ async fn leader_soak_churning_clients_no_leaks_no_zombies() {
);
tokio::time::sleep(Duration::from_millis(50)).await;
}
// Linear in cycles is a leak; flattening is the allocator.
if let Some(rss) = ResourceSnapshot::capture().rss {
eprintln!(
"[soak] cycle {cycles}: rss {:.1} MB",
rss as f64 / (1024.0 * 1024.0)
);
}
if cycles == 1 {
let snap = rpc(
&mut bootstrap,
r#"{"jsonrpc":"2.0","id":901,"method":"_x.ai/debug/agent","params":{}}"#
.to_string(),
901,
"x.ai/debug/agent",
)
.await;
eprintln!("[soak] registries after cycle 1: {}", snap["result"]["registries"]);
}
}
let snap = rpc(
&mut bootstrap,
r#"{"jsonrpc":"2.0","id":902,"method":"_x.ai/debug/agent","params":{}}"#
.to_string(),
902,
"x.ai/debug/agent",
)
.await;
eprintln!("[soak] registries at end: {}", snap["result"]["registries"]);
eprintln!("[soak] {cycles} cycles, {turns} turns in {soak_secs}s budget");
assert!(cycles > 0, "soak budget too small to complete one cycle");
// ── Convergence: only the bootstrap client remains, and the
// leader still serves a healthy round-trip ────────────────────
assert_eq!(
client_count.load(std::sync::atomic::Ordering::Relaxed),
1,
@ -337,14 +300,12 @@ async fn leader_soak_churning_clients_no_leaks_no_zombies() {
.await;
assert!(resp["result"]["sessionId"].is_string());
// ── No response was ever dropped on a live-client send ────────
assert_eq!(
send_failed_count(),
send_failed_before,
"leader.response.send_failed must not occur during the soak"
);
// ── RSS bound ─────────────────────────────────────────────────
let rss_after = ResourceSnapshot::capture();
let growth = rss_after.growth_from(&rss_before);
if let (Some(before), Some(after), Some(growth_bytes)) =
@ -357,11 +318,28 @@ async fn leader_soak_churning_clients_no_leaks_no_zombies() {
after as f64 / (1024.0 * 1024.0),
);
assert!(
growth_mb < max_growth_mb as f64,
growth_mb <= max_growth_mb as f64,
"leader RSS grew {growth_mb:.1} MB over the soak (bound {max_growth_mb} MB)"
);
} else {
eprintln!("[soak] rss measurement unavailable on this platform; bound skipped");
panic!("memory sample unavailable; the soak cannot bound it");
}
// A missing sample means the probe failed, which would silently
// retire the nightly budget. Threads are Linux-only.
match growth.threads {
Some(thread_growth) => {
eprintln!("[soak] threads: growth {thread_growth}");
assert!(
thread_growth <= max_thread_growth,
"leader threads grew by {thread_growth} over the soak \
(bound {max_thread_growth})"
);
}
None if cfg!(target_os = "linux") => {
panic!("thread growth sample unavailable; the soak cannot bound it")
}
None => {}
}
bootstrap.cancel();

12
crates/codegen/xai-grok-shell/tests/test_registry_churn.rs
View file

@ -32,6 +32,8 @@ const RPC_TIMEOUT: Duration = Duration::from_secs(60);
struct Counts {
sessions: usize,
session_threads: usize,
resident_resources: usize,
retained_resources: usize,
dispatch_locks: usize,
session_turn_numbers: usize,
permission_event_receivers: usize,
@ -73,11 +75,11 @@ async fn ext_method(
method: &str,
params: serde_json::Value,
) -> serde_json::Value {
let raw =
let params_json =
serde_json::value::RawValue::from_string(params.to_string()).expect("serialize ext params");
let resp = tokio::time::timeout(
RPC_TIMEOUT,
conn.ext_method(acp::ExtRequest::new(method, Arc::from(raw))),
conn.ext_method(acp::ExtRequest::new(method, Arc::from(params_json))),
)
.await
.unwrap_or_else(|_| panic!("{method} timed out"))
@ -255,6 +257,12 @@ fn session_churn_returns_registry_snapshot_to_baseline() {
baseline.sessions, 0,
"warmup session must be fully removed before baseline"
);
assert_eq!(
(baseline.resident_resources, baseline.retained_resources),
(0, 0),
"warmup must leave no per-session resource entries, including \
entries holding no resources"
);
assert_eq!(
baseline.workspace_bindings,
Some(0),

2
crates/codegen/xai-grok-shell/tests/test_session_load_memory.rs
View file

@ -3,7 +3,7 @@
//! Resuming a large session once OOM-killed the process under a cgroup cap.
//!
//! Run:
//! cargo test -p xai-grok-shell --features dhat-heap --test test_session_load_memory \
//! cargo test -p xai-grok-shell --features dhat-heap,test-support --test test_session_load_memory \
//! session_load_dhat_bounded_and_freed -- --ignored --nocapture
#![cfg(unix)]