bingshuo/grok-build-upstream-mirror
Watch
1
0
Fork
You've already forked grok-build-upstream-mirror
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Synced from monorepo

Browse source 
Synced from monorepo

Changes:
- Shell: accept target response id on rewind execute
- Shell: stamp response id on chat user message chunks
- Worktree: optional rebuild and stale git registration cleanup in auto-GC
- Worktree: kind-aware auto-GC TTLs and config knobs
- Worktree: macOS process CWD scan and Unix PID liveness for GC guards
- Worktree: automatic throttled GC on startup (Linux age-based; non-Linux dead-only)
- Pager: add `[ui].combine_queued_prompts` to batch queued follow-ups
- Shell: stop overwriting user skills
- Tools: read markdown in `skills/` directories untruncated
- `/usage` shows per-session token and dollar usage in the TUI
- Security: prompt on environment-dumping `ps` variants
- Security: always-safe `kubectl` no longer runs arbitrary kubeconfig credential plugins without permission
- Tools: make scheduler deletion durable
- Shell: add relocation storage primitives
- Shell: give side model calls their own conversation ids
- Fix five workflow-runtime bugs (budget, pause, cancel, reconnect)
- Security: peel `env -S` / `--split-string` operands in the Bash permission gate (managed deny/ask)
- Pager: expose doctor in the TUI
- Security: block unauthorized RCE via abused safe commands
- Pager idle watcher cue: "1 subagent still running" instead of "watching · 1 subagent"
- Security: block `rg --pre` arbitrary code execution in auto-mode
- Voice: diagnose silent-mic failures (macOS permission) and add doctor/terminal-setup Voice section
- App builder deployer: `allow_forking` and `show_built_with_grok`
- Pager: stop stacking duplicate "Worked for" markers on parked turns
- Shell: support `max` as a distinct reasoning effort tier
- Tools: serialize background `/loop` fires on the whole work unit
- Shell: add working-directory relocation state primitives
- Proto: `ClientToolResult` and `ChatConfig` client-side tools
- Shell: model providers
- Chat: select App Builder product on the Build path
- Shell: attach author identity to feedback when the deployment opts in
- Doctor: fix for SSH wrap setup
- Workflow authoring skills: create-workflow and import-claude-workflow docs
- Add read-only grok doctor
- Sandbox: apply Landlock without a controlling TTY
- Pager: recover image paste over grok wrap on headless remotes
- Pager: make actions screen-mode aware
- Shell: resume sessions when the working directory moves
- Pager: centralize terminal diagnostics
- Workspace: gate inline shell file access
- Pager: centralize terminal probes
- Pager: edit minimal prompts in an external editor
- Pager: standardize backgrounding on Ctrl+B
- Shell: recap rides the parent turn's prompt cache
- Tools: add scheduler lifecycle version clock

Source-Revision: 0f4d7c91b8b2b408333f6de1e8a76cb8eaa71899
This commit is contained in:
grokkybara[bot] 2026-07-21 18:10:23 +00:00
commit 3af4d5d398
556 changed files with 56609 additions and 21892 deletions
Download patch file Download diff file Expand all files Collapse all files

38
crates/codegen/xai-grok-shell/src/auth/auth_provider.rs
View file

@ -53,17 +53,23 @@ pub struct AuthProviderRef {
/// `false` and never mints or reads until [`AuthProviderRef::attach_trusted_config`]
/// joins the shared slot for its name.
resolved: bool,
fail_closed: bool,
}
/// Serialized form: the name only, so persisted bytes never carry a command.
#[derive(serde::Serialize, serde::Deserialize)]
struct AuthProviderRefData {
name: String,
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
fail_closed: bool,
}
impl From<AuthProviderRefData> for AuthProviderRef {
fn from(data: AuthProviderRefData) -> Self {
AuthProviderRef::unresolved(data.name)
if data.fail_closed {
AuthProviderRef::fail_closed(data.name)
} else {
AuthProviderRef::unresolved(data.name)
}
}
}
@ -71,6 +77,7 @@ impl From<AuthProviderRef> for AuthProviderRefData {
fn from(provider: AuthProviderRef) -> Self {
Self {
name: provider.name,
fail_closed: provider.fail_closed,
}
}
}
@ -85,6 +92,7 @@ impl AuthProviderRef {
config,
slot,
resolved: true,
fail_closed: false,
}
}
@ -96,13 +104,31 @@ impl AuthProviderRef {
config: AuthProviderConfig::default(),
slot: ProviderSlot::default(),
resolved: false,
fail_closed: false,
}
}
pub(crate) fn fail_closed(name: String) -> Self {
Self {
name,
config: AuthProviderConfig::default(),
slot: ProviderSlot::default(),
resolved: true,
fail_closed: true,
}
}
pub(crate) fn is_fail_closed(&self) -> bool {
self.fail_closed
}
/// Re-attach the trusted config for this name at model resolution
/// (`None` = the table was removed, leaving an unusable config). The ref
/// becomes authoritative, joins the shared slot for its name, and may mint.
pub(crate) fn attach_trusted_config(&mut self, config: Option<&AuthProviderConfig>) {
if self.fail_closed {
return;
}
self.config = config.cloned().unwrap_or_default();
self.slot = provider_slot(&self.name);
self.resolved = true;
@ -441,7 +467,9 @@ impl AuthProviderRef {
"auth provider removed from config: dropping its cached token"
);
}
warn_empty_command(&self.name);
if !self.fail_closed {
warn_empty_command(&self.name);
}
return None;
}
Some(slot)
@ -455,7 +483,9 @@ impl AuthProviderRef {
return None;
}
if !self.config.is_usable() {
warn_empty_command(&self.name);
if !self.fail_closed {
warn_empty_command(&self.name);
}
return None;
}
// A mint in progress holds the lock; treat it as a miss rather than

134
crates/codegen/xai-grok-shell/src/auth/external_auth.rs
View file

@ -1,5 +1,12 @@
use crate::auth::AuthMode;
use crate::auth::GrokAuth;
use crate::auth::token_output::parse_token_output;
use crate::auth::{AuthMode, GrokAuth};
use crate::util::subprocess::CommandLog;
use crate::util::subprocess::RunError;
use crate::util::subprocess::RunOptions;
use crate::util::subprocess::run_detached_with_timeout;
use crate::util::subprocess::sh_c;
use std::time::Duration;
/// Parse stdout into a session-credential `GrokAuth`.
pub(crate) fn parse_output(output: &std::process::Output) -> anyhow::Result<GrokAuth> {
@ -32,69 +39,42 @@ pub(crate) fn parse_output(output: &std::process::Output) -> anyhow::Result<Grok
})
}
/// Sync version for mid-session refresh. 5s timeout for refresh, 60s for initial.
pub(crate) fn run_external_auth_sync(command: &str, is_refresh: bool) -> Option<GrokAuth> {
let timeout_secs = if is_refresh { 5 } else { 60 };
run_auth_command(command, timeout_secs, is_refresh)
}
/// Short timeout for a mid-session refresh: it must not hang the session.
const EXTERNAL_AUTH_REFRESH_TIMEOUT: Duration = Duration::from_secs(5);
/// Runs `command` via `sh -c`; `mark_expired` sets `GROK_AUTH_EXPIRED=1` so the
/// helper can distinguish re-mints from first runs.
fn run_auth_command(command: &str, timeout_secs: u64, mark_expired: bool) -> Option<GrokAuth> {
use std::process::{Command, Stdio};
/// Runs the external auth binary for a headless mid-session refresh. Initial,
/// interactive sign-in takes a separate path (`flow::run_external_auth_provider`,
/// which bridges the provider's stderr link), so this handles refresh only.
pub(crate) async fn run_external_refresh(command: &str) -> Option<GrokAuth> {
tracing::info!(cmd = %command, timeout_secs = EXTERNAL_AUTH_REFRESH_TIMEOUT.as_secs(), "auth: running external auth provider (headless refresh)");
tracing::info!(cmd = %command, mark_expired, timeout_secs, "auth: running external auth provider (sync)");
let mut cmd = Command::new("sh");
cmd.args(["-c", command])
.stdin(Stdio::null())
.stdout(Stdio::piped())
// Pipe stderr — inherit would corrupt the TUI alternate screen.
.stderr(Stdio::piped());
if mark_expired {
cmd.env("GROK_AUTH_EXPIRED", "1");
}
xai_grok_tools::util::detach_std_command(&mut cmd);
cmd.envs(xai_grok_tools::util::pager_env());
let mut child = cmd.spawn()
.map_err(|e| {
tracing::warn!(error = %e, cmd = %command, "auth: failed to start external auth provider");
e
})
.ok()?;
let timeout = std::time::Duration::from_secs(timeout_secs);
let start = std::time::Instant::now();
loop {
match child.try_wait() {
Ok(Some(_status)) => break,
Ok(None) => {
if start.elapsed() > timeout {
tracing::warn!(
cmd = %command,
timeout_secs,
"auth: external auth provider timed out (likely needs interactive auth), killing"
);
let _ = child.kill();
let _ = child.wait();
return None;
let mut cmd = sh_c(command);
cmd.env("GROK_AUTH_EXPIRED", "1");
// Route through the group-killing runner so a provider that spawns helpers
// is torn down as a unit on timeout.
let output = match run_detached_with_timeout(
cmd,
EXTERNAL_AUTH_REFRESH_TIMEOUT,
RunOptions {
label: "external auth provider",
command_log: CommandLog::Shown(command),
},
)
.await
{
Ok(output) => output,
Err(e) => {
let reason = match e {
RunError::TimedOut => {
"timed out (a timeout usually means it needs interactive sign-in)"
}
std::thread::sleep(std::time::Duration::from_millis(100));
}
Err(e) => {
tracing::warn!(error = %e, "auth: error waiting for external auth provider");
return None;
}
RunError::SpawnFailed => "failed to start",
RunError::WaitFailed => "errored while running",
};
tracing::warn!(cmd = %command, "auth: external auth provider {reason}");
return None;
}
}
let output = child
.wait_with_output()
.map_err(|e| {
tracing::warn!(error = %e, "auth: failed to read external auth provider output");
e
})
.ok()?;
};
match parse_output(&output) {
Ok(auth) => {
@ -109,8 +89,8 @@ fn run_auth_command(command: &str, timeout_secs: u64, mark_expired: bool) -> Opt
}
/// Run external auth provider, carrying forward `/user`-derived fields from previous auth.
pub(crate) fn refresh_with_command(command: &str, prev_auth: &GrokAuth) -> Option<GrokAuth> {
let mut auth = run_external_auth_sync(command, true)?;
pub(crate) async fn refresh_with_command(command: &str, prev_auth: &GrokAuth) -> Option<GrokAuth> {
let mut auth = run_external_refresh(command).await?;
auth.carry_user_profile_from(prev_auth);
Some(auth)
}
@ -189,19 +169,21 @@ mod tests {
assert!(parse_output(&output).is_err());
}
#[test]
fn sync_spawn_failure_returns_none() {
assert!(run_external_auth_sync("/nonexistent/binary", false).is_none());
#[tokio::test]
async fn spawn_failure_returns_none() {
assert!(run_external_refresh("/nonexistent/binary").await.is_none());
}
#[test]
fn sync_sets_grok_auth_expired_env_on_refresh() {
let auth = run_external_auth_sync("echo $GROK_AUTH_EXPIRED", true).unwrap();
#[tokio::test]
async fn sets_grok_auth_expired_env_on_refresh() {
let auth = run_external_refresh("echo $GROK_AUTH_EXPIRED")
.await
.unwrap();
assert_eq!(auth.key, "1");
}
#[test]
fn refresh_carries_zdr_flags_forward() {
#[tokio::test]
async fn refresh_carries_zdr_flags_forward() {
let prev = GrokAuth {
user_blocked_reason: Some("BLOCKED_REASON_OTHER".into()),
team_blocked_reasons: vec!["BLOCKED_REASON_NO_LOGS".into()],
@ -209,7 +191,9 @@ mod tests {
organization_id: Some("org-1".into()),
..GrokAuth::test_default()
};
let auth = refresh_with_command("echo fresh-token", &prev).unwrap();
let auth = refresh_with_command("echo fresh-token", &prev)
.await
.unwrap();
assert_eq!(auth.key, "fresh-token");
assert!(auth.is_zdr_team(), "ZDR flag must survive refresh");
assert!(auth.coding_data_retention_opt_out);
@ -221,12 +205,12 @@ mod tests {
assert_eq!(auth.organization_id.as_deref(), Some("org-1"));
}
#[test]
fn sync_refresh_interactive_times_out() {
// Binary writes link to stderr then blocks — 5s refresh timeout kills it.
#[tokio::test]
async fn refresh_interactive_times_out() {
// Binary writes a link to stderr then blocks; the 5s refresh timeout kills it.
let cmd = r#"echo 'Visit http://example.com/auth' >&2; sleep 20; echo token"#;
let start = std::time::Instant::now();
let result = run_external_auth_sync(cmd, true);
let result = run_external_refresh(cmd).await;
let elapsed = start.elapsed();
assert!(result.is_none(), "should timeout and return None");
assert!(

11
crates/codegen/xai-grok-shell/src/auth/manager.rs
View file

@ -61,10 +61,9 @@ pub(crate) enum RefreshReason {
/// `recovery.rs`.
pub(crate) const AUTH_LOCK_TIMEOUT: StdDuration = StdDuration::from_secs(10);
/// Longer timeout for `refresh_chain` — the critical path that must
/// hold the file lock across the IdP call to prevent refresh-token
/// reuse. Must exceed `EXTERNAL_REFRESH_TIMEOUT` (30 s) so followers
/// wait for the leader to finish rather than timing out and retrying.
/// Lock timeout for `refresh_chain`, held across the IdP call to prevent
/// refresh-token reuse. Must exceed the external-auth refresh timeout
/// (`EXTERNAL_AUTH_REFRESH_TIMEOUT`, 5 s) so followers wait rather than retry.
const REFRESH_LOCK_TIMEOUT: StdDuration = StdDuration::from_secs(45);
/// Long poll interval used by the proactive refresh task when no
@ -957,9 +956,9 @@ impl AuthManager {
/// Run the external auth command and parse its output. Pure: no
/// state mutation, no logging (refresher logs once on its arm).
pub(crate) fn run_external_refresh_command(&self, command: &str) -> Option<GrokAuth> {
pub(crate) async fn run_external_refresh_command(&self, command: &str) -> Option<GrokAuth> {
let prev = self.inner_auth_or_external_default();
crate::auth::refresh_with_command(command, &prev)
crate::auth::refresh_with_command(command, &prev).await
}
/// Hot-swap credentials (called by config watcher). Does NOT write to disk.

101
crates/codegen/xai-grok-shell/src/auth/refresh/external_refresher.rs
View file

@ -3,93 +3,51 @@ use std::sync::Arc;
use crate::auth::error::RefreshTokenFailedReason;
use crate::auth::manager::RefreshReason;
use super::{ExternalCommandRunner, RefreshOutcome, TokenRefresher};
use super::ExternalCommandRunner;
use super::RefreshOutcome;
use super::TokenRefresher;
/// Refreshes by re-running the operator's external auth binary via
/// `spawn_blocking`. Pure data return -- mutation lives in
/// Refreshes by re-running the operator's external auth binary via the async
/// external-command runner. Returns data only; mutation lives in
/// `refresh_chain` (honors the [`TokenRefresher`] no-mutation contract).
pub(crate) struct ExternalBinaryRefresher {
runner: Arc<dyn ExternalCommandRunner>,
command: String,
timeout: std::time::Duration,
}
impl ExternalBinaryRefresher {
pub(crate) fn new(runner: Arc<dyn ExternalCommandRunner>, command: String) -> Self {
Self {
runner,
command,
timeout: EXTERNAL_REFRESH_TIMEOUT,
}
Self { runner, command }
}
/// Override the binary timeout (tests use a short one to exercise the
/// timeout arm without a real 30s wait).
#[cfg(test)]
pub(crate) fn with_timeout(mut self, timeout: std::time::Duration) -> Self {
self.timeout = timeout;
self
}
/// A failed binary run is a single-strike `Other` permanent failure; the
/// `PERMANENT_FAILURE_TTL` lets a flaky binary self-heal without `/login`.
/// No consecutive-blip tolerance like OIDC: a local binary failure is a
/// stronger signal than a network refresh blip.
fn record_failure(&self, message: String) -> RefreshOutcome {
tracing::warn!(%message, "auth: external binary refresh failed -> permanent");
/// A failed or timed-out binary run is a single-strike `Other` permanent
/// failure. `Other` is non-sticky, so `PERMANENT_FAILURE_TTL` lets a flaky
/// or briefly slow binary self-heal without `/login`. The async runner
/// bounds every run and group-kills the child on timeout, so there is no
/// wedged-process case that would need a separate transient outcome.
fn record_failure(&self, message: &str) -> RefreshOutcome {
tracing::warn!(%message, "auth: external binary refresh failed permanently");
// No token key in the binary flow; the caller scopes the verdict.
RefreshOutcome::permanent(RefreshTokenFailedReason::Other, None)
}
}
/// Timeout for the external auth binary. If the binary hangs, the
/// `spawn_blocking` thread is leaked (it cannot be interrupted), but this is
/// acceptable: the thread holds no locks and mutates no shared state.
const EXTERNAL_REFRESH_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(30);
#[async_trait::async_trait]
impl TokenRefresher for ExternalBinaryRefresher {
async fn refresh(&self, reason: RefreshReason) -> RefreshOutcome {
tracing::debug!(?reason, "auth: external binary refresh starting");
let runner = self.runner.clone();
let cmd = self.command.clone();
let timeout_ms = self.timeout.as_millis() as u64;
match tokio::time::timeout(
self.timeout,
tokio::task::spawn_blocking(move || runner.run_external_command(&cmd)),
)
.await
{
Err(_elapsed) => {
// Transient: a hard-expired access token after idle must still
// allow 401 / pre-flight retry. Mapping timeout to permanent
// failure poisoned recovery for PERMANENT_FAILURE_TTL.
tracing::warn!(
timeout_ms,
"auth: external binary refresh timed out (thread leaked)"
);
crate::unified_log::warn(
"auth.refresh.external_timeout",
None,
Some(serde_json::json!({ "timeout_ms": timeout_ms })),
);
RefreshOutcome::transient(format!("external binary timed out after {timeout_ms}ms"))
}
Ok(Ok(Some(auth))) => {
match self.runner.run_external_command(&self.command).await {
Some(auth) => {
crate::unified_log::info("auth: external binary refresh succeeded", None, None);
RefreshOutcome::success(auth)
}
Ok(Ok(None)) => {
None => {
crate::unified_log::warn(
"auth: external binary refresh returned no token",
None,
None,
);
self.record_failure("external binary returned no token".into())
}
Ok(Err(e)) => {
tracing::warn!(error = %e, "auth: external binary refresh task failed");
self.record_failure(format!("external binary task failed: {e}"))
self.record_failure("external binary returned no token")
}
}
}
@ -104,8 +62,9 @@ mod tests {
struct FakeRunner {
external_result: Option<GrokAuth>,
}
#[async_trait::async_trait]
impl ExternalCommandRunner for FakeRunner {
fn run_external_command(&self, _command: &str) -> Option<GrokAuth> {
async fn run_external_command(&self, _command: &str) -> Option<GrokAuth> {
self.external_result.clone()
}
}
@ -134,28 +93,6 @@ mod tests {
}
}
#[tokio::test]
async fn external_binary_timeout_is_transient() {
struct SlowRunner;
impl ExternalCommandRunner for SlowRunner {
fn run_external_command(&self, _command: &str) -> Option<GrokAuth> {
std::thread::sleep(std::time::Duration::from_millis(50));
Some(GrokAuth::test_default())
}
}
let refresher = ExternalBinaryRefresher::new(Arc::new(SlowRunner), "auth-binary".into())
.with_timeout(std::time::Duration::from_millis(5));
match refresher.refresh(RefreshReason::ServerRejected).await {
RefreshOutcome::TransientFailure { message } => {
assert!(
message.contains("timed out"),
"timeout message must be greppable, got {message}"
);
}
other => panic!("a timed-out binary must be TransientFailure, got {other:?}"),
}
}
#[tokio::test]
async fn external_binary_success_returns_fresh_token() {
let token = GrokAuth {

8
crates/codegen/xai-grok-shell/src/auth/refresh/mod.rs
View file

@ -52,14 +52,16 @@ impl AuthSnapshot for AuthManager {
/// Capability to run the operator's external auth binary. Split out of
/// [`AuthSnapshot`] so OIDC refreshers (read-only) physically cannot reach it
/// (interface segregation); only [`ExternalBinaryRefresher`] depends on it.
#[async_trait::async_trait]
pub(crate) trait ExternalCommandRunner: Send + Sync {
/// Run the external auth binary and return the parsed output.
fn run_external_command(&self, command: &str) -> Option<GrokAuth>;
async fn run_external_command(&self, command: &str) -> Option<GrokAuth>;
}
#[async_trait::async_trait]
impl ExternalCommandRunner for AuthManager {
fn run_external_command(&self, command: &str) -> Option<GrokAuth> {
self.run_external_refresh_command(command)
async fn run_external_command(&self, command: &str) -> Option<GrokAuth> {
self.run_external_refresh_command(command).await
}
}