bingshuo/grok-build-upstream-mirror
Watch
1
0
Fork
You've already forked grok-build-upstream-mirror
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Synced from monorepo

Browse source 
Synced from monorepo

Changes:
- grok-shell: send an expired external-provider credential to the sign-in flow, not a 401 loop
- pager: clickable ▲ jumps to the top of the response being read
- grok-shell: keep a large task log from making the completion message too long
- Plan viewer scrollbar: widen grab zone to the border column; fix striped thumb in Terminal.app
- pager: poll the tmux probe teardown grace instead of sleeping it
- security: vendor-compat MCP kill switch is now actually enforced when reported as on
- grok-shell: restore session eviction when a leader client disconnects
- Bump rust-toolchain to 1.93.0
- workspace: lexical-normalize permission path patterns before glob matching
- pager: reject garbage Enter in the /resume picker
- pager: show Mermaid affordances in plan mode preview
- pager: drop manage-account link from /session-info
- workspace: auto-approve read-only git queries; defer write floor to auto classifier
- Add free-form pattern editor to the "Always allow" command prompt
- grok-shell: fix /btw caching
- pager: Tab walks answers in the ask_user_question card
- External-provider auth refresh: single 7s attempt instead of 3×5s
- pager: don't resurrect finished background tasks as Running when completion arrives first
- pager: report tmux truecolor clamping in Doctor
- Fix plan viewer scrollbar click+drag hijacked by comment gutter
- pager/shell: stop double Recap after the same last turn
- sampler: preserve x-should-retry through stream collection
- pager: clear plan-mode indicator immediately when the user approves a plan
- pager: tmux does not re-read its config on reattach

Source-Revision: 64c4de99cc822b25ce9c54ab5a4f372093d0885d
This commit is contained in:
grokkybara[bot] 2026-08-03 08:17:57 +00:00
commit 780d1388ff
323 changed files with 12258 additions and 7226 deletions
Download patch file Download diff file Expand all files Collapse all files

1
crates/codegen/xai-grok-tools/src/bridge.rs
View file

@ -846,6 +846,7 @@ mod tests {
owner_session_id: owner.map(|s| s.to_string()),
description: None,
is_backgrounded: false,
output_total_bytes: 0,
}
}

193
crates/codegen/xai-grok-tools/src/computer/local/lifecycle.rs Normal file
View file

@ -0,0 +1,193 @@
//! Where a task sits between running and evicted, and the proof required to
//! move it. The transitions live here, away from the process plumbing, so the
//! state machine can be tested without a live child.
use std::time::Instant;
use super::ExitStatus;
/// Proof of whether the child was waited on. The field is private:
/// [`Collection::of`] reads the child handle, which tokio clears once a
/// `wait` or `try_wait` has returned, so a call site cannot claim a wait
/// that never happened. [`Collection::ABANDONED`] is always claimable,
/// since that direction only costs more polling.
pub(super) struct Collection(bool);
impl Collection {
pub(super) const ABANDONED: Collection = Collection(false);
pub(super) fn of(child: &tokio::process::Child) -> Collection {
Collection(child.id().is_none())
}
}
/// Each stage carries only what it can have, so a task cannot be collected
/// before it exits, or hold a sweep time before its output is final.
#[derive(Debug, Clone)]
pub(super) enum Lifecycle {
Running,
/// Over, but the pipes still have to be read.
Exiting {
status: ExitStatus,
since: Instant,
},
/// Output is final. A process that will not die reaches this uncollected.
Finished {
status: ExitStatus,
collected: bool,
},
/// The in-memory copy has been dropped for the log on disk. An
/// uncollected child keeps being polled after the sweep, until eviction
/// abandons it.
Swept {
status: ExitStatus,
at: Instant,
collected: bool,
},
}
impl Lifecycle {
pub(super) fn exit_status(&self) -> Option<&ExitStatus> {
match self {
Self::Running => None,
Self::Exiting { status, .. }
| Self::Finished { status, .. }
| Self::Swept { status, .. } => Some(status),
}
}
pub(super) fn has_exited(&self) -> bool {
self.exit_status().is_some()
}
/// Over, with all of its output read.
pub(super) fn is_complete(&self) -> bool {
matches!(self, Self::Finished { .. } | Self::Swept { .. })
}
/// Nothing left for the poll loop: complete *and* the child was waited
/// on. Sweeping does not settle a task on its own.
pub(super) fn is_settled(&self) -> bool {
matches!(
self,
Self::Finished {
collected: true,
..
} | Self::Swept {
collected: true,
..
}
)
}
pub(super) fn swept_at(&self) -> Option<Instant> {
match self {
Self::Running | Self::Exiting { .. } | Self::Finished { .. } => None,
Self::Swept { at, .. } => Some(*at),
}
}
/// Output is final. A late collection upgrades a finished or swept task
/// in place; nothing moves back a stage. No-op before the task exits.
pub(super) fn finish_output(&mut self, collection: Collection) {
let Some(status) = self.exit_status().cloned() else {
return;
};
let collected = collection.0;
*self = match self {
Self::Swept {
at,
collected: already,
..
} => Self::Swept {
status,
at: *at,
collected: *already || collected,
},
Self::Finished {
collected: already, ..
} => Self::Finished {
status,
collected: *already || collected,
},
Self::Running | Self::Exiting { .. } => Self::Finished { status, collected },
};
}
/// Drops to the log on disk. Only a finished task can be swept, and
/// `collected` carries over.
pub(super) fn sweep(&mut self) {
if let Self::Finished { status, collected } = self {
*self = Self::Swept {
status: status.clone(),
at: Instant::now(),
collected: *collected,
};
}
}
}
#[cfg(test)]
mod tests {
use super::{Collection, ExitStatus, Lifecycle};
use std::time::Instant;
fn exiting() -> Lifecycle {
Lifecycle::Exiting {
status: ExitStatus {
exit_code: None,
signal: Some("timeout".to_owned()),
},
since: Instant::now(),
}
}
/// The walk the out-of-memory and give-up kill paths take. Both once
/// settled a task whose child was never collected; every step here pins
/// the boundary they crossed.
#[test]
fn a_kill_without_a_reap_keeps_the_task_polled_until_collected() {
let mut lifecycle = exiting();
assert!(lifecycle.has_exited());
assert!(!lifecycle.is_complete(), "exited is not yet complete");
lifecycle.finish_output(Collection::ABANDONED);
assert!(lifecycle.is_complete(), "output is final, so waits answer");
assert!(!lifecycle.is_settled(), "the child still needs a try_wait");
lifecycle.sweep();
assert!(!lifecycle.is_settled(), "sweeping must not end the polling");
lifecycle.finish_output(Collection(true));
assert!(lifecycle.is_settled(), "the late reap settles it");
assert!(lifecycle.swept_at().is_some(), "and it stays swept");
}
/// Output still being read must not be dropped.
#[test]
fn a_task_still_draining_cannot_be_swept() {
let mut lifecycle = exiting();
lifecycle.sweep();
assert!(lifecycle.swept_at().is_none());
}
/// The evidence reads the child handle: no collection can be claimed
/// until a `wait` has returned.
#[cfg(unix)]
#[tokio::test]
async fn a_collection_claim_requires_the_child_to_have_been_waited_on() {
let mut child = tokio::process::Command::new("true")
.spawn()
.expect("spawn `true`");
let mut lifecycle = exiting();
lifecycle.finish_output(Collection::of(&child));
assert!(!lifecycle.is_settled(), "no wait has returned");
child.wait().await.expect("wait");
lifecycle.finish_output(Collection::of(&child));
assert!(lifecycle.is_settled(), "the wait is the evidence");
}
}

4
crates/codegen/xai-grok-tools/src/computer/local/mod.rs
View file

@ -8,6 +8,10 @@ pub mod shell_state;
#[cfg(unix)]
pub mod static_shell;
pub mod terminal;
// Unix only, because the tests build their logs with shell tools.
// See `computer::task_log` for the tests that run everywhere.
#[cfg(all(test, unix))]
mod terminal_snapshot_tests;
pub use cgroup::{CgroupMemoryConfig, PROCESS_OOM_EXIT_CODE};
pub use file_system::LocalFs;

282
crates/codegen/xai-grok-tools/src/computer/local/terminal.rs
View file

@ -18,11 +18,13 @@ use tokio_util::sync::CancellationToken;
use crate::computer::local::cgroup::{
CgroupGuard, CgroupMemoryConfig, MemoryMonitor, PROCESS_OOM_EXIT_CODE,
};
use crate::computer::task_log;
use crate::computer::types::{
BackgroundHandle, ComputerError, KillOutcome, TaskSnapshot, TerminalBackend,
TerminalRunRequest, TerminalRunResult,
};
use crate::notification::types::{BashNotificationBase, BashOutputChunk, ToolNotificationHandle};
use crate::util::truncate::FRONT_BACK_TRUNCATION_MARKER;
use super::SearchShadowConfig;
#[cfg(unix)]
@ -77,6 +79,9 @@ fn output_file_cap_from_env() -> u64 {
/// Max time to drain stdout/stderr after process exit. Prevents `cmd &`
/// (inherited pipe, no redirect) from blocking the actor loop forever.
const DRAIN_TIMEOUT: Duration = Duration::from_secs(2);
/// How long completion waits on a kill before taking the output there is: a
/// process that never dies must not hold its task open forever.
const REAP_GRACE: Duration = Duration::from_secs(5);
/// Max bytes retained in the output file after process exit. Truncated
/// so `to_task_snapshot` / `read_file` don't materialize huge strings.
const MAX_RETAINED_OUTPUT_FILE_BYTES: u64 = 64 * 1024 * 1024; // 64 MiB
@ -89,6 +94,10 @@ fn notification_interval() -> Duration {
Duration::from_millis(DEFAULT_NOTIFICATION_INTERVAL_MS)
}
#[path = "lifecycle.rs"]
mod lifecycle;
use lifecycle::{Collection, Lifecycle};
/// Exit status of a terminal process
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct ExitStatus {
@ -243,8 +252,7 @@ struct ProcessState {
truncated: bool,
/// Total bytes written to file (before truncation)
total_bytes: usize,
/// Exit status once process completes
exit_status: Option<ExitStatus>,
lifecycle: Lifecycle,
/// Whether process was backgrounded and how
bg_status: BackgroundStatus,
/// Waiters for this process to complete (foreground only)
@ -267,8 +275,6 @@ struct ProcessState {
cwd: String,
/// Wall-clock start time (for TaskSnapshot)
start_wall_time: std::time::SystemTime,
/// When the process completed (for TTL-based eviction of background tasks)
completed_at: Option<Instant>,
/// Wall-clock end time (for TaskSnapshot duration calculation)
end_wall_time: Option<std::time::SystemTime>,
@ -285,10 +291,6 @@ struct ProcessState {
/// is a truncated tail that *shrinks* once `maybe_truncate` fires; a
/// length-based gate would go (and stay) false after truncation.
last_notified_total: usize,
/// Whether stdout/stderr have already been drained after exit.
/// Prevents repeated 2s drain timeouts on every poll tick when
/// orphaned children hold pipes open.
drained: bool,
/// Set when a `block=true` waiter consumed this task's result.
block_waited: bool,
/// Set when the model explicitly killed this task via the kill tool,
@ -308,28 +310,17 @@ struct ProcessState {
impl ProcessState {
fn to_result(&self) -> TerminalRunResult {
let combined_output = if let Some(ref front) = self.front_buffer {
let front_str = String::from_utf8_lossy(front);
let back_str = String::from_utf8_lossy(&self.output_buffer);
format!(
"{}\n\n... (output truncated) ...\n\n{}",
front_str.trim_end(),
back_str.trim_start()
)
} else {
String::from_utf8_lossy(&self.output_buffer).into_owned()
};
TerminalRunResult {
combined_output,
exit_code: self.exit_status.as_ref().and_then(|s| s.exit_code),
combined_output: self.ring_output(),
exit_code: self.lifecycle.exit_status().and_then(|s| s.exit_code),
truncated: self.truncated,
signal: match self.bg_status {
BackgroundStatus::Backgrounded { reason } => Some(reason.as_signal().to_string()),
_ => self.exit_status.as_ref().and_then(|s| s.signal.clone()),
_ => self.lifecycle.exit_status().and_then(|s| s.signal.clone()),
},
timed_out: self
.exit_status
.as_ref()
.lifecycle
.exit_status()
.map(|s| s.signal.as_deref() == Some("timeout"))
.unwrap_or(false),
output_file: self.output_file.clone(),
@ -396,25 +387,32 @@ impl ProcessState {
self.start_time.elapsed() > self.timeout
}
fn is_complete(&self) -> bool {
self.lifecycle.is_complete()
}
/// The output is not final until `finish_output`.
fn mark_exited(&mut self, status: ExitStatus) {
if !self.lifecycle.has_exited() {
self.lifecycle = Lifecycle::Exiting {
status,
since: Instant::now(),
};
}
}
fn finish_output(&mut self, collection: Collection) {
self.lifecycle.finish_output(collection);
}
/// Build a snapshot of this process's current state.
/// Uses async I/O to read output from disk for completed background tasks.
async fn to_task_snapshot(&self, task_id: &str) -> TaskSnapshot {
// For completed background tasks, the in-memory buffer is cleared to free
// memory. Fall back to reading from the output file (non-blocking).
let output = if self.output_buffer.is_empty() && self.exit_status.is_some() {
tokio::fs::read_to_string(&self.output_file)
.await
.unwrap_or_default()
} else if let Some(ref front) = self.front_buffer {
let front_str = String::from_utf8_lossy(front);
let back_str = String::from_utf8_lossy(&self.output_buffer);
format!(
"{}\n\n... (output truncated) ...\n\n{}",
front_str.trim_end(),
back_str.trim_start()
)
let swept = matches!(self.lifecycle, Lifecycle::Swept { .. });
let (output, short_of_full_log) = if swept && !self.output_file.as_os_str().is_empty() {
task_log::read_prefix(&self.output_file, task_log::MAX_SNAPSHOT_BYTES).await
} else {
String::from_utf8_lossy(&self.output_buffer).into_owned()
(self.ring_output(), false)
};
TaskSnapshot {
@ -423,7 +421,7 @@ impl ProcessState {
display_command: self.display_command.clone(),
cwd: self.cwd.clone(),
start_time: self.start_wall_time,
end_time: if self.exit_status.is_some() {
end_time: if self.lifecycle.has_exited() {
// Use the recorded wall-clock end time if available,
// otherwise fall back to now (process just completed this tick).
Some(
@ -435,10 +433,11 @@ impl ProcessState {
},
output,
output_file: self.output_file.clone(),
truncated: self.truncated,
exit_code: self.exit_status.as_ref().and_then(|s| s.exit_code),
signal: self.exit_status.as_ref().and_then(|s| s.signal.clone()),
completed: self.exit_status.is_some(),
truncated: self.truncated || short_of_full_log,
output_total_bytes: self.total_bytes,
exit_code: self.lifecycle.exit_status().and_then(|s| s.exit_code),
signal: self.lifecycle.exit_status().and_then(|s| s.signal.clone()),
completed: self.is_complete(),
block_waited: self.block_waited,
explicitly_killed: self.explicitly_killed,
kind: self.kind,
@ -447,6 +446,19 @@ impl ProcessState {
is_backgrounded: self.bg_status.is_backgrounded(),
}
}
/// Output held in memory: the latest part, after the earliest part once
/// the task has run past its live limit.
fn ring_output(&self) -> String {
match self.front_buffer.as_ref() {
Some(front) => format!(
"{}{FRONT_BACK_TRUNCATION_MARKER}{}",
String::from_utf8_lossy(front).trim_end(),
String::from_utf8_lossy(&self.output_buffer).trim_start()
),
None => String::from_utf8_lossy(&self.output_buffer).into_owned(),
}
}
}
// ============================================================================
@ -1079,7 +1091,7 @@ impl LocalTerminalActor {
front_buffer: None,
truncated: false,
total_bytes: 0,
exit_status: None,
lifecycle: Lifecycle::Running,
bg_status: BackgroundStatus::Foreground {
auto_bg_on_timeout: request.auto_background_on_timeout,
},
@ -1096,13 +1108,11 @@ impl LocalTerminalActor {
display_command: request.display_command.clone(),
cwd: request.working_directory.display().to_string(),
start_wall_time: std::time::SystemTime::now(),
completed_at: None,
end_wall_time: None,
notification_handle: request.notification_handle.clone(),
tool_call_id: request.tool_call_id.clone(),
kind: request.kind,
last_notified_total: 0,
drained: false,
block_waited: false,
explicitly_killed: false,
state_dump_handle,
@ -1133,7 +1143,7 @@ impl LocalTerminalActor {
return KillOutcome::NotFound;
};
if process.exit_status.is_some() {
if process.lifecycle.has_exited() {
return KillOutcome::AlreadyExited;
}
@ -1211,7 +1221,7 @@ impl LocalTerminalActor {
front_buffer: None,
truncated: false,
total_bytes: 0,
exit_status: None,
lifecycle: Lifecycle::Running,
bg_status: BackgroundStatus::Backgrounded {
reason: BackgroundReason::Explicit,
},
@ -1229,13 +1239,11 @@ impl LocalTerminalActor {
display_command: request.display_command.clone(),
cwd: request.working_directory.display().to_string(),
start_wall_time: std::time::SystemTime::now(),
completed_at: None,
end_wall_time: None,
notification_handle: request.notification_handle.clone(),
tool_call_id: request.tool_call_id.clone(),
kind: request.kind,
last_notified_total: 0,
drained: false,
block_waited: false,
explicitly_killed: false,
// Background commands don't update the canonical shell state —
@ -1306,7 +1314,7 @@ impl LocalTerminalActor {
let prev_block_waited = process.block_waited;
process.block_waited = true;
if process.exit_status.is_some() {
if process.is_complete() {
let snapshot = process.to_task_snapshot(&task_id).await;
if reply.send(Some(snapshot)).is_err() {
// Receiver dropped (e.g. the awaiting turn was cancelled):
@ -1346,7 +1354,7 @@ impl LocalTerminalActor {
let newest_id = self
.processes
.iter()
.filter(|(_, p)| p.exit_status.is_none())
.filter(|(_, p)| !p.lifecycle.has_exited())
.max_by_key(|(_, p)| p.start_time)
.map(|(id, _)| id.clone());
@ -1355,12 +1363,13 @@ impl LocalTerminalActor {
{
send_sigkill_to_group(process);
drain_remaining_output(process).await;
process.exit_status = Some(ExitStatus {
process.mark_exited(ExitStatus {
exit_code: Some(PROCESS_OOM_EXIT_CODE),
signal: Some("oom".to_owned()),
});
process.end_wall_time = Some(std::time::SystemTime::now());
process.flush_and_truncate_output_file().await;
process.finish_output(Collection::of(&process.child));
let result = Ok(process.to_result());
process.notify_waiters(result);
}
@ -1372,7 +1381,7 @@ impl LocalTerminalActor {
.iter()
.filter(|(_, p)| {
p.bg_status.is_backgrounded()
&& p.exit_status.is_none()
&& !p.lifecycle.has_exited()
&& p.start_time.elapsed() > BACKGROUND_MAX_RUNTIME
})
.map(|(id, _)| id.clone())
@ -1384,7 +1393,7 @@ impl LocalTerminalActor {
// Fire-and-forget SIGTERM — poll loop escalates to SIGKILL
// on the next tick if the process doesn't exit.
send_sigterm_to_group(process);
process.exit_status = Some(ExitStatus {
process.mark_exited(ExitStatus {
exit_code: None,
signal: Some("max_runtime".to_owned()),
});
@ -1399,7 +1408,7 @@ impl LocalTerminalActor {
let size_exceeded: Vec<String> = self
.processes
.iter()
.filter(|(_, p)| p.exit_status.is_none() && p.total_bytes as u64 > output_cap)
.filter(|(_, p)| !p.lifecycle.has_exited() && p.total_bytes as u64 > output_cap)
.map(|(id, _)| id.clone())
.collect();
@ -1414,7 +1423,7 @@ impl LocalTerminalActor {
// Fire-and-forget SIGTERM — poll loop escalates to SIGKILL
// on the next tick if the process doesn't exit.
send_sigterm_to_group(process);
process.exit_status = Some(ExitStatus {
process.mark_exited(ExitStatus {
exit_code: None,
signal: Some("output_limit".to_owned()),
});
@ -1462,7 +1471,7 @@ impl LocalTerminalActor {
continue;
};
// Only foreground processes update the canonical state.
if process.exit_status.is_none() || process.bg_status.is_backgrounded() {
if !process.lifecycle.has_exited() || process.bg_status.is_backgrounded() {
continue;
}
process.state_dump_handle.take()
@ -1492,7 +1501,7 @@ impl LocalTerminalActor {
let completed = self
.processes
.get(&task_id)
.map(|p| p.exit_status.is_some())
.map(ProcessState::is_complete)
.unwrap_or(true); // process gone = treat as completed
if completed && let Some(waiters) = self.completion_waiters.remove(&task_id) {
@ -1555,33 +1564,29 @@ impl LocalTerminalActor {
}
}
// 3. Set completed_at and clear output buffer for completed background tasks
// 3. Sweep finished background tasks: drop the in-memory copy
// First pass: mark completed and clear buffers, collect IDs for notification
let mut newly_completed: Vec<String> = Vec::new();
for (task_id, process) in self.processes.iter_mut() {
if process.exit_status.is_some()
if process.is_complete()
&& process.bg_status.is_backgrounded()
&& process.completed_at.is_none()
&& process.lifecycle.swept_at().is_none()
{
process.completed_at = Some(Instant::now());
process.lifecycle.sweep();
if process.end_wall_time.is_none() {
process.end_wall_time = Some(std::time::SystemTime::now());
}
// Drop in-memory buffer — output file on disk has the full content
// The log file has everything, and a drained task adds no more.
process.output_buffer.clear();
process.front_buffer = None;
newly_completed.push(task_id.clone());
}
}
// Second pass: send completion notifications (requires async file read).
//
// The `block_waited` gate that suppresses the redundant auto-wake
// synthetic prompt for awaited tasks lives in
// `tools/notification_bridge.rs` (the `TaskCompleted` arm checks
// `task_snapshot.block_waited` before the auto-wake injection
// branch — see the comment there). This pass must still fire
// `send_task_complete` unconditionally for newly-completed
// background tasks so the pager UI, persistence, and
// `TaskCompletionReservations` bookkeeping all still get the snapshot.
// Fires unconditionally: the pager UI, persistence, and reservation
// bookkeeping all need the snapshot. The auto-wake suppression for
// awaited tasks lives in the bridge's `TaskCompleted` arm.
for task_id in newly_completed {
if let Some(process) = self.processes.get(&task_id) {
let snapshot = process.to_task_snapshot(&task_id).await;
@ -1596,14 +1601,14 @@ impl LocalTerminalActor {
.processes
.iter()
.filter(|(_, p)| {
if p.exit_status.is_none() {
if !p.lifecycle.has_exited() {
return false; // still running, keep
}
if !p.bg_status.is_backgrounded() {
return true; // foreground, already replied, evict
}
// Backgrounded + completed: evict after TTL
matches!(p.completed_at, Some(t) if t.elapsed() >= self.completed_task_ttl)
matches!(p.lifecycle.swept_at(), Some(t) if t.elapsed() >= self.completed_task_ttl)
})
.map(|(id, _)| id.clone())
.collect();
@ -1624,9 +1629,10 @@ impl LocalTerminalActor {
end_time: p.end_wall_time,
output: String::new(),
output_file: p.output_file.clone(),
truncated: p.truncated,
exit_code: p.exit_status.as_ref().and_then(|s| s.exit_code),
signal: p.exit_status.as_ref().and_then(|s| s.signal.clone()),
// The output is dropped here; the log file keeps it.
truncated: p.truncated || p.total_bytes > 0,
exit_code: p.lifecycle.exit_status().and_then(|s| s.exit_code),
signal: p.lifecycle.exit_status().and_then(|s| s.signal.clone()),
completed: true,
kind: p.kind,
block_waited: p.block_waited,
@ -1634,6 +1640,7 @@ impl LocalTerminalActor {
owner_session_id: p.owner_session_id.clone(),
description: p.description.clone(),
is_backgrounded: true,
output_total_bytes: p.total_bytes,
};
self.completed_task_snapshots.insert(id.clone(), snapshot);
}
@ -1659,29 +1666,39 @@ impl LocalTerminalActor {
return;
};
// If exit_status is already set (e.g., by timeout handler or external signal),
// the process may still be running. Escalate to SIGKILL if needed, and drain
// output once it exits.
if process.exit_status.is_some() {
if process.drained {
// Already drained — nothing left to do for this process.
// An exited task may still hold a live child. Escalate to SIGKILL if
// needed, drain the pipes once it dies, and keep trying to collect it.
if process.lifecycle.has_exited() {
if process.lifecycle.is_settled() {
return;
}
let waiting_since = match &process.lifecycle {
Lifecycle::Exiting { since, .. } => Some(*since),
Lifecycle::Running | Lifecycle::Finished { .. } | Lifecycle::Swept { .. } => None,
};
match process.child.try_wait() {
Ok(None) if process.is_complete() => {
// Already given up on this one; keep the kill signal fresh
// and keep trying to collect it.
send_sigkill_to_group(process);
}
Ok(None) => {
// Process was told to die but is still running — escalate to SIGKILL
send_sigkill_to_group(process);
let gave_up = waiting_since.is_some_and(|since| since.elapsed() >= REAP_GRACE);
if gave_up {
// It is not dying. Take the output there is so the task
// can report completion instead of waiting forever.
take_available_output(process).await;
process.flush_and_truncate_output_file().await;
process.finish_output(Collection::ABANDONED);
}
}
Ok(Some(_)) => {
// Process finally exited — drain any remaining output
Ok(Some(_)) | Err(_) => {
// A second drain is harmless: the first one closes the pipes.
drain_remaining_output(process).await;
process.flush_and_truncate_output_file().await;
process.drained = true;
}
Err(_) => {
drain_remaining_output(process).await;
process.flush_and_truncate_output_file().await;
process.drained = true;
process.finish_output(Collection::of(&process.child));
}
}
return;
@ -1788,7 +1805,7 @@ impl LocalTerminalActor {
// can override via BashParams.foreground_block_budget_ms (0 = disable
// short budget so only `timeout` auto-bgs). The `timeout` check below
// also auto-bgs when auto_bg is on, or kills when it is off.
if process.exit_status.is_none()
if !process.lifecycle.has_exited()
&& matches!(
process.bg_status,
BackgroundStatus::Foreground {
@ -1802,7 +1819,7 @@ impl LocalTerminalActor {
}
// Check for timeout.
if process.is_timed_out() && process.exit_status.is_none() {
if process.is_timed_out() && !process.lifecycle.has_exited() {
if matches!(
process.bg_status,
BackgroundStatus::Foreground {
@ -1815,7 +1832,7 @@ impl LocalTerminalActor {
// Default: kill the process on timeout.
send_sigterm_to_group(process);
process.exit_status = Some(ExitStatus {
process.mark_exited(ExitStatus {
exit_code: None,
signal: Some("timeout".to_owned()),
});
@ -1836,9 +1853,10 @@ impl LocalTerminalActor {
// buffers are read, resulting in empty output.
drain_remaining_output(process).await;
process.exit_status = Some(extract_exit_status(status));
process.mark_exited(extract_exit_status(status));
process.end_wall_time = Some(std::time::SystemTime::now());
process.flush_and_truncate_output_file().await;
process.finish_output(Collection::of(&process.child));
let result = Ok(process.to_result());
process.notify_waiters(result);
}
@ -1849,12 +1867,16 @@ impl LocalTerminalActor {
// Still running
}
Err(e) => {
process.exit_status = Some(ExitStatus {
drain_remaining_output(process).await;
process.mark_exited(ExitStatus {
exit_code: None,
signal: Some(format!("error: {}", e)),
});
process.end_wall_time = Some(std::time::SystemTime::now());
process.flush_and_truncate_output_file().await;
// An erroring `try_wait` is no proof the child was
// collected; keep polling.
process.finish_output(Collection::of(&process.child));
let result = Ok(process.to_result());
process.notify_waiters(result);
}
@ -1920,7 +1942,7 @@ impl LocalTerminalActor {
let fg_ids: Vec<String> = self
.processes
.iter()
.filter(|(_, p)| !p.bg_status.is_backgrounded() && p.exit_status.is_none())
.filter(|(_, p)| !p.bg_status.is_backgrounded() && !p.lifecycle.has_exited())
.map(|(id, _)| id.clone())
.collect();
@ -1945,7 +1967,7 @@ impl LocalTerminalActor {
handle.abort();
}
process.exit_status = Some(ExitStatus {
process.mark_exited(ExitStatus {
exit_code: None,
signal: Some("cancelled".to_owned()),
});
@ -1972,7 +1994,7 @@ impl LocalTerminalActor {
.filter(|(_, p)| {
p.owner_session_id.as_deref() == Some(owner_session_id)
&& !p.bg_status.is_backgrounded()
&& p.exit_status.is_none()
&& !p.lifecycle.has_exited()
})
.map(|(id, _)| id.clone())
.collect();
@ -1986,7 +2008,7 @@ impl LocalTerminalActor {
if let Some(handle) = process.state_dump_handle.take() {
handle.abort();
}
process.exit_status = Some(ExitStatus {
process.mark_exited(ExitStatus {
exit_code: None,
signal: Some("cancelled".to_owned()),
});
@ -2010,7 +2032,7 @@ impl LocalTerminalActor {
.iter()
.filter(|(_, p)| {
p.owner_session_id.as_deref() == Some(owner_session_id)
&& p.exit_status.is_none()
&& !p.lifecycle.has_exited()
&& p.bg_status.is_backgrounded()
})
.map(|(id, _)| id.clone())
@ -2044,7 +2066,7 @@ impl LocalTerminalActor {
for (task_id, process) in self.processes.iter_mut() {
if process.owner_session_id.as_deref() == Some(old_owner_session_id)
&& process.bg_status.is_backgrounded()
&& process.exit_status.is_none()
&& !process.lifecycle.has_exited()
{
// Only reparent backgrounded, still-running tasks. Foreground
// processes keep the child's owner_session_id so the subsequent
@ -2739,6 +2761,41 @@ async fn drain_remaining_output(process: &mut ProcessState) {
process.maybe_truncate();
}
/// Take the output already sitting in the pipes, then drop the handles.
/// Never waits: a live pipe would hold the single threaded actor for the
/// full drain timeout, so this is safe on a process that is still running.
async fn take_available_output(process: &mut ProcessState) {
let mut collected = Vec::new();
if let Some(stdout) = process.child.stdout.as_mut() {
read_available(stdout, &mut collected);
}
if let Some(stderr) = process.child.stderr.as_mut() {
read_available(stderr, &mut collected);
}
process.child.stdout.take();
process.child.stderr.take();
if collected.is_empty() {
return;
}
process.output_buffer.extend_from_slice(&collected);
process.total_bytes += collected.len();
if let Some(file) = process.file_handle.as_mut() {
let _ = file.write_all(&collected).await;
}
process.maybe_truncate();
}
fn read_available(reader: &mut (impl tokio::io::AsyncRead + Unpin), out: &mut Vec<u8>) {
let mut buf = [0u8; READ_BUFFER_SIZE];
loop {
match try_read_nonblocking(reader, &mut buf) {
Some(Ok(0)) | Some(Err(_)) | None => return,
Some(Ok(n)) => out.extend_from_slice(&buf[..n]),
}
}
}
/// Two-phase kill that synchronously waits for the process to exit.
/// Used ONLY by `kill_and_finalize` (the explicit kill_task API) where
/// the caller expects the process to be dead when the call returns.
@ -2787,7 +2844,7 @@ async fn graceful_kill_and_wait(process: &mut ProcessState) {
)]
async fn kill_and_finalize(process: &mut ProcessState) -> KillOutcome {
// Already reaped between the caller's check and here (race with poll_process)
if process.exit_status.is_some() {
if process.lifecycle.has_exited() {
return KillOutcome::AlreadyExited;
}
@ -2829,13 +2886,15 @@ async fn kill_and_finalize(process: &mut ProcessState) -> KillOutcome {
KillOutcome::Killed
}
/// Set exit_status, flush the output file, and notify foreground waiters.
/// Mark the task exited, flush the output file, and notify foreground
/// waiters. Callers read the remaining output first. A process that could
/// not be collected stays unsettled, so the poll loop keeps trying.
async fn finalize_process(process: &mut ProcessState, status: Option<std::process::ExitStatus>) {
if process.exit_status.is_some() {
if process.lifecycle.has_exited() {
return;
}
process.exit_status = Some(match status {
process.mark_exited(match status {
Some(s) => extract_exit_status(s),
None => ExitStatus {
exit_code: None,
@ -2847,6 +2906,7 @@ async fn finalize_process(process: &mut ProcessState, status: Option<std::proces
}
process.flush_and_truncate_output_file().await;
process.finish_output(Collection::of(&process.child));
let result = Ok(process.to_result());
process.notify_waiters(result);
@ -5008,6 +5068,14 @@ mod tests {
assert!(snap_after.completed);
assert_eq!(snap_after.exit_code, Some(0));
assert_eq!(snap_after.task_id, bg.task_id);
// The tombstone drops the output but still reports the size the task
// produced, so it has to say the output is incomplete.
assert!(snap_after.output.is_empty());
assert!(snap_after.output_total_bytes > 0);
assert!(
snap_after.truncated,
"a tombstone that reports bytes must not claim complete output"
);
// 5. list_tasks should include the evicted task.
let all = backend.list_tasks().await;

135
crates/codegen/xai-grok-tools/src/computer/local/terminal_snapshot_tests.rs Normal file
View file

@ -0,0 +1,135 @@
//! Completion snapshots for background tasks, through the terminal actor.
//! Unix only: the test logs are built with `head`, `tr`, and `/dev/zero`.
use std::collections::HashMap;
use std::time::Duration;
use pretty_assertions::assert_eq;
use crate::computer::task_log::MAX_SNAPSHOT_BYTES;
use crate::computer::types::{TaskKind, TaskSnapshot, TerminalBackend, TerminalRunRequest};
use crate::notification::types::ToolNotificationHandle;
use crate::util::truncate::FRONT_BACK_TRUNCATION_MARKER;
use super::LocalTerminalBackend;
/// The snapshot a later `get_task_output` sees, once output has left memory.
async fn snapshot_after_completion(command: &str, output_byte_limit: usize) -> TaskSnapshot {
let (tx, _rx) = tokio::sync::mpsc::unbounded_channel();
let backend = LocalTerminalBackend::new();
let tmp = tempfile::TempDir::new().unwrap();
let request = TerminalRunRequest {
command: command.to_string(),
working_directory: tmp.path().to_path_buf(),
env: HashMap::new(),
timeout: Duration::from_secs(60),
output_byte_limit,
output_file: tmp.path().join("task.log"),
notification_handle: ToolNotificationHandle::from_sender(tx),
tool_call_id: "snapshot-call".to_string(),
display_command: None,
auto_background_on_timeout: false,
foreground_block_budget: None,
kind: TaskKind::Bash,
owner_session_id: None,
description: None,
};
let bg = backend.run_background(request).await.unwrap();
backend
.wait_for_completion(&bg.task_id, Some(Duration::from_secs(30)))
.await
.expect("bg task should complete");
for _ in 0..100 {
let snapshot = backend.get_task(&bg.task_id).await.expect("task snapshot");
if !snapshot
.output
.contains(FRONT_BACK_TRUNCATION_MARKER.trim())
{
return snapshot;
}
tokio::time::sleep(Duration::from_millis(50)).await;
}
panic!("completed task never reloaded its output from disk");
}
/// The in-memory limit applies while a task runs, not to what it reports at
/// the end. `truncated` stays true because the task did run past that limit.
#[tokio::test]
async fn a_finished_task_reports_its_whole_log() {
let snapshot = snapshot_after_completion(
"head -c 50000 /dev/zero | tr '\\0' 'X'",
/*output_byte_limit*/ 500,
)
.await;
assert_eq!(snapshot.output, "X".repeat(50_000));
}
#[tokio::test]
async fn a_log_past_the_bound_is_cut_and_marked() {
let bytes = MAX_SNAPSHOT_BYTES + 200_000;
let snapshot = snapshot_after_completion(
&format!("head -c {bytes} /dev/zero | tr '\\0' 'X'"),
/*output_byte_limit*/ 500,
)
.await;
assert_eq!(snapshot.output.len(), MAX_SNAPSHOT_BYTES);
assert!(snapshot.truncated);
assert!(
snapshot.output_total_bytes >= bytes,
"the snapshot holds {} bytes but must report the task's {bytes}",
snapshot.output.len()
);
}
#[tokio::test]
async fn a_finished_task_reports_completed_with_its_output() {
let snapshot = snapshot_after_completion("echo done", /*output_byte_limit*/ 10_000).await;
assert!(snapshot.completed);
assert_eq!(snapshot.output, "done\n");
}
#[tokio::test]
async fn a_missing_log_is_not_reported_as_empty_output() {
let (tx, _rx) = tokio::sync::mpsc::unbounded_channel();
let backend = LocalTerminalBackend::new();
let tmp = tempfile::TempDir::new().unwrap();
let output_file = tmp.path().join("task.log");
let request = TerminalRunRequest {
command: "echo gone".to_string(),
working_directory: tmp.path().to_path_buf(),
env: HashMap::new(),
timeout: Duration::from_secs(60),
output_byte_limit: 10_000,
output_file: output_file.clone(),
notification_handle: ToolNotificationHandle::from_sender(tx),
tool_call_id: "missing-log-call".to_string(),
display_command: None,
auto_background_on_timeout: false,
foreground_block_budget: None,
kind: TaskKind::Bash,
owner_session_id: None,
description: None,
};
let bg = backend.run_background(request).await.unwrap();
backend
.wait_for_completion(&bg.task_id, Some(Duration::from_secs(30)))
.await
.expect("bg task should complete");
for _ in 0..100 {
tokio::fs::remove_file(&output_file).await.ok();
let snapshot = backend.get_task(&bg.task_id).await.expect("task snapshot");
if snapshot.output.is_empty() {
assert!(snapshot.truncated);
return;
}
tokio::time::sleep(Duration::from_millis(50)).await;
}
panic!("snapshot never fell back to the deleted log");
}

1
crates/codegen/xai-grok-tools/src/computer/mod.rs
View file

@ -1,3 +1,4 @@
pub mod local;
pub(crate) mod task_log;
/// Contains the computer implementation
pub mod types;

54
crates/codegen/xai-grok-tools/src/computer/task_log.rs Normal file
View file

@ -0,0 +1,54 @@
//! Reads part of a task's log file without loading all of it.
use std::path::Path;
use tokio::io::AsyncReadExt;
/// Far more than any tool shows the model, so this bounds memory only. The
/// assertion pins the built-in budget; a runtime budget raised past this
/// would see the prefix instead of the whole log.
pub(crate) const MAX_SNAPSHOT_BYTES: usize = 1024 * 1024;
const _: () = assert!(MAX_SNAPSHOT_BYTES > crate::DEFAULT_TOOL_OUTPUT_BYTES);
/// Reads up to `max_bytes` from the start of `path`, and whether the file
/// continues past it. An unreadable file reads as empty and incomplete.
pub(crate) async fn read_prefix(path: &Path, max_bytes: usize) -> (String, bool) {
match read_bounded(path, max_bytes).await {
Ok((bytes, more)) => {
let (text, cut) = decode(&bytes);
(text, more || cut)
}
Err(error) => {
tracing::debug!(%error, path = %path.display(), "task log could not be read");
(String::new(), true)
}
}
}
async fn read_bounded(path: &Path, max_bytes: usize) -> std::io::Result<(Vec<u8>, bool)> {
let file = tokio::fs::File::open(path).await?;
let mut buf = Vec::new();
file.take(max_bytes as u64 + 1)
.read_to_end(&mut buf)
.await?;
let more = buf.len() > max_bytes;
buf.truncate(max_bytes);
Ok((buf, more))
}
/// The flag reports a trailing split character that was dropped, so a cut
/// cannot read as the whole log.
fn decode(buf: &[u8]) -> (String, bool) {
match std::str::from_utf8(buf) {
Ok(text) => (text.to_owned(), false),
Err(error) if error.error_len().is_none() => (
String::from_utf8_lossy(&buf[..error.valid_up_to()]).into_owned(),
true,
),
Err(_) => (String::from_utf8_lossy(buf).into_owned(), false),
}
}
#[cfg(test)]
#[path = "task_log_tests.rs"]
mod tests;

73
crates/codegen/xai-grok-tools/src/computer/task_log_tests.rs Normal file
View file

@ -0,0 +1,73 @@
use pretty_assertions::assert_eq;
use super::*;
#[tokio::test]
async fn a_log_at_the_budget_is_complete() {
let dir = tempfile::TempDir::new().unwrap();
let path = dir.path().join("exact.log");
let body = "y".repeat(64);
tokio::fs::write(&path, &body).await.unwrap();
assert_eq!(read_prefix(&path, /*max_bytes*/ 64).await, (body, false));
}
#[tokio::test]
async fn stops_at_the_budget_and_reports_more() {
let dir = tempfile::TempDir::new().unwrap();
let path = dir.path().join("huge.log");
tokio::fs::write(&path, "X".repeat(200_000)).await.unwrap();
assert_eq!(
read_prefix(&path, /*max_bytes*/ 500).await,
("X".repeat(500), true)
);
}
/// Large enough to take several reads: a short read must be consumed, not
/// mistaken for the end of the file.
#[tokio::test]
async fn reads_a_log_within_the_budget_whole() {
let dir = tempfile::TempDir::new().unwrap();
let path = dir.path().join("chunked.log");
let body = "a".repeat(300_000);
tokio::fs::write(&path, &body).await.unwrap();
assert_eq!(read_prefix(&path, MAX_SNAPSHOT_BYTES).await, (body, false));
}
#[tokio::test]
async fn drops_a_character_split_by_the_budget() {
let dir = tempfile::TempDir::new().unwrap();
let path = dir.path().join("cjk.log");
tokio::fs::write(&path, "日".repeat(10)).await.unwrap();
assert_eq!(
read_prefix(&path, /*max_bytes*/ 8).await,
("日日".to_string(), true)
);
}
/// The file fits the budget but ends mid character, so the dropped bytes
/// must not read as the whole log.
#[tokio::test]
async fn a_log_ending_mid_character_reads_as_incomplete() {
let dir = tempfile::TempDir::new().unwrap();
let path = dir.path().join("torn.log");
tokio::fs::write(&path, &"日".as_bytes()[..2])
.await
.unwrap();
assert_eq!(
read_prefix(&path, /*max_bytes*/ 100).await,
(String::new(), true)
);
}
#[tokio::test]
async fn a_missing_log_reads_as_incomplete() {
assert_eq!(
read_prefix(Path::new("/nonexistent/task.log"), /*max_bytes*/ 100).await,
(String::new(), true)
);
}

12
crates/codegen/xai-grok-tools/src/computer/types.rs
View file

@ -190,7 +190,13 @@ pub struct TaskSnapshot {
pub end_time: Option<std::time::SystemTime>,
pub output: String,
pub output_file: PathBuf,
/// `output` may not be the whole output: read `output_file` for the rest.
/// Says the copy is partial, not how it came to be.
pub truncated: bool,
/// Total bytes the task has written, when the source tracks it. `output`
/// may hold only part of that; zero means unknown.
#[serde(default)]
pub output_total_bytes: usize,
pub exit_code: Option<i32>,
pub signal: Option<String>,
pub completed: bool,
@ -242,6 +248,12 @@ impl TaskSnapshot {
!self.completed
}
/// The output on hand, and the size of the output it came from. Readers
/// take the size from here so the "not tracked" case is handled once.
pub fn output_view(&self) -> crate::util::truncate::PartialOutput<'_> {
crate::util::truncate::PartialOutput::part_of(&self.output, self.output_total_bytes)
}
/// Incomplete and backgrounded — tray/`tasks_snapshot` predicate (not FG in-flight).
pub fn is_outstanding_background(&self) -> bool {
!self.completed && self.is_backgrounded

1
crates/codegen/xai-grok-tools/src/implementations/grok_build/task_output/mod.rs
View file

@ -1045,6 +1045,7 @@ pub(crate) mod test_helpers {
owner_session_id: None,
description: None,
is_backgrounded: false,
output_total_bytes: 0,
}
}

22
crates/codegen/xai-grok-tools/src/implementations/task_output/tool.rs
View file

@ -14,10 +14,13 @@ pub(crate) fn snapshot_to_result(
read_file_tool_name: &str,
max_output_bytes: usize,
) -> TaskOutputResult {
let output_view = s.output_view();
let raw_output_bytes = output_view.total_bytes();
// Truncate output to protect model's context window
let (output, truncated) = if s.output.len() > max_output_bytes {
truncate_with_preview(
&s.output,
output_view,
max_output_bytes,
PREVIEW_SIZE,
Some(&format!(
@ -43,9 +46,7 @@ pub(crate) fn snapshot_to_result(
);
// Compute duration before moving fields.
// Capture raw byte count before moving `s.output`.
let duration_secs = s.duration_secs();
let raw_output_bytes = s.output.len();
TaskOutputResult {
task_id: s.task_id,
@ -113,9 +114,24 @@ mod tests {
owner_session_id: None,
description: None,
is_backgrounded: false,
output_total_bytes: 0,
}
}
/// A snapshot that holds only part of a large log must still report the
/// task's real size, which polling uses to tell progress from a stall.
#[test]
fn raw_output_bytes_reports_the_task_total_not_the_part_held() {
let mut snapshot = make_test_snapshot("test-1", true, Some(0));
snapshot.output = "x".repeat(1024);
snapshot.output_total_bytes = 5 * 1024 * 1024;
snapshot.truncated = true;
let result = snapshot_to_result(snapshot, "read_file", DEFAULT_TOOL_OUTPUT_BYTES);
assert_eq!(result.raw_output_bytes, 5 * 1024 * 1024);
}
#[test]
fn test_snapshot_to_result_running() {
let snapshot = make_test_snapshot("test-1", false, None);

39
crates/codegen/xai-grok-tools/src/reminders/task_completion.rs
View file

@ -20,7 +20,7 @@ use crate::types::TaskSnapshot;
use crate::types::output::ToolOutput;
use crate::types::resources::{SharedResources, State, Terminal};
use crate::types::tool::{Reminder, ToolKind};
use crate::util::truncate::{PREVIEW_SIZE, truncate_with_preview};
use crate::util::truncate::{PREVIEW_SIZE, PartialOutput, truncate_with_preview};
use std::collections::{HashMap, HashSet};
use std::sync::Arc;
use xai_tool_types::KillTaskOutput;
@ -151,7 +151,7 @@ pub fn format_bash_completion(
render_completion_output_delivery(
&mut msg,
&task.task_id,
&task.output,
task.output_view(),
task_output_name,
disk_pointer_footer.as_deref(),
);
@ -357,7 +357,7 @@ pub(crate) fn task_owned_by_session(task: &TaskSnapshot, my_owner: Option<&str>)
pub fn render_completion_output_delivery(
buf: &mut String,
subagent_id: &str,
output: &str,
output: PartialOutput<'_>,
task_output_name: Option<&str>,
disk_pointer_footer: Option<&str>,
) {
@ -377,7 +377,7 @@ pub fn render_completion_output_delivery(
let _ = write!(buf, "response:\n{output}");
}
None => {
let _ = write!(buf, "response:\n{output}");
let _ = write!(buf, "response:\n{}", output.text());
}
},
}
@ -430,7 +430,13 @@ pub fn format_subagent_completion(
Some(_) => "\n",
None => "\n\n",
});
render_completion_output_delivery(&mut out, &c.subagent_id, &c.output, task_output_name, None);
render_completion_output_delivery(
&mut out,
&c.subagent_id,
PartialOutput::whole(&c.output),
task_output_name,
None,
);
out
}
/// Format buffered between-turn subagent completions into a system-reminder
@ -463,7 +469,7 @@ pub fn format_between_turn_completions(
render_completion_output_delivery(
&mut buf,
&c.subagent_id,
&c.output,
PartialOutput::whole(&c.output),
task_output_name,
None,
);
@ -808,6 +814,7 @@ mod tests {
owner_session_id: None,
description: None,
is_backgrounded: false,
output_total_bytes: 0,
};
let msg = format_bash_completion(&task, Some("get_command_or_subagent_output"), None);
assert!(msg.contains("abc-123"));
@ -836,6 +843,7 @@ mod tests {
owner_session_id: None,
description: None,
is_backgrounded: false,
output_total_bytes: 0,
};
let msg = format_monitor_completion(&task, Some("get_command_or_subagent_output"));
assert!(
@ -870,6 +878,7 @@ mod tests {
owner_session_id: None,
description: None,
is_backgrounded: false,
output_total_bytes: 0,
};
let msg = format_monitor_completion(&task, None);
assert!(
@ -899,6 +908,7 @@ mod tests {
owner_session_id: None,
description: None,
is_backgrounded: false,
output_total_bytes: 0,
};
let msg = format_bash_completion(&task, Some("get_command_or_subagent_output"), None);
assert!(msg.contains("cargo test"));
@ -925,6 +935,7 @@ mod tests {
owner_session_id: None,
description: None,
is_backgrounded: false,
output_total_bytes: 0,
};
let msg = format_bash_completion(&task, Some("get_command_or_subagent_output"), None);
assert!(msg.contains("exit code: unknown"));
@ -954,6 +965,7 @@ mod tests {
owner_session_id: None,
description: None,
is_backgrounded: false,
output_total_bytes: 0,
};
let msg = format_bash_completion(&task, Some("get_command_or_subagent_output"), None);
assert!(
@ -994,6 +1006,7 @@ mod tests {
owner_session_id: None,
description: None,
is_backgrounded: false,
output_total_bytes: 0,
};
let msg = format_bash_completion(&task, Some("get_command_or_subagent_output"), None);
assert!(
@ -1033,6 +1046,7 @@ mod tests {
owner_session_id: None,
description: None,
is_backgrounded: false,
output_total_bytes: 0,
};
let msg = format_bash_completion(&task, Some("get_command_or_subagent_output"), None);
assert!(msg.contains("exit code: 0"));
@ -1195,8 +1209,20 @@ mod tests {
owner_session_id: None,
description: None,
is_backgrounded: false,
output_total_bytes: 0,
}
}
/// The snapshot holds part of a large log. The footer the model reads must
/// state the task's real size, not the size of the part on hand.
#[test]
fn bash_completion_footer_states_the_real_log_size() {
let mut task = make_completed("bg-large");
task.output = "x".repeat(20_000);
task.output_total_bytes = 5_000_000;
task.output_file = std::path::PathBuf::from("/tmp/bg-large.log");
let msg = format_bash_completion(&task, None, Some("read_file"));
assert!(msg.contains("5000000 bytes total"), "{msg}");
}
fn make_running(id: &str) -> TaskSnapshot {
TaskSnapshot {
task_id: id.into(),
@ -1217,6 +1243,7 @@ mod tests {
owner_session_id: None,
description: None,
is_backgrounded: false,
output_total_bytes: 0,
}
}
fn make_bg_started(id: &str) -> crate::types::output::BackgroundTaskStarted {

79
crates/codegen/xai-grok-tools/src/util/truncate.rs
View file

@ -92,25 +92,61 @@ pub fn truncate_str(s: &str, max_bytes: usize) -> &str {
&s[..end]
}
/// Text on hand, and the size of the output it came from. The two differ when
/// the caller holds only part of a larger output and the reader still needs
/// the real size.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct PartialOutput<'a> {
text: &'a str,
total_bytes: usize,
}
impl<'a> PartialOutput<'a> {
pub fn whole(text: &'a str) -> Self {
Self {
text,
total_bytes: text.len(),
}
}
/// Part of an output of `total_bytes`.
pub fn part_of(text: &'a str, total_bytes: usize) -> Self {
Self {
text,
total_bytes: total_bytes.max(text.len()),
}
}
pub fn text(&self) -> &'a str {
self.text
}
pub fn total_bytes(&self) -> usize {
self.total_bytes
}
}
/// Truncate output to a UTF-8-safe preview plus a model-visible footer.
///
/// The cap decides whether truncation happens. When triggered, the returned
/// value contains the first `preview_bytes` bytes snapped to a char boundary
/// followed by `[Output truncated - <N> bytes total...]`.
/// followed by `[Output truncated - <N> bytes total...]`, where `N` is the
/// size of the whole output, not of the part on hand.
pub fn truncate_with_preview(
output: &str,
output: PartialOutput<'_>,
max_bytes: usize,
preview_bytes: usize,
footer_hint: Option<&str>,
) -> (String, bool) {
if output.len() <= max_bytes {
return (output.to_string(), false);
let PartialOutput { text, total_bytes } = output;
if text.len() <= max_bytes {
return (text.to_string(), false);
}
let preview = truncate_str(output, preview_bytes.min(output.len()));
let preview = truncate_str(text, preview_bytes.min(text.len()));
let footer = match footer_hint {
Some(hint) => format!("[Output truncated - {} bytes total. {hint}]", output.len()),
None => format!("[Output truncated - {} bytes total]", output.len()),
Some(hint) => format!("[Output truncated - {total_bytes} bytes total. {hint}]"),
None => format!("[Output truncated - {total_bytes} bytes total]"),
};
(format!("{preview}\n\n{footer}"), true)
}
@ -226,6 +262,9 @@ pub fn soft_wrap_lines(text: &str, wrap_width: usize) -> String {
result
}
/// Separator between the retained head and tail of a truncated output.
pub(crate) const FRONT_BACK_TRUNCATION_MARKER: &str = "\n\n... (output truncated) ...\n\n";
/// Truncate a string keeping the first half and last half of the character
/// budget, inserting a separator in the middle.
///
@ -252,7 +291,7 @@ pub fn truncate_front_and_back(s: &str, max_chars: usize) -> (String, bool) {
.unwrap_or(0)
}
};
let ellipsis = "\n\n... (output truncated) ...\n\n";
let ellipsis = FRONT_BACK_TRUNCATION_MARKER;
let mut result = String::with_capacity(front_end + ellipsis.len() + (s.len() - back_start));
result.push_str(&s[..front_end]);
result.push_str(ellipsis);
@ -505,7 +544,7 @@ mod tests {
#[test]
fn truncate_with_preview_short_output_unchanged() {
let (result, truncated) = truncate_with_preview("hello", 10, 5, None);
let (result, truncated) = truncate_with_preview(PartialOutput::whole("hello"), 10, 5, None);
assert_eq!(result, "hello");
assert!(!truncated);
}
@ -513,7 +552,8 @@ mod tests {
#[test]
fn truncate_with_preview_caps_large_output() {
let output = "x".repeat(5_000_000);
let (result, truncated) = truncate_with_preview(&output, 4_000, 2_000, None);
let (result, truncated) =
truncate_with_preview(PartialOutput::whole(&output), 4_000, 2_000, None);
assert!(truncated);
assert!(result.len() < 2_200, "result was {} bytes", result.len());
@ -524,7 +564,8 @@ mod tests {
#[test]
fn truncate_with_preview_utf8_boundary() {
let output = "😀".repeat(1_500);
let (result, truncated) = truncate_with_preview(&output, 4_000, 2_001, None);
let (result, truncated) =
truncate_with_preview(PartialOutput::whole(&output), 4_000, 2_001, None);
assert!(truncated);
assert!(result.starts_with(&"😀".repeat(500)));
@ -535,7 +576,7 @@ mod tests {
fn truncate_with_preview_with_footer_hint() {
let output = "x".repeat(10_000);
let (result, truncated) = truncate_with_preview(
&output,
PartialOutput::whole(&output),
4_000,
2_000,
Some("Use read_file for full content"),
@ -545,10 +586,22 @@ mod tests {
assert!(result.contains("Use read_file for full content"));
}
/// A caller holding part of a larger output states the real size.
#[test]
fn truncate_with_preview_reports_the_size_it_is_given() {
let held = "x".repeat(10_000);
let (result, truncated) =
truncate_with_preview(PartialOutput::part_of(&held, 5_000_000), 4_000, 2_000, None);
assert!(truncated);
assert!(result.contains("5000000 bytes total"), "{result}");
}
#[test]
fn truncate_with_preview_without_footer_hint() {
let output = "x".repeat(10_000);
let (result, truncated) = truncate_with_preview(&output, 4_000, 2_000, None);
let (result, truncated) =
truncate_with_preview(PartialOutput::whole(&output), 4_000, 2_000, None);
assert!(truncated);
assert!(result.contains("[Output truncated - 10000 bytes total]"));

60
crates/codegen/xai-grok-tools/tests/test_subagent_soak.rs
View file

@ -1,5 +1,5 @@
//! Subagent lifecycle soak: churn spawn/run/completion/eviction and assert
//! threads, fds, and heap/RSS reach steady state. A stub `ChildRunner` drives
//! threads, open files, and heap/RSS reach steady state. A stub `ChildRunner` drives
//! the real coordinator/transport.
//!
//! SUBAGENT_SOAK_CYCLES=20000 cargo test -p xai-grok-tools \
@ -45,7 +45,7 @@ impl Metric {
match self {
Metric::Rss => "rss",
Metric::Threads => "threads",
Metric::Fds => "fds",
Metric::Fds => "open_files",
}
}
@ -54,7 +54,7 @@ impl Metric {
match self {
Metric::Rss => "rss_bytes",
Metric::Threads => "threads",
Metric::Fds => "fds",
Metric::Fds => "open_files",
}
}
@ -69,7 +69,7 @@ impl Metric {
match self {
Metric::Rss => bounds.max_rss_growth_mib as f64,
Metric::Threads => bounds.max_thread_growth as f64,
Metric::Fds => bounds.max_fd_growth as f64,
Metric::Fds => bounds.max_open_files_growth as f64,
}
}
@ -81,7 +81,7 @@ impl Metric {
}
}
/// RSS is sampled on every unix; thread and fd counts are Linux-only.
/// RSS is sampled on every unix; thread and open-file counts are Linux-only.
fn expected_on_this_platform(self) -> bool {
match self {
Metric::Rss => true,
@ -100,22 +100,30 @@ impl MetricValue for ResourceSnapshot {
fn value_of(&self, metric: Metric) -> Option<usize> {
// Destructure so a new resource field is a compile error here, not a
// silently dropped metric.
let ResourceSnapshot { rss, threads, fds } = *self;
let ResourceSnapshot {
rss,
threads,
open_files,
} = *self;
match metric {
Metric::Rss => rss,
Metric::Threads => threads,
Metric::Fds => fds,
Metric::Fds => open_files,
}
}
}
impl MetricValue for ResourceGrowth {
fn value_of(&self, metric: Metric) -> Option<usize> {
let ResourceGrowth { rss, threads, fds } = *self;
let ResourceGrowth {
rss,
threads,
open_files,
} = *self;
match metric {
Metric::Rss => rss,
Metric::Threads => threads,
Metric::Fds => fds,
Metric::Fds => open_files,
}
}
}
@ -172,7 +180,7 @@ struct Bounds {
measure: u64,
concurrency: u64,
max_thread_growth: u64,
max_fd_growth: u64,
max_open_files_growth: u64,
max_rss_growth_mib: u64,
max_blocks_per_cycle: f64,
max_bytes_per_cycle: f64,
@ -187,9 +195,9 @@ impl Bounds {
warmup: env_parse("SUBAGENT_SOAK_WARMUP", MAX_COMPLETED_ENTRIES as u64),
measure: env_parse("SUBAGENT_SOAK_CYCLES", 512u64),
concurrency: env_parse("SUBAGENT_SOAK_CONCURRENCY", 16u64),
// RSS is looser than threads and fds to absorb allocator noise.
// RSS is looser than threads and open files to absorb allocator noise.
max_thread_growth: env_parse("SUBAGENT_SOAK_MAX_THREAD_GROWTH", 8u64),
max_fd_growth: env_parse("SUBAGENT_SOAK_MAX_FD_GROWTH", 16u64),
max_open_files_growth: env_parse("SUBAGENT_SOAK_MAX_OPEN_FILES_GROWTH", 16u64),
max_rss_growth_mib: env_parse("SUBAGENT_SOAK_MAX_RSS_GROWTH_MIB", 256u64),
max_blocks_per_cycle: env_parse("SUBAGENT_SOAK_MAX_BLOCKS_PER_CYCLE", 2.0f64),
max_bytes_per_cycle: env_parse("SUBAGENT_SOAK_MAX_BYTES_PER_CYCLE", 4096.0f64),
@ -595,7 +603,7 @@ fn assert_bounds(bounds: &Bounds, m: &Measurement) {
/// Keep this the only test in the binary that creates a `dhat::Profiler`.
#[tokio::test(flavor = "current_thread")]
#[ignore = "subagent soak; run with --ignored (SUBAGENT_SOAK_CYCLES bounds the measured window)"]
async fn subagent_lifecycle_soak_bounds_threads_fds_and_heap() {
async fn subagent_lifecycle_soak_bounds_threads_open_files_and_heap() {
#[cfg(feature = "dhat-heap")]
let _profiler = dhat::Profiler::builder().testing().build();
@ -646,7 +654,7 @@ mod tests {
let snapshot = ResourceSnapshot {
rss: Some(11),
threads: Some(22),
fds: Some(33),
open_files: Some(33),
};
assert_eq!(snapshot.value_of(Metric::Rss), Some(11));
assert_eq!(snapshot.value_of(Metric::Threads), Some(22));
@ -655,7 +663,7 @@ mod tests {
let growth = ResourceGrowth {
rss: Some(1),
threads: None,
fds: Some(3),
open_files: Some(3),
};
assert_eq!(growth.value_of(Metric::Rss), Some(1));
assert_eq!(growth.value_of(Metric::Threads), None);
@ -669,10 +677,10 @@ mod tests {
let snapshot = ResourceSnapshot {
rss: Some(1),
threads: None,
fds: Some(3),
open_files: Some(3),
};
let json = serde_json::to_string(&Wrap(snapshot)).expect("snapshot serializes");
assert_eq!(json, r#"{"rss_bytes":1,"threads":null,"fds":3}"#);
assert_eq!(json, r#"{"rss_bytes":1,"threads":null,"open_files":3}"#);
}
#[test]
@ -696,7 +704,7 @@ mod tests {
measure: 0,
concurrency: 0,
max_thread_growth: 3,
max_fd_growth: 5,
max_open_files_growth: 5,
max_rss_growth_mib: 7,
max_blocks_per_cycle: 1.0,
max_bytes_per_cycle: 2.0,
@ -729,7 +737,7 @@ mod tests {
measure: 4,
concurrency: 4,
max_thread_growth: 100,
max_fd_growth: 100,
max_open_files_growth: 100,
max_rss_growth_mib: 100,
max_blocks_per_cycle: 10.0,
max_bytes_per_cycle: 10_000.0,
@ -741,7 +749,7 @@ mod tests {
ResourceGrowth {
rss: Some(0),
threads: Some(0),
fds: Some(0),
open_files: Some(0),
}
}
@ -771,7 +779,7 @@ mod tests {
let growth = ResourceGrowth {
rss: None,
threads: Some(0),
fds: Some(0),
open_files: Some(0),
};
let failures = check_bounds(&generous_bounds(), &drained(growth, None));
assert!(
@ -814,7 +822,7 @@ mod tests {
let growth = ResourceGrowth {
rss: Some(200 * 1024 * 1024),
threads: Some(0),
fds: Some(0),
open_files: Some(0),
};
let failures = check_bounds(&generous_bounds(), &drained(growth, None));
assert!(
@ -828,7 +836,7 @@ mod tests {
let growth = ResourceGrowth {
rss: Some(100 * 1024 * 1024),
threads: Some(100),
fds: Some(100),
open_files: Some(100),
};
assert!(check_bounds(&generous_bounds(), &drained(growth, None)).is_empty());
}
@ -885,11 +893,11 @@ mod tests {
}
#[test]
fn check_bounds_flags_thread_and_fd_over_budget() {
fn check_bounds_flags_thread_and_open_files_over_budget() {
let growth = ResourceGrowth {
rss: Some(0),
threads: Some(200),
fds: Some(200),
open_files: Some(200),
};
let failures = check_bounds(&generous_bounds(), &drained(growth, None));
assert!(
@ -897,7 +905,7 @@ mod tests {
"{failures:?}"
);
assert!(
failures.iter().any(|f| f.starts_with("fds:")),
failures.iter().any(|f| f.starts_with("open_files:")),
"{failures:?}"
);
}