sandcastle: cap pr-fixer maxIterations and escalate to human on exhaustion #41

New issue

Closed

opened 2026-07-05 20:03:38 +08:00 by weiwen · 0 comments

weiwen commented

2026-07-05 20:03:38 +08:00

Owner

Priority: highest.

Problem

The pr-fixer step (.sandcastle/pr-fixer-prompt.md, invoked at .sandcastle/main.mts:188) runs with maxIterations: 100 on sonnet. A CI fix that needs 100 turns is a runaway, not a fix.

Token accounting (see .sandcastle/token-usage.sh) shows a single pr-fixer run (issue #23's CI rework) consumed 5.26M billed input tokens — 8% of all sandcastle spend across the entire project, from one invocation. The bloat is structural: pr-fixer re-runs the full nix develop .#ci -c just check gate and the large build/test log lands in context, then gets re-read every turn. Over up to 100 turns that compounds badly.

Change

In .sandcastle/main.mts:190, lower maxIterations from 100 to ~25, and make the prompt escalate to a human on exhaustion instead of silently churning.

Add to .sandcastle/pr-fixer-prompt.md: if the check gate is still red after exhausting iterations, post a PR comment summarising what was tried and what is still failing, and stop — do not keep churning. (The PR stays ready-for-human; a human takes over.)

Scope

Scoped to the iteration cap + escalation only. Deliberately NOT in scope (premature until we see the fixer still bloating after the cap):

running only the failing check instead of the whole just check gate on the verify step
truncating/limiting CI log ingestion

Acceptance

pr-fixer caps at ~25 iterations.
On exhaustion it posts a summary comment and stops rather than continuing.

**Priority: highest.** ## Problem The `pr-fixer` step (`.sandcastle/pr-fixer-prompt.md`, invoked at `.sandcastle/main.mts:188`) runs with `maxIterations: 100` on sonnet. A CI fix that needs 100 turns is a runaway, not a fix. Token accounting (see `.sandcastle/token-usage.sh`) shows a **single** pr-fixer run (issue #23's CI rework) consumed **5.26M billed input tokens — 8% of all sandcastle spend across the entire project, from one invocation.** The bloat is structural: pr-fixer re-runs the full `nix develop .#ci -c just check` gate and the large build/test log lands in context, then gets re-read every turn. Over up to 100 turns that compounds badly. ## Change In `.sandcastle/main.mts:190`, lower `maxIterations` from `100` to `~25`, and make the prompt escalate to a human on exhaustion instead of silently churning. Add to `.sandcastle/pr-fixer-prompt.md`: if the check gate is still red after exhausting iterations, post a PR comment summarising what was tried and what is still failing, and stop — do not keep churning. (The PR stays `ready-for-human`; a human takes over.) ## Scope Scoped to the iteration cap + escalation **only**. Deliberately NOT in scope (premature until we see the fixer still bloating after the cap): - running only the failing check instead of the whole `just check` gate on the verify step - truncating/limiting CI log ingestion ## Acceptance - pr-fixer caps at ~25 iterations. - On exhaustion it posts a summary comment and stops rather than continuing.

weiwen added the

ready-for-agent

label

2026-07-05 20:03:38 +08:00

weiwen referenced this issue from a commit

2026-07-06 01:15:31 +08:00

chore(sandcastle): cap pr-fixer at 25 iterations, escalate to human on exhaustion

weiwen closed this issue

2026-07-06 01:15:31 +08:00