sandcastle: run pr-opener on haiku instead of sonnet #40

New issue

Closed

opened 2026-07-05 20:03:38 +08:00 by weiwen · 0 comments

weiwen commented

2026-07-05 20:03:38 +08:00

Owner

Priority: highest.

Problem

The pr-opener step (.sandcastle/pr-prompt.md, invoked at .sandcastle/main.mts:230) runs on claude-sonnet-4-6 but does almost no reasoning: STEP 0–4 are a fixed sequence of git/tea commands, and the only genuinely LLM-shaped subtask is writing the PR title/body from the diff.

Token accounting (reconstructed from the Claude Code session store — sandcastle does not record usage itself, see .sandcastle/token-usage.sh) shows pr-opener burning ~370K billed input tokens/run across 8 runs (~2.96M total, 4.5% of all sandcastle spend) for what is essentially plumbing.

Change

In .sandcastle/main.mts:233, change the pr-opener agent from sandcastle.claudeCode("claude-sonnet-4-6") to sandcastle.claudeCode("claude-haiku-4-5").

The prompt already loads minimal context (git log --oneline main..HEAD + git diff main...HEAD --stat, not the full diff), so no prompt change is needed. Haiku is more than capable of the title/body prose + fixed command sequence.

Risk to watch (do not build up front)

The correctness-sensitive part is STEP 4's label reconciliation — "never relabel the issue before the PR exists" — because getting the ordering wrong silently drops the issue out of the planner. Haiku is likelier than sonnet to fumble a conditional ordering. It is guarded (STEP 0 idempotency check, STEP 1 push-fail-stop), so blast radius is small.

Follow-up only if haiku actually flakes there: script the final tea issue edit ... --add-labels in-review --remove-labels ready-for-agent call directly in main.mts instead of leaving it to the agent. Do not do this pre-emptively.

Acceptance

pr-opener runs on haiku.
A subsequent run opens a PR, labels it ready-for-human, and moves the issue ready-for-agent -> in-review correctly.

**Priority: highest.** ## Problem The `pr-opener` step (`.sandcastle/pr-prompt.md`, invoked at `.sandcastle/main.mts:230`) runs on `claude-sonnet-4-6` but does almost no reasoning: STEP 0–4 are a fixed sequence of `git`/`tea` commands, and the only genuinely LLM-shaped subtask is writing the PR title/body from the diff. Token accounting (reconstructed from the Claude Code session store — sandcastle does not record usage itself, see `.sandcastle/token-usage.sh`) shows pr-opener burning ~370K billed input tokens/run across 8 runs (~2.96M total, 4.5% of all sandcastle spend) for what is essentially plumbing. ## Change In `.sandcastle/main.mts:233`, change the pr-opener agent from `sandcastle.claudeCode("claude-sonnet-4-6")` to `sandcastle.claudeCode("claude-haiku-4-5")`. The prompt already loads minimal context (`git log --oneline main..HEAD` + `git diff main...HEAD --stat`, not the full diff), so **no prompt change is needed**. Haiku is more than capable of the title/body prose + fixed command sequence. ## Risk to watch (do not build up front) The correctness-sensitive part is STEP 4's label reconciliation — *"never relabel the issue before the PR exists"* — because getting the ordering wrong silently drops the issue out of the planner. Haiku is likelier than sonnet to fumble a conditional ordering. It is guarded (STEP 0 idempotency check, STEP 1 push-fail-stop), so blast radius is small. **Follow-up only if haiku actually flakes there:** script the final `tea issue edit ... --add-labels in-review --remove-labels ready-for-agent` call directly in `main.mts` instead of leaving it to the agent. Do not do this pre-emptively. ## Acceptance - pr-opener runs on haiku. - A subsequent run opens a PR, labels it `ready-for-human`, and moves the issue `ready-for-agent` -> `in-review` correctly.

weiwen added the

ready-for-agent

label

2026-07-05 20:03:38 +08:00

weiwen referenced this issue from a commit

2026-07-06 01:25:51 +08:00

chore(sandcastle): run pr-opener on haiku instead of sonnet — closes #40

weiwen closed this issue

2026-07-06 01:25:51 +08:00