weiwen/sandcastle
1
0
Fork 0
Code Issues 3 Pull requests Projects Releases Packages Wiki Activity Actions

Extract sandcastle core into shared flake; wire nibble as first consumer #1

New issue
Open
opened 2026-07-08 02:49:04 +08:00 by weiwen · 0 comments
weiwen commented 2026-07-08 02:49:04 +08:00
Owner
Copy link

Problem Statement

The sandcastle autonomous-agent PR loop (planner → implementer → reviewer → pr-fixer) currently lives entirely inside the evie repo: its Nix image builder, configure-tea helper, orchestrator (.sandcastle/main.mts), and the four agent prompts are all evie-specific. Every prompt hardcodes --repo weiwen/evie and the forge.weiwen.dev/weiwen/evie.git push URL; the image tag, models, and iteration/concurrency knobs are baked into main.mts. As a result there is no way to run the same loop against a second repository without copy-pasting and hand-editing the whole .sandcastle/ tree and the flake — which then drifts. The maintainer wants to run the loop against nibble (and later others) from a single, shared source of truth.

Solution

Extract a reusable sandcastle core into a new standalone repo (weiwen/sandcastle) that both current and future projects consume as a Nix flake input. The core owns everything generic: the sandbox-image builder, the baked helper scripts (configure-tea, fmt, check), the base package set (including claude-code), the orchestrator, and the four genericized prompts. Each consuming repo contributes only what is genuinely project-specific: a small .sandcastle/config.toml, secrets in .sandcastle/.env, a .sandcastle/CODING_STANDARDS.md, the project's own binary/toolchain/native-lib wiring in its flake, and the forge-side triage workflow.

nibble is the first consumer (the guinea pig): its flake is rewritten to the shared shape, wired to the sandcastle input, and its origin remote is aligned to the forge host so tea auto-derives the repo slug. evie is left untouched in this work and migrated to the extracted core later.

From the maintainer's perspective: edit the loop once in weiwen/sandcastle, bump the flake input in each consumer, and every project's loop updates together — while each project keeps full control over its build, its coding standards, and its tuning knobs.

User Stories

  1. As the maintainer, I want the sandcastle image builder, helper scripts, orchestrator, and prompts to live in one shared repo, so that I fix a bug or improve a prompt once and every consumer benefits.
  2. As the maintainer, I want to consume the sandcastle core as a Nix flake input, so that adopting it in a new repo is a few lines of flake.nix plus a small config file.
  3. As the maintainer, I want the shared flake to expose a mkSandcastleImage { name, tag, extraContents, extraEnv, forgeUrl } function, so that each repo supplies only its own toolchain and native libraries and gets a correct sandbox image.
  4. As the maintainer, I want the shared flake to re-export claude-code (via llm-agents), so that consumers don't each have to declare that input.
  5. As the maintainer, I want configure-tea, fmt, and check baked into the shared base package set as shell scripts, so that the prompts can call bare fmt / check and no consumer needs a Justfile.
  6. As the maintainer, I want the orchestrator delivered as a self-contained node app (apps.sandcastle), so that no consumer repo needs its own package.json, node_modules, or main.mts.
  7. As the maintainer, I want the orchestrator to read all project-specific settings from ./.sandcastle/config.toml at runtime, so that the same binary drives any repo without recompilation.
  8. As the maintainer, I want config.toml to carry repo, forgeHost, imageName, maxIterations, maxConcurrency, and per-role models, so that each project tunes cost/behaviour independently.
  9. As the maintainer, I want the orchestrator to error clearly when a required config field is missing, so that a misconfigured repo fails fast instead of running with silent wrong defaults.
  10. As the maintainer, I want the four prompts genericized (no hardcoded repo slug or forge URL), so that a single prompt set drives every consumer.
  11. As the maintainer, I want --repo dropped from all tea invocations so the slug is auto-derived from the sandbox's git remote, so that the prompts carry no project identity.
  12. As the maintainer, I want the push URL built from {{FORGE_HOST}} + {{REPO}} injected from config, so that pushes target the right repo without editing prompts.
  13. As the maintainer, I want the reviewer prompt to @-include .sandcastle/CODING_STANDARDS.md relative to the sandbox cwd, so that each project's standards are applied without changing the shared prompt.
  14. As the maintainer, I want nibble's flake rewritten to the shared shape (flake-utils + rust-overlay + checks), so that it and future consumers look and build the same way.
  15. As the maintainer, I want nibble's sandbox image to include its native build deps (libjxl, libheif, pkg-config) plus the rust toolchain, so that cargo build/cargo test link correctly inside the sandbox.
  16. As the maintainer, I want nibble's existing Playwright dev shell preserved, so that the manual screenshot workflow (scripts/preview.mjs) still works outside the sandbox.
  17. As the maintainer, I want a nix flake check gate that builds the sandcastle image, so that a broken image or toolchain regression is caught before running the loop.
  18. As the maintainer, I want nibble's origin remote moved from the internal SSH host to the forge host (forge.weiwen.dev), so that the sandbox's inherited remote matches the tea login and slug auto-derivation works.
  19. As the maintainer, I want the Forgejo triage workflow ported to nibble, so that a human REQUEST_CHANGES review auto-applies ready-for-agent and the pr-rework lane fires.
  20. As the maintainer, I want the triage label vocabulary (ready-for-agent, in-review, ready-for-human, epic) present on nibble, so that the loop's label lifecycle functions.
  21. As the maintainer, I want nibble's .sandcastle/ to contain only config.toml, .env, and CODING_STANDARDS.md (no JS, no lockfiles), so that the consumer footprint is minimal.
  22. As the maintainer, I want secrets confined to .sandcastle/.env (Anthropic/Forgejo only — no Immich, since tests run offline against cached fixtures), so that the sandbox carries the least credential surface needed.
  23. As the maintainer, I want to run the loop with nix run .#sandcastle, so that invocation is uniform across consumers.
  24. As the maintainer, I want to build and load the image with nix run .#sandcastleImage | docker load, so that the sandbox is reproducible from the flake.
  25. As the maintainer, I want evie left completely untouched by this work, so that the extraction is validated on nibble first and evie migrates on its own schedule.
  26. As a future consumer repo, I want the invariant behaviours (branch format sandcastle/issue-{id}, label lifecycle, the configure-tea onboarding hook) baked into the shared core, so that I inherit correct loop semantics for free.

Implementation Decisions

New repo weiwen/sandcastle (the shared core). Standalone Nix flake plus the orchestrator source and prompts. This is the single source of truth for everything generic.

  • flake.nix — inputs: nixpkgs, flake-utils, rust-overlay, llm-agents (the latter re-exported so consumers need not declare it). Structured with flake-utils.lib.eachDefaultSystem. Outputs:
    • lib.${system}.mkSandcastleImage { name, tag, extraContents ? [], extraEnv ? [], forgeUrl } — wraps dockerTools.fakeNss + dockerTools.streamLayeredImage over the base package set + extraContents; sets PATH, HOME, cert, and locale env plus extraEnv; entrypoint sleep infinity, workdir /home/agent.
    • Base sandcastlePackages — the toolchain-agnostic set (bash, coreutils, curl, findutils, gawk, git, gnugrep, gnused, jq, nix, nodejs, openssl, pkg-config, which, tea, cacert) plus claude-code and the three baked writeShellScriptBin helpers below.
    • configure-tea — baked script parameterized by forgeUrl; writes ~/.config/tea/config.yml from FORGEJO_TOKEN / FORGEJO_USER at runtime.
    • fmt and check — baked scripts with hardcoded Rust commands: check runs cargo fmt --check && cargo clippy --all-targets -- -D warnings && cargo test; fmt runs cargo clippy --fix --allow-dirty --allow-staged -- -D warnings && cargo fmt. (Rust assumption is intentional; a future non-Rust consumer overrides via extraContents PATH precedence.)
    • apps.${system}.sandcastle — the orchestrator built as a self-contained buildNpmPackage node app bundling @ai-hero/sandcastle + zod + the four prompt files; requires a committed package-lock.json and npmDepsHash in this repo.
  • Orchestrator (main.mts) — generic. New responsibility: a pure loadConfig(tomlString) → ResolvedConfig step producing { repo, forgeHost, imageName, maxIterations, maxConcurrency, models: { planner, implementer, reviewer, prFixer } }, applying documented defaults and throwing on missing required fields (repo, forgeHost). All downstream behaviour (sandbox imageName, per-role model selection, and the {{REPO}} / {{FORGE_HOST}} prompt args) consumes this one object. The onSandboxReady hook reduces to configure-tea only (no npm install); copyToWorktree is removed (no node_modules).
  • Prompts (plan, implement, review, pr-fixer) — genericized: every tea call drops --repo (relying on remote-derived slug); the push URL becomes https://${FORGEJO_USER}:${FORGEJO_TOKEN}@{{FORGE_HOST}}/{{REPO}}.git; just fmt / just check become bare fmt / check. The reviewer prompt retains the cwd-relative @.sandcastle/CODING_STANDARDS.md include. The branch format sandcastle/issue-{id} and the label lifecycle remain hardcoded.

nibble (first consumer).

  • flake.nix — rewritten to the shared shape (flake-utils.eachDefaultSystem + rust-overlay, checks.clippy, devShells.default + devShells.ci). Adds the sandcastle input (git+https://forge.weiwen.dev/weiwen/sandcastle.git). Defines packages.sandcastleImage = sandcastle.lib.${system}.mkSandcastleImage { name = "sandcastle"; tag = "nibble"; extraContents = [ rustToolchain pkg-config libjxl libheif ]; forgeUrl = "https://forge.weiwen.dev"; } and re-exposes apps.sandcastle. Preserves the existing Playwright dev shell (PLAYWRIGHT_* env, playwright-test, python3) for scripts/preview.mjs. Adds checks.sandcastleImage so nix flake check builds the image.
  • .sandcastle/ — config.toml (schema below), .env (Anthropic + Forgejo secrets only), CODING_STANDARDS.md (nibble-specific), .gitignore (.env, logs/, worktrees/). No main.mts, package.json, node_modules, or Justfile.
  • .forgejo/workflows/ — the triage workflow ported from evie's reference (label names identical → expected zero content changes).
  • VCS remote — git and jj origin updated from ssh://forgejo@matsutake/weiwen/nibble.git to ssh://forgejo@forge.weiwen.dev/weiwen/nibble.git.
  • Triage labels — ready-for-agent, in-review, ready-for-human, epic created on weiwen/nibble (done as part of setup).

config.toml schema (consumer-supplied):

repo           = "weiwen/nibble"     # push-URL slug; tea auto-derives for its own calls
forgeHost      = "forge.weiwen.dev"
imageName      = "sandcastle:nibble"
maxIterations  = 5
maxConcurrency = 1

[models]
planner     = "claude-opus-4-8"
implementer = "claude-sonnet-4-6"
reviewer    = "claude-opus-4-8"
prFixer     = "claude-sonnet-4-6"

Testing Decisions

Good tests here exercise external behaviour, not implementation detail. The overwhelming majority of this work is declarative (Nix expressions, prompt markdown, forge YAML, a remote change) and is validated by building, not by unit tests. Exactly one seam carries real branching logic, and it is the one seam that gets a unit test.

  • Unit-tested seam — loadConfig (in weiwen/sandcastle). Test the pure function only through its contract: valid TOML → the expected ResolvedConfig object; omitted optional fields → documented defaults applied (maxIterations, maxConcurrency, models.*); a missing required field (repo, forgeHost) → throws. Do not test how it parses internally or reach into private state. This is the single highest seam through which all downstream orchestrator behaviour flows, so testing it covers config resolution, model selection, and prompt-arg derivation at one point. Prior art: evie's existing main.mts had no such seam (values were hardcoded), so this is a new, deliberately-introduced testable boundary; the repo's tsc --noEmit typecheck script runs alongside it as the type gate.
  • Build-gate (not a unit test) — checks.sandcastleImage in nibble's flake. nix flake check builds the image; it fails if the image won't build or the toolchain/native-lib wiring regresses. This gates the consumer wiring without executing the loop. Prior art: evie's checks.clippy / checks.tests flake-check pattern.
  • Manual e2e (out of automated scope) — one loop run. nix run .#sandcastleImage | docker load then nix run .#sandcastle against a real ready-for-agent issue on nibble, validated by hand once. Deliberately not automated: it spawns agents and mutates forge state, so it is too expensive and nondeterministic to gate CI.

Out of Scope

  • Any change to evie. It stays as-is and is migrated to the extracted core in later, separate work.
  • Automated integration/e2e tests around the running loop (agents, sandboxes, forge mutations).
  • Playwright/browser tooling inside the sandbox image — the sole nibble test runs offline against cached fixtures; Playwright remains a dev-shell-only concern for manual screenshots.
  • The token-usage.sh analytics helper — referenced by nothing in the loop; may be revisited as a shared utility later.
  • Parameterizing fmt/check for non-Rust consumers — the Rust commands are intentionally hardcoded now; generalize only when a non-Rust consumer appears.
  • Multi-system support beyond what flake-utils.eachDefaultSystem yields incidentally; the only exercised system is x86_64-linux.

Further Notes

  • The extraction is validated on nibble first precisely because it is lower-stakes than evie; a successful nibble loop is the acceptance signal for the core's genericity.
  • tea's slug auto-derivation depends on the local git remote's host matching a configured tea login host — which is exactly why nibble's origin is realigned to forge.weiwen.dev. The push URL still needs the explicit slug, which is why repo remains in config.toml even though --repo is dropped from tea calls.
  • The configure-tea / fmt / check scripts are baked into the image the same way configure-tea already is in evie — fixed script bodies, runtime env for the only varying parts (forge URL, credentials).
  • Consumer footprint target: adopting the core in a new repo should be the sandcastle flake input + mkSandcastleImage call + a .sandcastle/{config.toml,.env,CODING_STANDARDS.md} triplet + the triage workflow + label vocabulary — nothing more.
## Problem Statement The sandcastle autonomous-agent PR loop (planner → implementer → reviewer → pr-fixer) currently lives entirely inside the `evie` repo: its Nix image builder, `configure-tea` helper, orchestrator (`.sandcastle/main.mts`), and the four agent prompts are all evie-specific. Every prompt hardcodes `--repo weiwen/evie` and the `forge.weiwen.dev/weiwen/evie.git` push URL; the image tag, models, and iteration/concurrency knobs are baked into `main.mts`. As a result there is no way to run the same loop against a second repository without copy-pasting and hand-editing the whole `.sandcastle/` tree and the flake — which then drifts. The maintainer wants to run the loop against `nibble` (and later others) from a single, shared source of truth. ## Solution Extract a reusable **sandcastle core** into a new standalone repo (`weiwen/sandcastle`) that both current and future projects consume as a Nix flake input. The core owns everything generic: the sandbox-image builder, the baked helper scripts (`configure-tea`, `fmt`, `check`), the base package set (including `claude-code`), the orchestrator, and the four genericized prompts. Each consuming repo contributes only what is genuinely project-specific: a small `.sandcastle/config.toml`, secrets in `.sandcastle/.env`, a `.sandcastle/CODING_STANDARDS.md`, the project's own binary/toolchain/native-lib wiring in its flake, and the forge-side triage workflow. `nibble` is the first consumer (the guinea pig): its flake is rewritten to the shared shape, wired to the sandcastle input, and its `origin` remote is aligned to the forge host so `tea` auto-derives the repo slug. `evie` is left untouched in this work and migrated to the extracted core later. From the maintainer's perspective: edit the loop once in `weiwen/sandcastle`, bump the flake input in each consumer, and every project's loop updates together — while each project keeps full control over its build, its coding standards, and its tuning knobs. ## User Stories 1. As the maintainer, I want the sandcastle image builder, helper scripts, orchestrator, and prompts to live in one shared repo, so that I fix a bug or improve a prompt once and every consumer benefits. 2. As the maintainer, I want to consume the sandcastle core as a Nix flake input, so that adopting it in a new repo is a few lines of `flake.nix` plus a small config file. 3. As the maintainer, I want the shared flake to expose a `mkSandcastleImage { name, tag, extraContents, extraEnv, forgeUrl }` function, so that each repo supplies only its own toolchain and native libraries and gets a correct sandbox image. 4. As the maintainer, I want the shared flake to re-export `claude-code` (via `llm-agents`), so that consumers don't each have to declare that input. 5. As the maintainer, I want `configure-tea`, `fmt`, and `check` baked into the shared base package set as shell scripts, so that the prompts can call bare `fmt` / `check` and no consumer needs a `Justfile`. 6. As the maintainer, I want the orchestrator delivered as a self-contained node app (`apps.sandcastle`), so that no consumer repo needs its own `package.json`, `node_modules`, or `main.mts`. 7. As the maintainer, I want the orchestrator to read all project-specific settings from `./.sandcastle/config.toml` at runtime, so that the same binary drives any repo without recompilation. 8. As the maintainer, I want `config.toml` to carry `repo`, `forgeHost`, `imageName`, `maxIterations`, `maxConcurrency`, and per-role `models`, so that each project tunes cost/behaviour independently. 9. As the maintainer, I want the orchestrator to error clearly when a required config field is missing, so that a misconfigured repo fails fast instead of running with silent wrong defaults. 10. As the maintainer, I want the four prompts genericized (no hardcoded repo slug or forge URL), so that a single prompt set drives every consumer. 11. As the maintainer, I want `--repo` dropped from all `tea` invocations so the slug is auto-derived from the sandbox's git remote, so that the prompts carry no project identity. 12. As the maintainer, I want the push URL built from `{{FORGE_HOST}}` + `{{REPO}}` injected from config, so that pushes target the right repo without editing prompts. 13. As the maintainer, I want the reviewer prompt to `@`-include `.sandcastle/CODING_STANDARDS.md` relative to the sandbox cwd, so that each project's standards are applied without changing the shared prompt. 14. As the maintainer, I want `nibble`'s flake rewritten to the shared shape (flake-utils + rust-overlay + `checks`), so that it and future consumers look and build the same way. 15. As the maintainer, I want `nibble`'s sandbox image to include its native build deps (`libjxl`, `libheif`, `pkg-config`) plus the rust toolchain, so that `cargo build`/`cargo test` link correctly inside the sandbox. 16. As the maintainer, I want `nibble`'s existing Playwright dev shell preserved, so that the manual screenshot workflow (`scripts/preview.mjs`) still works outside the sandbox. 17. As the maintainer, I want a `nix flake check` gate that builds the sandcastle image, so that a broken image or toolchain regression is caught before running the loop. 18. As the maintainer, I want `nibble`'s `origin` remote moved from the internal SSH host to the forge host (`forge.weiwen.dev`), so that the sandbox's inherited remote matches the `tea` login and slug auto-derivation works. 19. As the maintainer, I want the Forgejo triage workflow ported to `nibble`, so that a human `REQUEST_CHANGES` review auto-applies `ready-for-agent` and the pr-rework lane fires. 20. As the maintainer, I want the triage label vocabulary (`ready-for-agent`, `in-review`, `ready-for-human`, `epic`) present on `nibble`, so that the loop's label lifecycle functions. 21. As the maintainer, I want `nibble`'s `.sandcastle/` to contain only `config.toml`, `.env`, and `CODING_STANDARDS.md` (no JS, no lockfiles), so that the consumer footprint is minimal. 22. As the maintainer, I want secrets confined to `.sandcastle/.env` (Anthropic/Forgejo only — no Immich, since tests run offline against cached fixtures), so that the sandbox carries the least credential surface needed. 23. As the maintainer, I want to run the loop with `nix run .#sandcastle`, so that invocation is uniform across consumers. 24. As the maintainer, I want to build and load the image with `nix run .#sandcastleImage | docker load`, so that the sandbox is reproducible from the flake. 25. As the maintainer, I want `evie` left completely untouched by this work, so that the extraction is validated on `nibble` first and `evie` migrates on its own schedule. 26. As a future consumer repo, I want the invariant behaviours (branch format `sandcastle/issue-{id}`, label lifecycle, the `configure-tea` onboarding hook) baked into the shared core, so that I inherit correct loop semantics for free. ## Implementation Decisions **New repo `weiwen/sandcastle` (the shared core).** Standalone Nix flake plus the orchestrator source and prompts. This is the single source of truth for everything generic. - **`flake.nix`** — inputs: `nixpkgs`, `flake-utils`, `rust-overlay`, `llm-agents` (the latter re-exported so consumers need not declare it). Structured with `flake-utils.lib.eachDefaultSystem`. Outputs: - `lib.${system}.mkSandcastleImage { name, tag, extraContents ? [], extraEnv ? [], forgeUrl }` — wraps `dockerTools.fakeNss` + `dockerTools.streamLayeredImage` over the base package set + `extraContents`; sets `PATH`, `HOME`, cert, and locale env plus `extraEnv`; entrypoint `sleep infinity`, workdir `/home/agent`. - Base `sandcastlePackages` — the toolchain-agnostic set (bash, coreutils, curl, findutils, gawk, git, gnugrep, gnused, jq, nix, nodejs, openssl, pkg-config, which, `tea`, `cacert`) plus `claude-code` and the three baked `writeShellScriptBin` helpers below. - `configure-tea` — baked script parameterized by `forgeUrl`; writes `~/.config/tea/config.yml` from `FORGEJO_TOKEN` / `FORGEJO_USER` at runtime. - `fmt` and `check` — baked scripts with hardcoded Rust commands: `check` runs `cargo fmt --check && cargo clippy --all-targets -- -D warnings && cargo test`; `fmt` runs `cargo clippy --fix --allow-dirty --allow-staged -- -D warnings && cargo fmt`. (Rust assumption is intentional; a future non-Rust consumer overrides via `extraContents` PATH precedence.) - `apps.${system}.sandcastle` — the orchestrator built as a self-contained `buildNpmPackage` node app bundling `@ai-hero/sandcastle` + `zod` + the four prompt files; requires a committed `package-lock.json` and `npmDepsHash` in this repo. - **Orchestrator (`main.mts`)** — generic. New responsibility: a pure `loadConfig(tomlString) → ResolvedConfig` step producing `{ repo, forgeHost, imageName, maxIterations, maxConcurrency, models: { planner, implementer, reviewer, prFixer } }`, applying documented defaults and throwing on missing required fields (`repo`, `forgeHost`). All downstream behaviour (sandbox `imageName`, per-role model selection, and the `{{REPO}}` / `{{FORGE_HOST}}` prompt args) consumes this one object. The `onSandboxReady` hook reduces to `configure-tea` only (no `npm install`); `copyToWorktree` is removed (no `node_modules`). - **Prompts (plan, implement, review, pr-fixer)** — genericized: every `tea` call drops `--repo` (relying on remote-derived slug); the push URL becomes `https://${FORGEJO_USER}:${FORGEJO_TOKEN}@{{FORGE_HOST}}/{{REPO}}.git`; `just fmt` / `just check` become bare `fmt` / `check`. The reviewer prompt retains the cwd-relative `@.sandcastle/CODING_STANDARDS.md` include. The branch format `sandcastle/issue-{id}` and the label lifecycle remain hardcoded. **`nibble` (first consumer).** - **`flake.nix`** — rewritten to the shared shape (`flake-utils.eachDefaultSystem` + `rust-overlay`, `checks.clippy`, `devShells.default` + `devShells.ci`). Adds the `sandcastle` input (`git+https://forge.weiwen.dev/weiwen/sandcastle.git`). Defines `packages.sandcastleImage = sandcastle.lib.${system}.mkSandcastleImage { name = "sandcastle"; tag = "nibble"; extraContents = [ rustToolchain pkg-config libjxl libheif ]; forgeUrl = "https://forge.weiwen.dev"; }` and re-exposes `apps.sandcastle`. Preserves the existing Playwright dev shell (`PLAYWRIGHT_*` env, `playwright-test`, `python3`) for `scripts/preview.mjs`. Adds `checks.sandcastleImage` so `nix flake check` builds the image. - **`.sandcastle/`** — `config.toml` (schema below), `.env` (Anthropic + Forgejo secrets only), `CODING_STANDARDS.md` (nibble-specific), `.gitignore` (`.env`, `logs/`, `worktrees/`). No `main.mts`, `package.json`, `node_modules`, or `Justfile`. - **`.forgejo/workflows/`** — the triage workflow ported from evie's reference (label names identical → expected zero content changes). - **VCS remote** — git and jj `origin` updated from `ssh://forgejo@matsutake/weiwen/nibble.git` to `ssh://forgejo@forge.weiwen.dev/weiwen/nibble.git`. - **Triage labels** — `ready-for-agent`, `in-review`, `ready-for-human`, `epic` created on `weiwen/nibble` (done as part of setup). **`config.toml` schema (consumer-supplied):** ```toml repo = "weiwen/nibble" # push-URL slug; tea auto-derives for its own calls forgeHost = "forge.weiwen.dev" imageName = "sandcastle:nibble" maxIterations = 5 maxConcurrency = 1 [models] planner = "claude-opus-4-8" implementer = "claude-sonnet-4-6" reviewer = "claude-opus-4-8" prFixer = "claude-sonnet-4-6" ``` ## Testing Decisions Good tests here exercise external behaviour, not implementation detail. The overwhelming majority of this work is declarative (Nix expressions, prompt markdown, forge YAML, a remote change) and is validated by *building*, not by unit tests. Exactly one seam carries real branching logic, and it is the one seam that gets a unit test. - **Unit-tested seam — `loadConfig` (in `weiwen/sandcastle`).** Test the pure function only through its contract: valid TOML → the expected `ResolvedConfig` object; omitted optional fields → documented defaults applied (`maxIterations`, `maxConcurrency`, `models.*`); a missing required field (`repo`, `forgeHost`) → throws. Do not test how it parses internally or reach into private state. This is the single highest seam through which all downstream orchestrator behaviour flows, so testing it covers config resolution, model selection, and prompt-arg derivation at one point. Prior art: evie's existing `main.mts` had no such seam (values were hardcoded), so this is a new, deliberately-introduced testable boundary; the repo's `tsc --noEmit` typecheck script runs alongside it as the type gate. - **Build-gate (not a unit test) — `checks.sandcastleImage` in nibble's flake.** `nix flake check` builds the image; it fails if the image won't build or the toolchain/native-lib wiring regresses. This gates the consumer wiring without executing the loop. Prior art: evie's `checks.clippy` / `checks.tests` flake-check pattern. - **Manual e2e (out of automated scope) — one loop run.** `nix run .#sandcastleImage | docker load` then `nix run .#sandcastle` against a real `ready-for-agent` issue on nibble, validated by hand once. Deliberately not automated: it spawns agents and mutates forge state, so it is too expensive and nondeterministic to gate CI. ## Out of Scope - Any change to `evie`. It stays as-is and is migrated to the extracted core in later, separate work. - Automated integration/e2e tests around the running loop (agents, sandboxes, forge mutations). - Playwright/browser tooling inside the sandbox image — the sole nibble test runs offline against cached fixtures; Playwright remains a dev-shell-only concern for manual screenshots. - The `token-usage.sh` analytics helper — referenced by nothing in the loop; may be revisited as a shared utility later. - Parameterizing `fmt`/`check` for non-Rust consumers — the Rust commands are intentionally hardcoded now; generalize only when a non-Rust consumer appears. - Multi-system support beyond what `flake-utils.eachDefaultSystem` yields incidentally; the only exercised system is `x86_64-linux`. ## Further Notes - The extraction is validated on `nibble` first precisely because it is lower-stakes than `evie`; a successful nibble loop is the acceptance signal for the core's genericity. - `tea`'s slug auto-derivation depends on the local git remote's host matching a configured `tea` login host — which is exactly why nibble's `origin` is realigned to `forge.weiwen.dev`. The push URL still needs the explicit slug, which is why `repo` remains in `config.toml` even though `--repo` is dropped from `tea` calls. - The `configure-tea` / `fmt` / `check` scripts are baked into the image the same way `configure-tea` already is in evie — fixed script bodies, runtime env for the only varying parts (forge URL, credentials). - Consumer footprint target: adopting the core in a new repo should be the `sandcastle` flake input + `mkSandcastleImage` call + a `.sandcastle/{config.toml,.env,CODING_STANDARDS.md}` triplet + the triage workflow + label vocabulary — nothing more.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
sandcastle/issue-3
No results found.
Labels
Clear labels   
epic
Large body of work grouping related issues; break down before implementing.

  
in-review
Agent opened a PR; tracked there — no action needed on the issue

  
ready-for-agent
PRD ready for an agent to implement

  
ready-for-human
Implemented; awaiting human review/merge

No labels
epic
in-review
ready-for-agent
ready-for-human
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
weiwen/sandcastle#1
No description provided.