ci: build and publish the evie image to the Forgejo registry on main #88

Merged

weiwen merged 1 commit from fm/evie-ci-image-publish into main

2026-08-12 18:02:55 +08:00

weiwen commented

2026-08-05 03:15:00 +08:00

Owner

CI now builds packages.eviedaemonImage on every push to main and publishes it to the
private Forgejo container registry; deploy/docker/ pulls that image instead of assuming a
local nix build.

.forgejo/workflows/ci.yml: new publish job (needs: check).
flake.nix: devShells.publish (skopeo only), so the push tool is pinned by flake.lock
rather than by whatever the runner's flake registry resolves to.
deploy/docker/docker-compose.yaml: image: forge.weiwen.dev/weiwen/evie:latest.
deploy/docker/README.md, docs/deploy-docker-migration.md: the flow is now
docker login forge.weiwen.dev once, then docker compose pull && docker compose up -d.
CONTEXT.md, AGENTS.md: pointers, no duplicated detail.

⚠️ Needs one Actions secret before it can publish

The automatic Actions token cannot write packages on this instance. The first CI run on
this PR measured it: POST /v2/weiwen/evie/blobs/uploads/ as ${{ github.repository_owner }}
with ${{ github.token }} → HTTP 401 (Forgejo 15.0.6, matching Forgejo issues #1296 /
#3571). So this needs:

a Forgejo token with package: read and write, added as the Actions secret
REGISTRY_TOKEN on weiwen/evie (REGISTRY_USER too, but only if the token is not
owned by weiwen).

Nothing is hardcoded and no credential was invented. The job reads REGISTRY_TOKEN and does
not fall back to github.token — a fallback to a credential now known to 401 would only
convert "missing secret" into a confusing auth error. With the secret absent, the credential
check fails fast with exactly the instruction above; that is why CI / publish is red on
this PR. It goes green once the secret exists, with no code change.

Why the `native` runner, not the `ubuntu-latest` container `check` uses

The matsutake runner offers both labels (ubuntu-latest:docker://ghcr.io/catthehacker/ubuntu:act-latest
and native:host). publish takes native:

The image closure is ~5 GB and the host nix store already holds it: nix build .#eviedaemonImage completes in 19 s there. In a throwaway container it is a multi-GB
substitution on every push, and cache-nix-action is configured with
gc-max-store-size-linux: 1G, so it cannot hold that closure.
The host nix.conf carries the extra substituters that serve pi (cache.numtide.com).
A container provisioned by install-nix-action does not, so those paths risk being
built from source rather than fetched.
The container has no docker daemon (the runner mounts no socket), which also settles the
push mechanism below.

check is untouched and stays on ubuntu-latest.

Why skopeo, not `docker load` + `docker push`

eviedaemonImage is a streamLayeredImage: a script that writes a docker-archive tarball to
stdout. skopeo reads it straight off the pipe, so nothing multi-GB is staged on disk and no
daemon is involved:

"$stream" | skopeo copy --dest-compress docker-archive:/dev/stdin docker://$IMAGE:$sha

docker load + docker push was not an option anyway (no daemon on the runner), and it
would also mean writing 4.9 GB into a daemon's storage first. --dest-compress gzips the
layers on the way up: 4.9 GB of archive becomes ~1.67 GB pushed, as a manifest.v2+json
with ...rootfs.diff.tar.gzip layers — exactly what docker compose pull wants.

Proven vs. unverified-until-merge

Proven in CI (this PR's run, on the real native runner):

The native label works for this repo, which no workflow used before: Set up job and
actions/checkout@v7 both succeeded in host mode under the forgejo-runner user.
The credential check works and its diagnosis is trustworthy: it is what produced the 401
finding above, and it left no package behind.
CI / check still green (2 m 12 s), unaffected.

Proven by hand on matsutake (the same host and nix store the native runner uses):

nix build .#eviedaemonImage — 19 s warm; the stream is 4.9 GB, 83 s to generate.
Daemonless push: stream | skopeo copy --dest-compress docker-archive:/dev/stdin docker://forge.weiwen.dev/... — 2 m 09 s, 1.67 GB / 96 gzip layers, manifest.v2+json,
verified with skopeo inspect --raw.
The :latest registry-side copy — 3.9 s.
nix develop .#publish -c skopeo, including REGISTRY_AUTH_FILE pass-through into the dev
shell and skopeo login --password-stdin.
The credential-check script run verbatim in all three branches: pass with a package-write
token (202 then 204), the unset-secret error, and a 401 with a bad token.
Those probes used a throwaway package (weiwen/evie-ci-probe) that was deleted; the
weiwen container package list is back to empty, so the first real image comes from CI.
just check green (490 tests), workflow YAML parses, every run: block passes bash -n
and shellcheck, compose YAML parses.

Unverified until the first push to main after this merges:

The publish itself has never run in CI. Every piece of the mechanism is proven above,
but the Publish step is skipped on pull requests (if: github.event_name == 'push'), so
the first real proof that :latest and :<sha> land in the registry is that first main
run. Do not deploy off :latest before it is green.
Also unproven in CI: nix build on the native runner (as forgejo-runner, against the
host store and daemon). The credential check fails ahead of it, so that step has not yet
run there — it is only proven as weiwen on the same host.

CI now builds `packages.eviedaemonImage` on every push to `main` and publishes it to the private Forgejo container registry; `deploy/docker/` pulls that image instead of assuming a local `nix build`. - `.forgejo/workflows/ci.yml`: new `publish` job (`needs: check`). - `flake.nix`: `devShells.publish` (skopeo only), so the push tool is pinned by `flake.lock` rather than by whatever the runner's flake registry resolves to. - `deploy/docker/docker-compose.yaml`: `image: forge.weiwen.dev/weiwen/evie:latest`. - `deploy/docker/README.md`, `docs/deploy-docker-migration.md`: the flow is now `docker login forge.weiwen.dev` once, then `docker compose pull && docker compose up -d`. - `CONTEXT.md`, `AGENTS.md`: pointers, no duplicated detail. ## ⚠️ Needs one Actions secret before it can publish **The automatic Actions token cannot write packages on this instance.** The first CI run on this PR measured it: `POST /v2/weiwen/evie/blobs/uploads/` as `${{ github.repository_owner }}` with `${{ github.token }}` → **HTTP 401** (Forgejo 15.0.6, matching Forgejo issues #1296 / #3571). So this needs: > a Forgejo token with **package: read and write**, added as the Actions secret > **`REGISTRY_TOKEN`** on `weiwen/evie` (`REGISTRY_USER` too, but only if the token is not > owned by `weiwen`). Nothing is hardcoded and no credential was invented. The job reads `REGISTRY_TOKEN` and does **not** fall back to `github.token` — a fallback to a credential now known to 401 would only convert "missing secret" into a confusing auth error. With the secret absent, the credential check fails fast with exactly the instruction above; **that is why `CI / publish` is red on this PR.** It goes green once the secret exists, with no code change. ## Tags `forge.weiwen.dev/weiwen/evie:<12-char short sha>` is pushed from the stream, then `:latest` is a registry-side copy of it (blobs already uploaded, so it is a manifest write and takes seconds). Deploys can pin to the short sha; `:latest` always means the tip of `main`. Version tags are **not** wired up: `ci.yml` triggers on `main` only, and adding `tags: ['v*']` would also drag the `check` job onto tag pushes for no benefit today. ## Why the `native` runner, not the `ubuntu-latest` container `check` uses The `matsutake` runner offers both labels (`ubuntu-latest:docker://ghcr.io/catthehacker/ubuntu:act-latest` and `native:host`). `publish` takes `native`: - The image closure is ~5 GB and the host nix store already holds it: `nix build .#eviedaemonImage` completes in **19 s** there. In a throwaway container it is a multi-GB substitution on every push, and `cache-nix-action` is configured with `gc-max-store-size-linux: 1G`, so it cannot hold that closure. - The host `nix.conf` carries the extra substituters that serve `pi` (`cache.numtide.com`). A container provisioned by `install-nix-action` does not, so those paths risk being **built** from source rather than fetched. - The container has no docker daemon (the runner mounts no socket), which also settles the push mechanism below. `check` is untouched and stays on `ubuntu-latest`. ## Why skopeo, not `docker load` + `docker push` `eviedaemonImage` is a `streamLayeredImage`: a script that writes a docker-archive tarball to stdout. skopeo reads it straight off the pipe, so nothing multi-GB is staged on disk and no daemon is involved: ```sh "$stream" | skopeo copy --dest-compress docker-archive:/dev/stdin docker://$IMAGE:$sha ``` `docker load` + `docker push` was not an option anyway (no daemon on the runner), and it would also mean writing 4.9 GB into a daemon's storage first. `--dest-compress` gzips the layers on the way up: 4.9 GB of archive becomes ~1.67 GB pushed, as a `manifest.v2+json` with `...rootfs.diff.tar.gzip` layers — exactly what `docker compose pull` wants. ## Proven vs. unverified-until-merge **Proven in CI** (this PR's run, on the real `native` runner): - The `native` label works for this repo, which no workflow used before: `Set up job` and `actions/checkout@v7` both succeeded in host mode under the `forgejo-runner` user. - The credential check works and its diagnosis is trustworthy: it is what produced the 401 finding above, and it left no package behind. - `CI / check` still green (2 m 12 s), unaffected. **Proven by hand on `matsutake`** (the same host and nix store the `native` runner uses): - `nix build .#eviedaemonImage` — 19 s warm; the stream is 4.9 GB, 83 s to generate. - Daemonless push: `stream | skopeo copy --dest-compress docker-archive:/dev/stdin docker://forge.weiwen.dev/...` — 2 m 09 s, 1.67 GB / 96 gzip layers, `manifest.v2+json`, verified with `skopeo inspect --raw`. - The `:latest` registry-side copy — 3.9 s. - `nix develop .#publish -c skopeo`, including `REGISTRY_AUTH_FILE` pass-through into the dev shell and `skopeo login --password-stdin`. - The credential-check script run verbatim in all three branches: pass with a package-write token (202 then 204), the unset-secret error, and a 401 with a bad token. - Those probes used a throwaway package (`weiwen/evie-ci-probe`) that was deleted; the `weiwen` container package list is back to empty, so the first real image comes from CI. - `just check` green (490 tests), workflow YAML parses, every `run:` block passes `bash -n` and `shellcheck`, compose YAML parses. **Unverified until the first push to `main` after this merges:** - **The publish itself has never run in CI.** Every piece of the mechanism is proven above, but the `Publish` step is skipped on pull requests (`if: github.event_name == 'push'`), so the first real proof that `:latest` and `:<sha>` land in the registry is that first `main` run. Do not deploy off `:latest` before it is green. - Also unproven in CI: `nix build` on the `native` runner (as `forgejo-runner`, against the host store and daemon). The credential check fails ahead of it, so that step has not yet run there — it is only proven as `weiwen` on the same host.

weiwen added 1 commit

2026-08-05 03:15:00 +08:00

ci: build and publish the evie image to the Forgejo registry on main

CI / check (pull_request) Successful in 2m12s

Details

PR Triage — label changes-requested reviews / triage-review (pull_request) Successful in 1s

Details

CI / publish (pull_request) Failing after 2s

Details

1bb716a8d6

Deploys built the image by hand (`nix build .#eviedaemonImage` then `docker load`,
~5 GB streamed) on a host that had to carry nix. CI now does it: the `publish` job
builds the image and pushes `forge.weiwen.dev/weiwen/evie:latest` plus a 12-char
short-sha tag to pin against, and `deploy/docker/` pulls instead of building.

It runs on the `native` runner rather than the `ubuntu-latest` container `check`
uses. The image closure is ~5 GB, and the host store already holds it along with the
substituters that serve `pi` from cache.numtide.com; a throwaway container would
re-fetch or rebuild that on every push. `needs: check` keeps `:latest` off a red
build.

skopeo pushes the `streamLayeredImage` archive straight off the pipe (gzipping the
layers: ~1.7 GB uploaded, nothing staged on disk). The runner has no docker daemon,
so `docker load` + `docker push` was never an option. `:latest` is then a
registry-side copy, which only writes a second manifest.

Pull requests run the build and a registry write probe (a blob upload opened and
immediately cancelled - the real permission test, unlike `skopeo login`) but skip
the push, so a broken credential surfaces before a merge instead of after one.

weiwen force-pushed fm/evie-ci-image-publish from 1bb716a8d6

CI / check (pull_request) Successful in 2m12s

Details

PR Triage — label changes-requested reviews / triage-review (pull_request) Successful in 1s

Details

CI / publish (pull_request) Failing after 2s

Details

to e5b6d47ef2

PR Triage — label changes-requested reviews / triage-review (pull_request) Successful in 2s

Details

CI / check (pull_request) Successful in 1m26s

Details