Restrict which networks a sandboxed session can reach (was: local_network capability) #95

New issue

Open

opened 2026-08-13 03:15:27 +08:00 by weiwen · 0 comments

weiwen commented

2026-08-13 03:15:27 +08:00

Owner

A local_network capability for this was added and then removed in sandbox: remove the unenforced local_network capability, because it was never enforced and could not be enforced where it was written. This issue tracks doing it properly.

Why it can't be done inside bwrap

Netfilter state is per-network-namespace, so a sandbox is in one of two states and neither can be filtered from within:

--share-net (what we do): the sandbox is in the host netns. Any rule added there is global to the host, not scoped to the sandbox.
--unshare-net: the sandbox gets a fresh netns with only lo. Nothing is reachable at all, so there is nothing left to filter — and this breaks every turn, since the provider call originates inside the sandbox (see the amendment in docs/adr/0023-multi-user-isolated-workspaces.md).

bwrap has no middle setting, and rules must not live inside the sandbox's own netns anyway: under --unshare-all the process sits in a new user namespace and may hold CAP_NET_ADMIN over namespaces it owns, so it could flush its own policy. Enforcement has to sit outside the sandbox.

Not an option: seccomp (--seccomp) cannot inspect the destination address, because a classic BPF filter cannot dereference the sockaddr pointer. It can block AF_INET wholesale but never per-CIDR.

Options

A. Host nftables keyed to a cgroup v2 path. Keep --share-net; place each sandbox in a cgroup and match it in the host output chain (socket cgroupv2 level N "..."). No NAT, no netns, no DNS rerouting. Caveats: nft resolves the cgroup path to an id at rule load time, so a dynamically created per-session cgroup yields stale rules — use one stable cgroup per policy class, created once, with sessions placed into it at spawn. nft is not currently installed on the deploy host (only iptables).

B. eBPF cgroup/connect4 + connect6 hooks. Attach a program to the sandbox's cgroup and reject connects to denied prefixes. Most precise option: per-connect, no stale ids, attaches per-cgroup dynamically, and udp4_sendmsg covers DNS. Raw sockets are already unavailable (no CAP_NET_RAW under --unshare-all), so hooking connect is sufficient. Cost: a BPF toolchain dependency (aya or libbpf-rs).

C. Named netns + veth + NAT, filtered on the host side. Most container-like. bwrap has no --netns FD, so entry is via ip netns exec ... bwrap --share-net .... Most moving parts: masquerade, routing, MTU, and DNS has to be reachable or forwarded.

D. --unshare-net plus a userspace proxy bound in as a unix socket. Default-deny by construction, and it composes with the LLM-proxy seam ADR 0023 already wants for the provider credential (which is the real exfiltration fix). Downside: only proxy-aware clients honour HTTP(S)_PROXY, so for arbitrary binaries it is a convention rather than a boundary — though combined with --unshare-net, non-proxy traffic simply fails closed rather than escaping.

The CIDR list was also wrong

Whatever mechanism is chosen, the policy the removed code carried did not describe this host. It was ["10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16", "127.0.0.0/8"], and on the deploy host:

The resolver is 100.100.100.100 (Tailscale MagicDNS) with fd7a:115c:a1e0::53 — and 100.64.0.0/10 was not in the list. The tailnet, arguably the most sensitive private network here, was fully reachable under the "restricted" policy.
No IPv6 entries at all (fc00::/7, fe80::/10, and Tailscale's fd7a:115c:a1e0::/48 all unlisted), so any policy was trivially bypassable over v6.
It would have blocked the resolver had DNS been on a private v4 address, silently breaking all name resolution. Any design must special-case the resolver or forward DNS.
Live private ranges on this host include 192.168.1.0/24 (LAN), 10.62.0.0/24 (incusbr0), and 172.18.0.0/16 + 172.19.0.0/16 (container bridges).

Blocking 127.0.0.0/8 is a real part of the value, not an afterthought: evie's own HTTP API listens on 0.0.0.0:3888 and the Telegram webhook on 8443.

Acceptance criteria

A sandboxed session can reach the model provider and the public internet.
A restricted session cannot open a connection to any denied prefix, over v4 or v6, verified by a test that actually attempts a connect rather than asserting on generated argv.
DNS still resolves for a restricted session.
The policy is not editable from inside the sandbox.
If reintroduced as a per-user grant, the default (grant absent) must be the working configuration, so a misconfigured user degrades to "cannot reach the LAN" and never to "cannot reach the provider" — that inversion is what made the first attempt break every turn.

Sandboxed sessions currently reach every network the host can reach. There is no way to say "this session may talk to the model provider and the public internet, but not to my LAN, my tailnet, my container bridges, or services on localhost". A `local_network` capability for this was added and then removed in `sandbox: remove the unenforced local_network capability`, because it was never enforced and could not be enforced where it was written. This issue tracks doing it properly. ## Why it can't be done inside bwrap Netfilter state is per-network-namespace, so a sandbox is in one of two states and neither can be filtered from within: - `--share-net` (what we do): the sandbox is in the host netns. Any rule added there is global to the host, not scoped to the sandbox. - `--unshare-net`: the sandbox gets a fresh netns with only `lo`. Nothing is reachable at all, so there is nothing left to filter — and this breaks every turn, since the provider call originates inside the sandbox (see the amendment in `docs/adr/0023-multi-user-isolated-workspaces.md`). bwrap has no middle setting, and rules must not live inside the sandbox's own netns anyway: under `--unshare-all` the process sits in a new user namespace and may hold `CAP_NET_ADMIN` over namespaces it owns, so it could flush its own policy. **Enforcement has to sit outside the sandbox.** Not an option: seccomp (`--seccomp`) cannot inspect the destination address, because a classic BPF filter cannot dereference the `sockaddr` pointer. It can block `AF_INET` wholesale but never per-CIDR. ## Options **A. Host nftables keyed to a cgroup v2 path.** Keep `--share-net`; place each sandbox in a cgroup and match it in the host `output` chain (`socket cgroupv2 level N "..."`). No NAT, no netns, no DNS rerouting. Caveats: nft resolves the cgroup path to an id **at rule load time**, so a dynamically created per-session cgroup yields stale rules — use one stable cgroup per policy class, created once, with sessions placed into it at spawn. `nft` is not currently installed on the deploy host (only `iptables`). **B. eBPF `cgroup/connect4` + `connect6` hooks.** Attach a program to the sandbox's cgroup and reject connects to denied prefixes. Most precise option: per-connect, no stale ids, attaches per-cgroup dynamically, and `udp4_sendmsg` covers DNS. Raw sockets are already unavailable (no `CAP_NET_RAW` under `--unshare-all`), so hooking connect is sufficient. Cost: a BPF toolchain dependency (aya or libbpf-rs). **C. Named netns + veth + NAT, filtered on the host side.** Most container-like. bwrap has no `--netns FD`, so entry is via `ip netns exec ... bwrap --share-net ...`. Most moving parts: masquerade, routing, MTU, and DNS has to be reachable or forwarded. **D. `--unshare-net` plus a userspace proxy bound in as a unix socket.** Default-deny by construction, and it composes with the LLM-proxy seam ADR 0023 already wants for the provider credential (which is the *real* exfiltration fix). Downside: only proxy-aware clients honour `HTTP(S)_PROXY`, so for arbitrary binaries it is a convention rather than a boundary — though combined with `--unshare-net`, non-proxy traffic simply fails closed rather than escaping. ## The CIDR list was also wrong Whatever mechanism is chosen, the policy the removed code carried did not describe this host. It was `["10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16", "127.0.0.0/8"]`, and on the deploy host: - The resolver is **`100.100.100.100`** (Tailscale MagicDNS) with `fd7a:115c:a1e0::53` — and `100.64.0.0/10` was **not** in the list. The tailnet, arguably the most sensitive private network here, was fully reachable under the "restricted" policy. - **No IPv6 entries at all** (`fc00::/7`, `fe80::/10`, and Tailscale's `fd7a:115c:a1e0::/48` all unlisted), so any policy was trivially bypassable over v6. - It *would* have blocked the resolver had DNS been on a private v4 address, silently breaking all name resolution. Any design must special-case the resolver or forward DNS. - Live private ranges on this host include `192.168.1.0/24` (LAN), `10.62.0.0/24` (incusbr0), and `172.18.0.0/16` + `172.19.0.0/16` (container bridges). Blocking `127.0.0.0/8` is a real part of the value, not an afterthought: evie's own HTTP API listens on `0.0.0.0:3888` and the Telegram webhook on `8443`. ## Acceptance criteria - A sandboxed session can reach the model provider and the public internet. - A restricted session cannot open a connection to any denied prefix, over v4 **or** v6, verified by a test that actually attempts a connect rather than asserting on generated argv. - DNS still resolves for a restricted session. - The policy is not editable from inside the sandbox. - If reintroduced as a per-user grant, the default (grant absent) must be the *working* configuration, so a misconfigured user degrades to "cannot reach the LAN" and never to "cannot reach the provider" — that inversion is what made the first attempt break every turn.