Skip to main content

Sandbox worker + supervisor runbook

Production hostile-code sandbox capability execution is disabled until the cutover gate is GO. Current default is NO_GO because ADR-022 cosign and eligible-host smoke artifacts are not recorded.

Native/systemd deployment​

  1. Provision a dedicated Linux host. Do not colocate general app workloads.
  2. Install pinned bubblewrap at /usr/bin/bwrap; record the package version and hash in the cutover evidence.
  3. Create the service identity:
    • user/group: humanwork-sandbox
    • runtime socket dir: /run/humanwork-sandbox
    • workspace root: /var/lib/humanwork-sandbox
  4. Enable project quotas for /var/lib/humanwork-sandbox (prjquota for ext4 or XFS project quota). Missing quota support is NO_GO.
  5. Enable cgroup v2 delegation for the supervisor service. humanwork-sandbox-supervisor.service uses Delegate=yes and sets HUMANWORK_SANDBOX_CGROUP_ROOT=/sys/fs/cgroup/system.slice/humanwork-sandbox-supervisor.service; that delegated subtree must allow pids/memory/cpu/io writes and cgroup.kill.
  6. Install:
    • /opt/humanwork/sandbox-supervisor/bin/humanwork-sandbox-supervisor
    • /opt/humanwork/sandbox-supervisor/bin/capability-shell
    • /opt/humanwork/sandbox-supervisor/bin/sandbox-init
    • /opt/humanwork/sandbox-supervisor/bin/browser-action-adapter
    • /opt/humanwork/sandbox-supervisor/bin/true (bound as /sandbox/bin/true for smoke and canonical exec allowlist checks)
    • Chromium runtime at /usr/lib/chromium/chromium (bound read-only as /sandbox/bin/chromium; the supervisor also binds Chromium's read-only /usr/lib* runtime directories when browser_action is dispatched)
    • /opt/humanwork/sandbox-supervisor/policy/*
    • /opt/humanwork/sandbox-worker/worker.py, supervisor_client.py, supervisor_protocol.py
  7. Install deploy/systemd/humanwork-sandbox-supervisor.service and deploy/systemd/humanwork-sandbox-worker.service. The supervisor unit must keep SANDBOX_INIT_BIN=/opt/humanwork/sandbox-supervisor/bin/sandbox-init, HUMANWORK_BROWSER_RUNTIME_BIN=/usr/lib/chromium/chromium, HUMANWORK_BROWSER_ADAPTER_BIN=/opt/humanwork/sandbox-supervisor/bin/browser-action-adapter, HUMANWORK_SANDBOX_EXECUTABLE_DIR=/opt/humanwork/sandbox-supervisor/bin, and the delegated HUMANWORK_SANDBOX_CGROUP_ROOT above.
  8. Put only SANDBOX_WORKER_TOKEN and non-secret primitive config in /etc/humanwork/sandbox-worker.env. Do not pass caller env into sandbox dispatch.
  9. Start services:
    • systemctl enable --now humanwork-sandbox-supervisor.service
    • systemctl enable --now humanwork-sandbox-worker.service
  10. Run smoke from the host or a controlled ops box:
  • SANDBOX_WORKER_URL=http://127.0.0.1:8080 SANDBOX_WORKER_TOKEN=... bun api/scripts/smoke-sandbox-supervisor.ts

Supervisor socket permissions must remain owner-only (0600) under /run/humanwork-sandbox/supervisor.sock; the worker runs as the same service identity.

Dedicated K8s node deployment​

Use infra/k8s/sandbox-worker.yaml only on a labelled dedicated node that satisfies infra/k8s/sandbox-worker-node-requirements.md.

  • The default ConfigMap keeps project quota/proxy facts false, so readiness is NO_GO until host smoke proves the node.
  • Pin the worker image by digest for production; do not deploy latest as the final cutover artifact.
  • The manifest must not mount the Docker socket, host home, .ssh, host credentials, broad /etc, broad /run, or host root.
  • /var/lib/humanwork-sandbox is the only host workspace bind and must have hard project quota enforcement.
  • /sys/fs/cgroup access is allowed only for the delegated cgroup v2 subtree on the dedicated node.
  • Enable CAP_NET_ADMIN only for proxy-only mode after smoke proves veth/nftables/proxy behavior.

GO / NO_GO checks​

GET /health from the worker must report:

  • status="GO" and productionReady=true.
  • checks.adrCosign.status="GO".
  • checks.hostPrimitives.status="GO" with bwrap/runtime/seccomp hashes.
  • checks.productionCutoverGate.status="GO".
  • no noGoReasons.

api/scripts/smoke-sandbox-supervisor.ts must then pass:

  • worker health and primitive readiness with bwrap, namespace, cgroup, seccomp, quota, Landlock, proxy, process/cancel, Chromium, and adapter evidence;
  • artifact write/read;
  • exec_argv with a canonical /sandbox/... executable;
  • browser_action through proxy-only, returning adapter-produced action evidence rather than a denied/error success substitute;
  • full supervisor evidence schema validation;
  • hostile-deny subset and the full eligible-host hostile battery artifact;
  • release/cutover evidence hash binding.

On GitHub-hosted or otherwise ineligible hosts, smoke may be run with ALLOW_NO_GO=1 to publish a NO_GO/skipped artifact. That artifact is not production readiness.

Proxy-only specifics​

network=proxy-only requires supervised netns/veth, host proxy config, nftables allow-only-proxy rules, and CAP_NET_ADMIN. If any primitive is absent, dispatch must return proxy_only_unavailable. It must not downgrade to host networking or report network=off as proxy-only success.

Landlock evidence​

Every supervisor result includes landlock. If the kernel ABI is available, sandbox-init must apply Landlock and report enabled=true, rulesetApplied=true. If unavailable, evidence must say available=false, reason="kernel_unsupported"; this is allowed only because bwrap mount policy, cgroups, quotas, and seccomp remain load-bearing.

Rollback​

Rollback routes traffic away from sandbox capability execution or deploys a previous supervisor/worker build that still satisfies the same evidence contract.

Rollback must never route to backend=none, sandbox=off, direct spawn, raw bwrap success, degraded success, or worker success without linux-bwrap-supervisor evidence. If no eligible worker exists, disable the sandbox capability path and fail closed rather than running unsandboxed code.

Last updated on by Degen