Sandbox generated artifact persistence implementation plan

Issue: #1719
Design: docs/architecture/sandbox-artifact-persistence-boundaries.md
Scope: Generated sandbox outputs promoted to durable R2 storage with org/Specialist/run boundaries.

Implementation goals

2026-06-04 follow-up: request-scoped artifact context is now wired end-to-end. API callers can pass runId; the agent accepts run_id/runId, forwards HUMANWORK_SPECIALIST_ID and HUMANWORK_RUN_ID to Hermes, and passes org_id/specialist_id/run_id explicitly into file-exchange persistence instead of relying on process-global env.

1. Generated sandbox output files are durable only after explicit promotion through the artifact API/proxy.
2. New R2 object keys for generated outputs include orgId, specialistId, and runId.
3. Artifact registration/access rejects unscoped or cross-scope keys.
4. Metadata/content is redacted or rejected for common secret patterns before persistence/exposure.
5. API caps generated artifact count and total generated artifact bytes per run.
6. Artifact downloads use short-lived signed leases and never persist signed URLs.
7. Deployment readiness clearly reports whether generated artifact persistence is durable.

Task 1 — Add generated artifact key helper

Candidate files

• api/src/runtime-control-plane/run-artifact.service.ts
• api/src/runtime-control-plane/artifact-credential-proxy.service.ts
• agent/r2_client.py
• New shared helper if practical: api/src/runtime-control-plane/run-artifact-key.ts

Work

• Create a canonical key builder:

  humanwork/{orgId}/{specialistId}/runs/{runId}/output/{filename}

• Sanitize every path component.

• Reject URLs, absolute paths, traversal, query strings, fragments, and control characters.

• Preserve legacy read compatibility for existing safe prefixes where necessary, but route new generated sandbox writes through the canonical key.

Acceptance criteria

• Unit tests prove a valid key is generated for an (orgId, specialistId, runId, filename) tuple.
• Unit tests reject cross-org, cross-run, cross-specialist, URL-shaped, traversal, query/fragment, and control-character keys.
• agent/r2_client.py outbound persistence can receive or derive specialistId and produces the canonical key for new generated outputs.

Task 2 — Carry Specialist/OSA context into artifact promotion

Candidate files

• api/src/runtime-control-plane/run-artifact.service.ts
• api/src/runtime-control-plane/artifact-credential-proxy.service.ts
• api/src/runtime-control-plane/runner-control-plane.service.ts
• agent/r2_client.py
• Agent runtime env wiring that already sets HUMANWORK_RUN_ID

Work

• Ensure artifact promotion has the run's Specialist context available.
• Prefer a concrete specialistId from the AgentRun/OSA record; if the current entity shape stores only osaId, resolve to Specialist before key validation.
• Include HUMANWORK_SPECIALIST_ID or an equivalent control-plane-provided value in the runner/agent environment only if it is derived server-side.
• Do not accept client/tool supplied Specialist IDs without comparing them to the run.

Acceptance criteria

• Registration rejects a generated artifact key whose Specialist segment does not match the run Specialist.
• Existing org/run lookup remains fail-closed if the run is missing.
• Tests cover both direct specialistId and OSA-derived Specialist context if both shapes exist.

Task 3 — Enforce API-level artifact caps

Candidate files

• api/src/runtime-control-plane/run-artifact.service.ts
• api/src/runtime-control-plane/runtime-control-plane.module.ts
• Tests in api/test/run-artifact.service.spec.ts

Work

• Add configurable caps:
  • RUNTIME_ARTIFACT_MAX_COUNT_PER_RUN, default 64.
  • RUNTIME_ARTIFACT_MAX_SINGLE_BYTES, default 25 MiB.
  • RUNTIME_ARTIFACT_MAX_TOTAL_BYTES_PER_RUN, default 100 MiB.
• Enforce caps before saving a new generated/file-backed artifact.
• Count and sum existing RunArtifact rows for the same orgId + runId and the explicit generated artifact kinds (sandbox_generated_artifact, sandbox_output). Metadata alone does not classify an artifact as generated for quota enforcement.
• Return 400 or 413 consistently when caps would be exceeded.

Acceptance criteria

• Tests prove max count, max single size, and max total size are enforced.
• Tests prove non-file-backed db_small_blob behavior remains compatible where intended.
• Ledger does not record successful creation events for rejected artifacts.

Task 4 — Redact or reject secret-bearing generated output

Candidate files

• api/src/runtime-control-plane/run-artifact.service.ts
• api/src/runtime-control-plane/runtime-redaction.ts
• agent/r2_client.py
• Tests in api/test/run-artifact.service.spec.ts and agent/tests/test_model_gateway_adapter.py

Work

• Keep the existing metadata guard that rejects secret-like metadata and persisted signed URLs.
• Add text artifact redaction before R2 upload where the agent owns file content.
• For content types that cannot be safely redacted, fail closed when common secret signatures are detected.
• Record redaction state in metadata using safe fields such as redactionStatus, redactedPatterns, or secret_free_hash.

Acceptance criteria

• Tests prove metadata containing raw API keys/tokens/passwords is rejected.
• Tests prove signed/presigned/public URLs are not persisted in metadata or ledger payloads.
• Tests prove text artifacts with secret-like assignments are redacted before upload, or rejected if redaction is not available.
• Tests prove binary/unknown secret-bearing artifacts fail closed rather than upload raw content.

Task 5 — Harden artifact proxy signing and leases

Candidate files

• api/src/runtime-control-plane/artifact-credential-proxy.service.ts
• api/src/runtime-control-plane/run-artifact-lease-issuer.service.ts
• api/src/runtime-control-plane/run-artifact.service.ts
• Tests in:
  • api/test/artifact-credential-proxy.service.spec.ts
  • api/test/run-artifact-lease-issuer.service.spec.ts
  • api/test/run-artifact.service.spec.ts

Work

• Ensure PUT and GET proxy leases validate canonical generated artifact keys.
• Keep generated artifact lease TTL at or below the 1-hour file-storage upper bound.
• Current API-issued R2 leases intentionally retain the existing stricter 15-minute cap; this satisfies the 1-hour maximum without widening exposure.
• Ensure no signed URL is stored in RunArtifact, run events, model transcripts, or file exchange manifests.

Acceptance criteria

• Tests prove expiresInSeconds above the service cap is clamped or rejected, and the service cap remains <= 3600 seconds.
• Tests prove signed URL values appear only in the immediate lease response.
• Tests prove ledger events contain lease ID, artifact ID, key/hash/size references, and TTL — not the signed URL.

Task 6 — Wire durable-storage readiness

Candidate files

• Agent/API startup or health modules that already expose readiness.
• agent/r2_client.py
• Deployment docs/runbooks.

Work

• Add an explicit readiness signal for generated artifact durability.
• In production or when HUMANWORK_FILE_STORAGE_MODE=r2, missing proxy/R2 config should fail readiness or warn loudly according to the service's existing readiness semantics.
• Do not print secret values; report env var names only.

Acceptance criteria

• Tests or smoke checks prove readiness reports durable artifact persistence as configured/missing.
• R2 mode fails closed when required storage/proxy configuration is absent.
• Local development can still use auto/local fallback without pretending artifacts are durable.

Task 7 — Documentation updates

Files

• agent/R2_INTEGRATION.md
• docs/design/R2_FILE_STORAGE_SPEC.md
• docs/architecture/sandbox-artifact-persistence-boundaries.md
• This file

Work

• Document the canonical key shape.
• Document transient sandbox vs durable promotion.
• Document caps, redaction, and signed URL TTL behavior, including the current stricter 15-minute runtime artifact lease cap.
• Cross-link #1719, #625, ADR-008, ADR-020, and #300.

Acceptance criteria

• Docs make clear this is R2 durable artifact persistence, not Durable Objects.
• Docs make clear sandbox-local files are not durable.
• Docs make clear generated sandbox artifacts are private client artifacts.

Suggested test matrix

Test area	Required cases
Key scoping	valid org/specialist/run key; wrong org; wrong specialist; wrong run; URL; traversal; query; fragment; control character
Proxy signing	PUT/GET use canonical key; unauthorized runner cannot sign; TTL max 1 hour or stricter; signed URL omitted from ledger
Registration	requires hash + byte size for file-backed artifacts; validates canonical generated key; rejects signed URLs in metadata
Caps	max count; max single bytes; max total bytes; rejected writes do not emit artifact.created
Redaction	metadata secret rejection; text content redaction/rejection; binary secret fail-closed; safe metadata accepted
Agent sweep	artifacts/outbound/ files produce canonical keys; R2 mode fails closed when proxy/R2 config missing; local mode remains dev-only fallback

Rollout notes

1. Land validation in observe/log mode only if needed for legacy compatibility, but new generated writes should use canonical keys immediately.
2. Backfill is not required for existing artifact rows unless a product path needs old files listed under the new Specialist segment.
3. Monitor rejection counts for artifact_write, proxy signing, and artifact registration to catch tools writing unsafe filenames or secret-bearing content.
4. Keep #625 open: this implementation hardens prefixes and API checks but does not create per-org R2 credentials.