Skip to main content

Final reviewed handoff plan: hard-cut Rig-derived runtime control plane for Human.work

✅ SHIPPED via PR #696 (merged 2026-05-25). Milestones 0–9 of this plan have landed; the issue-disposition matrix tracks per-issue status. This document is retained for architectural context and historical reference. For current sandbox supervisor design see HUMANWORK_SANDBOX_SUPERVISOR.md.

Architecture deck this plan implements:

Read those first if you need the architectural rationale behind any decision in this plan. This document is the implementation/cutover plan; the deck above is the spec it lands.

Review iteration result​

The three reviews all agreed with the main architecture: use Rig as the cleaned reference architecture, but implement a Human.work-native NestJS/API control plane; make Hermes/Python a runner/model adapter; route tools, credentials, approvals, events, artifacts, and lifecycle state through platform primitives; do not port Rig product/UI or old Splinter bridge semantics.

Valid critiques incorporated here:

  • Add a real webhook/channel freeze-drain-backfill-reopen cutover protocol with high-water marks, queue pause/resume, runner bootstrap gate, go/no-go, and forward-fix containment.
  • Treat org-less channel traffic explicitly: no runtime execution without org resolution; unresolved inbound events go to quarantine/remediation, not agent_runs.
  • Strengthen cross-org integrity beyond service checks: composite org keys/FKs where possible, denormalized org columns/triggers where existing product tables do not support composite FKs, and transactional assertions.
  • Hotfix or production-disable the current /agentic/tasks* IDOR/mutation paths before the big branch cutover.
  • Replace event sequence max()+1 with an actual concurrency primitive.
  • Define fail-open precisely for runner/agent failures, and fail-closed/retry behavior when DB persistence is unavailable.
  • Constrain runner dispatch endpoints to avoid SSRF/confused-deputy behavior; prefer outbound polling/service discovery over DB-provided URLs.
  • Resolve the status/approval mismatch with generic waiting statuses and first-class approval/input rows.
  • Add migration-only Postgres CI with synchronize=false; SQLite synchronize tests are not enough.
  • Cut additional legacy authority routes: /agentic/tasks* and /v1/agent-api/tools/:toolName.
  • Define RLS/session strategy for HTTP, webhooks, workers, backfills, runner callbacks, tests, and PgBouncer/transaction pooling.
  • Fix the policy_hash/tool_manifest_hash ordering conflict by precomputing immutable hashes before inserting the run row.
  • Make default HITL/no-autosend a first-class policy invariant.
  • Add approval authorization matrix, internal-role org filtering, redaction-before-audit/broadcast, no persisted signed artifact URLs, sandbox leases/cleanup, manifest signing/key rotation, side-effect idempotency, quotas/rate limits, and required indexes.
  • Promote packages/agent-harness disposition and KB/Haystack access to implementation-lock decisions.
  • Add issue dispositions for #417, #613/#559/#585/#587, #558, #505, and #450.

Weak/limited critiques rejected or narrowed:

  • “Add owners” is not actionable in this subagent handoff because named owners are outside the code context and require project management assignment. This plan adds a WBS/milestone structure and explicit gates; the parent/planner can assign owners.
  • “Durable model.delta for every token” is corrected rather than deleted wholesale: raw token streaming should be ephemeral; durable events should be coalesced chunks/checkpoints plus artifact-backed transcripts where needed.

Decision: port from Rig, not old Splinter​

Port from Rig’s cleaned architecture, not old Splinter. Rig is the successor/reference because it extracted the useful engine shape: explicit run lifecycle, append-only event grammar, read models, policy decision objects, declarative plugin/validator/tool contribution patterns, runtime isolation, and tool materialization.

This is not a wholesale import of Rig as a product, CLI, desktop/TUI, task tracker, worktree manager, or server. Human.work keeps its domain: Specialist, Expert, client/org, AM, SuperAdmin. Do not expose Rig/Splinter terminology in product UI.

Hard call: one atomic target architecture, no middle production stage, no compatibility layer, no dual authority. Build and rehearse in branch/staging; backfill active state during a maintenance window; deploy API/frontend/agent/runner/workers together; delete or return 410 for old production authority paths.

Target authority boundaries​

  1. NestJS/API control plane owns authority

    • org scope, IDOR checks, RLS context, and internal-role assignment filters
    • credentials, integration proxying, and model/provider credential brokerage
    • policy decisions
    • tool dispatch and side-effect idempotency
    • approval/input state
    • durable event ledger
    • artifacts and signed access generation
    • run state, resumability, cancel/retry
    • runner manifest registration and validation

  2. Hermes/Python becomes a runner/model adapter only

    • receives signed run contracts and short-lived run tokens
    • emits signed model/provider events
    • requests model calls through the platform model gateway/credential broker, not broad provider env keys
    • requests tools/capabilities from the platform
    • never owns vendor integration secrets, org policy, tool permission decisions, R2/platform credentials, or authoritative lifecycle state

  3. Sandbox worker owns constrained file/process/browser execution

    • typed capability operations only
    • no raw shell API
    • no broad env secrets
    • OS/container isolation is mandatory for process/browser work
    • leases, heartbeats, dead-letter states, TTL workspace cleanup, orphan process reaping

  4. Ops/workspace/client UIs are surfaces over platform primitives

    • resolve approval_requests and user_input_requests
    • inspect agent_runs, run_events, tool_calls, policy_decisions, run_artifacts
    • do not mutate task JSON plans as a workflow state machine

Legacy production authority to delete or 410​

At cutover, delete or return 410 with tests for:

  • LEGACY_AGENT_API and direct legacy /chat calling from AgentClient
  • AGENT_URL fallback semantics; use runner registration/manifest selection only
  • /tools/execute broad PLATFORM_API_TOKEN route
  • /v1/agent-api/tools/:toolName direct platform tool execution, unless rewritten as a thin internal run-scoped wrapper that requires a signed run token and calls ToolGatewayService.execute(runId, ...) with policy/tool_call records
  • Python direct Shopify/Amazon/vendor handlers and platform_proxy fallback
  • /agentic/tasks* as authoritative workflow state: mutating create/approve/reject/plan/SSE authority must be removed/410 or replaced with new run/approval endpoints; optional read-only traceability must read from agent_runs/run_events after org checks
  • /v1/tasks/* if present as task authority
  • duplicated AgentApiService.handleEvent / AgentEventsService event authority split
  • process-local event idempotency maps
  • AgenticTaskStatus legacy aliases as active runtime statuses (planning, pending_approval, approved, executing, paused_for_approval, needs_input, rejected, etc.)
  • AgenticTask.plan as approval source of truth
  • runtime writes/reads to actions for lifecycle/tool execution after cutover
  • production stub success paths and mock HP credentials outside explicit tests/dev fakes
  • broad model/vendor/R2/platform secrets in agent/runner/sandbox env

Pre-cutover emergency item: if /agentic/tasks* is reachable in production, hotfix org-scoped read/list/approve/reject or production-disable it before waiting for the full runtime branch.

Target schema and invariants​

Use explicit Postgres migrations. Do not rely on TypeORM synchronize; add a Postgres migration-only CI job with SYNCHRONIZE=false. SQLite synchronize tests may remain for unit speed, but they cannot be the migration/RLS source of truth.

Every org-scoped table needs:

  • org_id not null unless it is a quarantine/unresolved-ingest table by design
  • RLS policy and service-layer org checks
  • composite unique key (id, org_id) for new parent tables
  • composite foreign keys (parent_id, org_id) wherever the parent table supports it
  • DB triggers or denormalized org columns for references into existing tables that currently scope only indirectly (messages, expert_queue, nullable conversations.orgId)
  • transactional service assertions on every optional cross-reference
  • indexes for promised Ops/read-model queries

Cross-org referential integrity strategy​

Implement one of these before runtime tables ship; preferred path is explicit:

  1. Add/backfill denormalized org_id on product tables referenced by runs/events (messages, expert_queue), derived from conversations.orgId, and enforce it for all new org-resolved rows. Add composite unique keys (id, org_id).
  2. For historical nullable/ambiguous rows, add DB triggers on runtime tables that assert any referenced conversation_id, message_id, expert_queue_item_id, agentic_task_id, tool_call_id, approval_request_id, or artifact_id belongs to the same org_id, rejecting mismatches.
  3. Runtime-linked rows must never point to org-less product records. Org-less records are quarantine/remediation only.

Service checks are required but not sufficient; migrations/backfills/workers must also be unable to write mismatched org refs.

agent_runs​

Authoritative row for every org-resolved Specialist/model/runtime execution: conversation draft, Expert side-chat, agentic task, tool-only operation, validation job, browser automation, ops manual run.

Core columns:

  • id uuid pk
  • org_id uuid not null references organizations(id) plus composite (id, org_id) unique
  • optional refs: conversation_id, message_id, expert_queue_item_id, agentic_task_id, each org-consistency checked
  • specialist_id uuid null references specialists(id)
  • run_kind text not null: conversation_draft | expert_side_chat | agentic_task | tool_action | validation | browser_automation | ops_manual
  • status text not null: created | queued | preparing | running | waiting_approval | waiting_input | tool_executing | validating | reviewing | completed | failed | cancelled | paused
  • runtime_adapter text not null: initial hermes
  • runtime_mode text not null: interactive | supervised | autonomous, default supervised
  • execution_target text not null: control_plane | sandbox_worker | remote_worker
  • model text null
  • runner_id uuid null
  • runner_hash text null
  • policy_hash text not null
  • tool_manifest_hash text not null
  • prompt_bundle_hash text null
  • event_schema_version text not null
  • next_event_sequence bigint not null default 1
  • pending_approval_count int not null default 0
  • pending_input_count int not null default 0
  • timestamps: started_at, completed_at, updated_at, created_at
  • error_code text null, error_text text null
  • metadata jsonb not null default '{}'::jsonb

Creation invariant: allocate runId, compute immutable policy/tool/prompt hashes for the run preparation context, then insert the run row. Do not insert placeholder hashes. If hash generation fails and DB is available, create a failed/escalated run only with a well-defined failure hash/artifact policy; do not create a runnable row with fake hashes.

Indexes:

  • (org_id, status, updated_at)
  • (org_id, run_kind, status, updated_at)
  • (org_id, specialist_id, status, updated_at)
  • (org_id, conversation_id, created_at) where conversation ref exists

run_events​

Append-only durable event ledger replacing process-local callbacks and audit-log-as-timeline.

Columns:

  • id uuid pk
  • org_id uuid not null
  • run_id uuid not null with composite FK (run_id, org_id) → agent_runs(id, org_id)
  • optional refs: conversation_id, message_id, expert_queue_item_id, agentic_task_id, tool_call_id, approval_request_id, artifact_id, all org-consistency checked
  • sequence bigint not null; unique (run_id, sequence)
  • idempotency_key text null; unique (run_id, source, idempotency_key) where not null
  • type text not null
  • source text not null: api | agent_runtime | runner | sandbox | tool_gateway | policy | expert | client | system
  • actor_type text not null: specialist_runtime | expert | client_user | system | ops_user
  • actor_id text null
  • correlation_id text not null
  • payload jsonb not null default '{}'::jsonb
  • redaction_level text not null: masked | artifact_ref | secret_free_hash (none is disallowed for provider raw logs, transcripts, model chunks, sandbox logs, and any data class that may carry PII/secrets unless a validator proves secret/PII-free)
  • created_at timestamptz not null default now()
  • signature_verified_at timestamptz null for signed runner/callback envelopes

Event append primitive: RunEventLedgerService.append() runs in a transaction, locks the parent agent_runs row FOR UPDATE, reads/increments next_event_sequence, inserts the event, updates run status/counters if requested, and broadcasts only after commit. No max(sequence)+1.

Event signatures must bind {orgId, runId, source, type, correlationId, idempotencyKey, payloadHash, issuedAt}; if a runner supplies a sequence hint, it is advisory and must also be signed but the API assigns final sequence.

Indexes:

  • (run_id, sequence)
  • (org_id, created_at, type)
  • (org_id, run_id, created_at)
  • (org_id, source, created_at)

Streaming rule: raw token streaming should be ephemeral SSE/WebSocket. Durable model.delta rows should be coalesced chunks/checkpoints, not one row per token. Store full transcripts as redacted artifacts or artifact references with retention/partitioning rules.

Initial taxonomy:

  • run: run.created, run.queued, run.started, run.status_changed, run.completed, run.failed, run.cancelled
  • runner: runner.selected, runner.started, runner.manifest.loaded, runner.policy.loaded, runner.rejected, runner.heartbeat, runner.exited
  • model/provider: model.started, model.chunk, model.completed, model.failed, provider.raw_event as redacted artifact ref only
  • policy: policy.decision
  • tool: tool.requested, tool.allowed, tool.blocked, tool.approval_required, tool.started, tool.succeeded, tool.failed
  • approval/input: approval.requested, approval.resolved, input.requested, input.resolved
  • artifacts: artifact.created, artifact.redacted, artifact.accessed
  • sandbox/capability: capability.requested, capability.denied, sandbox.exec.started, sandbox.exec.denied, sandbox.exec.finished, process.timeout
  • validation/review: validation.started, validation.completed, review.started, review.completed
  • channel/message: draft.created, message.send_requested, message.sent, message.send_failed

approval_requests​

First-class approval primitive, not embedded in task JSON.

Columns:

  • id uuid pk, org_id, run_id with composite org FK
  • optional refs: conversation_id, expert_queue_item_id, tool_call_id, message_id
  • request_kind text not null: expert_send_approval | expert_tool_approval | client_approval | integration_write_approval | high_risk_response_approval | sandbox_capability_approval | missing_context
  • status text not null: pending | approved | rejected | cancelled | expired
  • risk_level text not null: low | medium | high | critical
  • policy_decision_id uuid null
  • payload jsonb not null redacted
  • requested_by_type, requested_by_id
  • assigned_expert_id uuid null, resolved_by_user_id uuid null
  • resolution_note text null
  • resolved_at, expires_at, created_at

Resolution authorization matrix:

  • Experts may resolve approvals assigned to them or to orgs where they hold an expert_access grant via ExpertAccessService (ADR-007 supersession; the legacy org_experts table is being removed — see #540 / #999).
  • Client users may resolve only client_approval/allowed client input in their own org.
  • AMs may resolve/view only assigned orgs unless a product rule explicitly gives read-only broader access.
  • SuperAdmin may resolve/view all.
  • AI operators/internal roles require explicit service filters; current “internal” RLS bypass must not imply all-org access.
  • Resolution must join approval_requests → agent_runs → org in the same transaction and append approval.resolved, update counters, and resume/block through RunOrchestrationService exactly once.

Indexes: pending queues by (org_id, status, request_kind, created_at), (assigned_expert_id, status, created_at), (run_id, status).

user_input_requests​

Separate missing-context row model.

Columns mirror approvals plus:

  • request_kind
  • prompt text
  • schema jsonb
  • answers jsonb
  • target_actor_type: expert | client_user | ops_user

Indexes mirror approval pending queues.

tool_calls​

Durable normalized call records independent of event payloads.

Columns:

  • id uuid pk, org_id, run_id, conversation_id null
  • tool_name, action, tool_version, provider_kind, integration_type
  • status: requested | blocked | waiting_approval | running | succeeded | failed
  • risk_level, policy_decision_id, approval_request_id null
  • input_schema_version, input_redacted jsonb, input_artifact_id null
  • output_redacted jsonb, output_artifact_id null
  • external_idempotency_key text null, provider_request_id text null, provider_response_id text null
  • correlation_id text not null
  • resume_token_hash text null
  • error_code, error_text, duration_ms
  • timestamps

Side-effect idempotency: for writes/sends/submits, tool_calls.id or external_idempotency_key must be used as the external idempotency key where providers support it. Retry-after-timeout must not duplicate external actions. Store masked provider request/response IDs.

Indexes: (org_id, run_id, status), (org_id, tool_name, status, created_at), unique provider/idempotency keys where applicable.

policy_decisions​

Structured policy evaluation record.

Columns:

  • id uuid pk, org_id, run_id, tool_call_id null
  • action_type text, resource_type text, resource_id text null
  • decision text not null: allow | observe | require_expert_approval | require_client_approval | block
  • mode text not null: observe | enforce
  • risk_level text not null
  • matched_rules jsonb not null
  • reason text not null
  • policy_hash text not null
  • created_at

run_artifacts​

Runtime evidence and data/file handoff records. R2 is production storage; DB holds small redacted snippets and metadata only.

Columns:

  • id uuid pk, org_id, run_id
  • optional refs: tool_call_id, approval_request_id, validation_run_id, message_id
  • kind text not null: prompt_snapshot | model_transcript | provider_raw_log | tool_input_redacted | tool_output_redacted | file_input | file_output | browser_screenshot | browser_trace | validation_log | sandbox_log | export_packet | legacy_agentic_task_snapshot
  • label text
  • storage_provider text: r2 | local_dev | db_small_blob
  • storage_key text null (no persisted signed/public URL)
  • content_type, byte_size, sha256
  • redaction_level text not null
  • retention_policy text not null
  • metadata jsonb not null
  • created_at

Artifact access generates short-TTL signed URLs after auth and logs artifact.accessed. Never store long-lived signed/public URLs in DB.

Isolation rule: raw provider logs, model transcripts with PII, browser traces/screenshots, sandbox logs, file inputs/outputs, and client documents require either bucket-level isolation or documented compensating controls plus strict retention. Prefix-only isolation is acceptable only if explicitly documented for that data class and tested; do not claim #625 closed unless bucket-level isolation ships.

Indexes: (org_id, run_id, kind, created_at), (org_id, kind, created_at), (sha256) where useful.

runner_manifests​

Compiled runner/provenance/policy manifest.

Columns:

  • id uuid pk, org_id uuid not null, specialist_id uuid null
  • runner_hash text not null
  • policy_hash text not null
  • tool_manifest_hash text not null
  • prompt_bundle_hash text null
  • event_schema_version text not null
  • allowed_tools text[], allowed_ops text[], sandbox_modes text[]
  • network_policy text
  • environment text
  • runner_version text
  • expires_at timestamptz
  • signature text not null
  • signing_key_id text not null
  • manifest jsonb not null
  • status text not null: active | disabled | revoked | expired
  • created_at

Manifest signing is not optional: require key custody, rotation, revocation, issuance audit, expiry, and a kill switch to disable a runner hash/signing key immediately.

runner_registrations​

Live runner heartbeat records. This replaces shallow agent_runtimes as dispatch authority.

Columns:

  • id uuid pk, org_id uuid not null, specialist_id uuid null
  • runner_manifest_id uuid not null references runner_manifests(id)
  • runtime_kind text not null: initial hermes
  • endpoint_ref text null preferred: service discovery identity / queue name / private service id
  • internal_url text null only if validated, never arbitrary DB-provided URL
  • external_runtime_id text null
  • status text not null: registered | ready | draining | unhealthy | disabled
  • last_heartbeat_at timestamptz null
  • last_error text null
  • metadata jsonb not null default '{}'::jsonb
  • timestamps

Dispatch/SSRF rule: prefer outbound runner polling or service-discovery IDs. If the API calls a runner URL, it must enforce protocol allowlist, private/service-network allowlist, loopback/link-local/metadata IP denylist, public internet deny by default, mTLS/service identity, DNS pinning/re-resolution protections, environment matching, and no redirects to unapproved hosts. Signatures on callbacks do not make an endpoint safe to call.

Quarantine/unresolved ingress tables​

Add a minimal table or queue for unresolved channel traffic, e.g. unresolved_channel_events:

  • external channel identifiers and dedupe key
  • raw payload stored redacted/artifact-ref only
  • reason: org_resolution_required | credential_mapping_missing | signature_failed | malformed
  • status: pending | resolved | ignored | expired
  • no agent_runs row and no Specialist runtime execution until org resolution succeeds

Existing org-less conversations/messages stay historical/remediation only. Do not backfill them into a fake/system org unless product/security explicitly approves a quarantine org with no runtime/tool access.

Existing tables: keep, demote, or delete​

  • agentic_tasks: keep only as product request/traceability if needed. Add run_id; remove lifecycle authority and legacy statuses. Mutating old endpoints must be 410/replaced.
  • agent_runtimes: migrate into runner_manifests/runner_registrations, then stop using as dispatch source.
  • actions: keep only for manually authored Expert/system actions if still needed. Static/runtime guard: no runtime lifecycle/tool execution writes/reads actions after cutover.
  • tool_permissions: keep as org-level policy input, not a whole policy system.
  • audit_log: keep compliance audit; runtime timeline is run_events. Audit rows may reference event ids. Redact before audit writes.

RLS/session/worker strategy​

RLS is useful but not magic here. Current internal roles and async workers mean service filters and transaction context are load-bearing.

  • HTTP requests: run DB work that relies on RLS inside explicit transactions that set app.current_org_id, app.current_user_id, and role context with SET LOCAL/set_config(..., true). Do not rely on session state outside the transaction.
  • PgBouncer/transaction pooling (#450): assume session variables do not persist. All RLS context must be transaction-local. Add integration tests under transaction-pooling-like behavior or a documented equivalent.
  • Webhooks using setImmediate: do not inherit request RLS context. Verify webhook signature, resolve org before persistence/runtime, pass explicit org/user/system context to a job/transaction wrapper that sets set_config manually.
  • Workers/queues/backfills/runner callbacks: require an explicit OrgDbContext helper that wraps each unit of work in a transaction and sets org context. Backfills must process per-org chunks or explicitly mark quarantine rows.
  • Internal roles: because existing RLS treats superadmin, expert, account_manager, and ai_operator as internal, every Ops/workspace service must apply AM org-assignment and Expert org-assignment filters explicitly; tests must prove unrelated-org events/artifacts/approvals are hidden.
  • SQLite tests: permitted for pure service unit tests only. RLS/migration/index/trigger tests must run on Postgres.

Core services/modules​

RunOrchestrationService (api/src/runs or api/src/agent-runs)​

Responsibilities:

  • create runs for conversation drafts, Expert side-chat, agentic tasks, tool-only actions, validation, browser automation
  • precompute run hashes before inserting authoritative run rows
  • own legal state transition table
  • enqueue/resume/cancel runs
  • select runner registration/manifest
  • emit durable events through RunEventLedgerService
  • maintain pending approval/input counters
  • expose read models for Ops/workspace/client surfaces
  • implement fail-open escalation for runner/agent failures

Invariant: no other service updates agent_runs.status directly.

Fail-open/degraded behavior:

  • If DB is available but runner selection, manifest signing, tool manifest generation, model gateway, or Hermes start fails: create/preserve agent_runs, append run.failed, create/preserve Expert queue item/draft safe escalation, notify Ops/Expert, and return a safe client-visible response without surfacing a raw 500.
  • If runner registry is empty/unhealthy for an org: same safe failed-run/escalation path.
  • If DB persistence is unavailable: do not pretend fail-open; return retryable webhook status where channel semantics support retry, or service-unavailable for synchronous requests. If the inbound message cannot be durably captured, do not acknowledge as processed.

RunEventLedgerService​

Responsibilities:

  • append events transactionally with row-lock sequence allocation
  • enforce idempotency in Postgres, not memory
  • redact before event write, audit write, artifact metadata write, and broadcast
  • link compliance audit rows where required
  • broadcast WebSocket/SSE/notification updates only after durable commit
  • provide authorized timelines for Ops/workspace/client

This replaces split AgentApiService.handleEvent and AgentEventsService authority paths.

RuntimeAdapterService​

Provider-neutral adapter interface:

interface RuntimeAdapter {
  startRun(ctx: RuntimeStartContext): Promise<RuntimeStartResult>;
  sendInput(runId: string, input: RuntimeInput): Promise<void>;
  approve(runId: string, approvalId: string, decision: ApprovalDecision): Promise<void>;
  resumeTool(runId: string, toolCallId: string, resume: ToolResumePayload): Promise<void>;
  cancelRun(runId: string): Promise<void>;
}

Initial adapter: Hermes/Python. Replace legacy public production contracts with signed runner RPC or internal /v2/runs/:id/start style API. The Nest caller creates the run and passes a short-lived run token.

Async approval/tool continuation:

  • Tool request creates tool_calls and either executes or enters waiting_approval with a hashed resume token/correlation id.
  • Runner must either block on a run-scoped wait API, yield a checkpoint, or exit cleanly after checkpointing. The choice must be uniform and contract-tested.
  • Approval resolution resumes by calling RuntimeAdapter.resumeTool(runId, toolCallId, ...) with the same correlation/idempotency chain. If the runner has exited, it must be rehydrated from checkpoint artifact/run state.
  • External side effects happen exactly once via tool-call idempotency. Approval replay must not rerun writes.

RunnerRegistryService​

Responsibilities:

  • create/validate signed runner manifests
  • register/heartbeat runners
  • reject stale/expired/wrong-org/wrong-env/wrong-hash runners
  • select runner for a run using org, optional Specialist, environment, allowed tools/ops, policy hash, tool manifest hash
  • issue short-lived run-scoped tokens
  • enforce endpoint/service discovery constraints
  • audit manifest issuance and runner selection

Reject:

  • unknown/revoked runner hash or signing key
  • mismatched policy/tool/prompt hash
  • wrong org/environment
  • expired manifest
  • unsupported operation/tool
  • unsigned/invalid event or tool callbacks
  • arbitrary/public/metadata/loopback runner endpoints

ModelGatewayService / credential broker​

Add explicitly; otherwise “no provider secrets in runner env” is ambiguous.

Responsibilities:

  • own provider API keys/secrets in the control plane or credential broker
  • accept run-scoped model requests from Hermes/runner
  • enforce org/model policy, rate limits, cost accounting, redaction, and event/artifact capture
  • return provider responses/events to runner without exposing broad keys

If a provider truly requires runner-side credentials, use a short-lived, run-scoped, non-env credential lease with expiry/audience binding and no R2/platform/vendor integration scope. This is an exception requiring security approval; default is platform-brokered model access.

ActionPolicyService​

Human.work equivalent of Rig’s guard.

Inputs:

  • actor and role/domain
  • org/conversation/Specialist/run
  • action type: message_send, tool_call, integration_read, integration_write, data_export, credential_access, artifact_read, artifact_write, exec_argv, browser_action, channel_send, model_call
  • resource ids/scopes
  • tool descriptor/manifests
  • org tool permissions/config
  • integration credential presence
  • risk classifier result
  • run mode and environment

Output:

{
  decision: 'allow' | 'observe' | 'require_expert_approval' | 'require_client_approval' | 'block',
  mode: 'observe' | 'enforce',
  riskLevel: 'low' | 'medium' | 'high' | 'critical',
  reason: string,
  matchedRules: Array<{ id: string; category: string; reason: string }>,
  requiredApprovalKind?: string,
  redactionPolicy: Record<string, unknown>,
  auditTags: string[]
}

Initial rules:

  • block cross-org resource access
  • preserve default HITL/no-autosend: message_send/channel_send requires Expert approval unless the org/Specialist assignment explicitly configures auto-send and policy permits it; default threshold 101 means never auto-send
  • require Expert approval for high-risk/low-confidence external sends
  • require client approval for irreversible client-account actions
  • require approval for integration writes and sandbox write/submit/browser actions
  • block raw credential access by runner
  • block unredacted PII/secrets in events/artifacts/audit/broadcast
  • block arbitrary direct HTTP except allowed proxy profiles
  • block raw shell command strings
  • enforce Nango/Jumio/vendor environment tier correctness
  • observe new tools only until explicitly enforced
  • block KB/Haystack access in run/tool manifests until #541/#291/#300 data scopes are decided and tested

ToolGatewayService​

Single canonical business tool authority in Nest/API.

Target:

  • one TypeScript registry with descriptor, JSON schema, risk metadata, credential requirements, redaction policy, and handler
  • Python receives generated signed tool manifest, not independent specs
  • every request enters ToolGatewayService.execute(runId, toolName, action, params, context)
  • pipeline: schema validate → org/resource scope check → org tool permission/config check → policy decision → approval if required → credential proxy/resolution → side-effect idempotency → handler → redaction → artifacts/events/audit/metrics
  • unknown tool/missing handler fails closed
  • stubs exist only in explicit tests/dev fakes
  • normalized result: { status, data, redactions, artifacts, externalIdsMasked, error }

Credentialed tools execute in the control plane or credential broker. Runner never receives Nango/Jumio/Shopify/Amazon/vendor integration secrets.

ApprovalService and InputRequestService​

Responsibilities:

  • create request rows
  • emit approval.requested / input.requested
  • pause run via RunOrchestrationService
  • resolve rows idempotently with authorization matrix
  • emit resolution events
  • resume or block runs durably
  • notify workspace/client/Ops surfaces

Approvals must be rows, not fields in task step JSON.

ArtifactService​

Responsibilities:

  • R2/local storage abstraction
  • artifact row creation
  • redaction enforcement
  • signed access URL generation on read only
  • artifact metadata secret scanning
  • retention/TTL enforcement
  • link artifacts to runs/tool calls/approvals/messages

Use org-scoped storage isolation. If separate R2 buckets (#625) are not done immediately, signed-prefix isolation needs explicit compensating controls and data-class limits.

CapabilityGatewayService + sandbox worker​

Responsibilities:

  • accept typed capability requests from runner
  • run policy evaluation
  • create capability/tool/policy events
  • dispatch allowed operations to sandbox worker
  • store outputs/logs/screenshots as artifacts
  • enforce worker leases/heartbeats/dead-letter behavior

Supported initial ops:

  • artifact_read
  • artifact_write
  • artifact_list
  • exec_argv with executable allowlist and argv array only
  • http_request through named control-plane proxy profiles only
  • browser_step with domain allowlist, screenshot/trace artifacts, and approval gates for submit/write actions

Envelope:

{
  "id": "op_...",
  "orgId": "...",
  "runId": "...",
  "correlationId": "...",
  "op": "exec_argv",
  "args": {},
  "policyDecisionId": "...",
  "timeoutMs": 30000,
  "maxOutputBytes": 262144,
  "network": "off"
}

Redact argv/stdout/stderr before events/artifacts. If sandbox worker is unavailable, fail closed: create failed capability/tool state, event, and approval/escalation if appropriate; never execute in the API process as fallback.

ValidationService​

Rig-inspired validators, implemented Human.work-natively.

Initial validators:

  • API registry ↔ generated runner/Python tool manifest drift
  • org-scope/IDOR checks for run/event/approval/input/tool/artifact endpoints
  • internal-role assignment filtering
  • event/artifact/audit/broadcast redaction checks
  • no raw secrets in runner env/manifests/artifacts/deployment assets
  • no legacy endpoints reachable in production except 410/read-only traceability as approved
  • terminology check: Specialist/Expert/client/org
  • Nango/Jumio/vendor credential tier validation
  • sandbox denial suite
  • test integrity guard (.only, .skip, focused tests)
  • migration/RLS/index/trigger inspection

Runtime flow​

  1. API receives product request: client message, Expert side-chat, ops task, tool action, validation run.
  2. Resolve org before runtime. If org cannot be resolved, write unresolved/quarantine event only; no Specialist run.
  3. Allocate run id and preparation context; compute policy hash, tool manifest hash, and prompt bundle hash before inserting run.
  4. Insert agent_runs and run.created in a transaction.
  5. Store prompt/tool/policy snapshots as artifacts where applicable.
  6. RunnerRegistryService selects matching runner manifest/registration.
  7. API signs short-lived token scoped to { orgId, runId, runnerHash, allowedOps, expiresAt }.
  8. Runner starts and emits signed events: runner.started, runner.manifest.loaded, runner.policy.loaded.
  9. Model/provider calls go through ModelGatewayService/credential broker; durable events are coalesced/redacted.
  10. Tool requests call ToolGatewayService; capability requests call CapabilityGatewayService.
  11. Approval/input requirements create first-class rows, set generic waiting_approval/waiting_input, notify UI, and resume/block only through services.
  12. Completion/failure/cancel writes final run status and terminal events.

Compiled runner, capability shell, sandbox​

Compiled runner​

Use a compiled runner/capability shell for provenance and attack-surface reduction, not as the security boundary. It must be paired with policy, scoped tokens, no ambient secrets, signed callbacks, and OS/container sandboxing.

Runner requirements:

  • manifest-bound allowed tools/ops
  • runner hash/signature validation
  • signed manifest with key rotation/revocation/expiry
  • short-lived run token only
  • no vendor/API platform/R2/integration secrets in env
  • no direct credential access
  • no raw shell interface
  • signed event/action callbacks
  • heartbeat and health reporting

Capability shell​

Implementation shape can be Zig/native or similarly small/typed. Required behavior:

  • JSON/typed protocol only
  • path normalization and allowlist enforcement
  • env scrub
  • executable allowlist
  • argv-only exec_argv, no shell interpolation
  • output byte caps
  • timeout/process-tree cleanup
  • structured events for allowed/denied/failed ops

OS/container sandbox​

For file/process/browser work, run in a worker substrate that can enforce:

  • read-only rootfs
  • bind mounts only for run workspace/input/output/tmp/logs
  • no host credential dirs
  • no Docker socket
  • seccomp/namespaces/cgroups/no-new-privileges/dropped capabilities
  • network default off or proxy-only
  • memory/CPU/PID/output/time limits
  • denial events such as sandbox.exec.denied
  • workspace TTL deletion and orphan process reaper

If Railway cannot provide these primitives, run sandbox workers on ECS/Fargate/Kubernetes/Firecracker-equivalent. Do not fake sandboxing in the API process.

Ops/workspace/client UI​

Do not port Rig desktop/TUI. Build native Human.work surfaces under existing route rules (/ops, /workspace, /client; no /am, /superadmin, /portal).

/ops/agent-runs​

List/filter:

  • active/stuck/failed runs
  • org/client/Specialist/run kind/status
  • runner hash, policy hash, tool manifest hash
  • pending approvals/inputs
  • last event/error
  • duration/SLA

Service filters must enforce AM org assignment and Expert assignment rules; do not rely on internal RLS alone.

/ops/agent-runs/:id​

Detail view:

  • full timeline from run_events
  • model transcript/artifacts
  • tool calls and policy decisions
  • approvals/input history
  • sandbox/capability denials
  • runner manifest/provenance
  • cancel/resume/retry controls gated to SuperAdmin/AM/Ops roles

/ops/approvals​

Ops-wide approval inbox filtered by org, risk, kind, status. Experts get workspace-scoped views; AM/SuperAdmin get Ops views subject to assignment rules.

/workspace/tasks replacement​

Replace JSON-plan step approvals with:

  • pending approval list from approval_requests
  • assigned Expert/org scope
  • run timeline preview
  • approve/reject/comment actions
  • client approval handoff where applicable
  • no direct mutation of task plan JSON

/client/approvals or cards in /client/chat​

For high-stakes client approval:

  • resolve approval_requests with request_kind=client_approval
  • show consequence text, requested action, expiration, and safe reject option
  • append approval.resolved and resume/block through service

Tool/integration/runtime health​

Ops pages/widgets for:

  • tool registry health
  • integration credential status (Nango/Jumio/etc.)
  • validation run history
  • denied tool/capability attempts
  • runner registration heartbeats
  • artifact storage/redaction status

Issue disposition matrix​

IssueDisposition in this port
#611 Agent control planeUmbrella: fully addressed by run orchestration, event ledger, policy, tool gateway, approvals, runner registry, sandbox, Ops inspector.
#362 Per-org sandboxed agent runtime MVPAddressed by manifest-backed per-org runners, sandbox workers, no shared ambient runtime, no runner vendor secrets.
#401 Per-(Org × Specialist) sandboxed runtimes v2Addressed structurally with optional specialist_id on manifests/registrations/runs. Implementation should support Org×Specialist now; default org-level only by explicit decision.
#332 Compiled per-org policy enforcementAddressed as compiled runner + signed manifests/policy/tool hashes + capability shell + OS sandbox + no ambient secrets. Binary alone is not security boundary.
#251 Per-client-agent auth secretAddressed by replacing global broad tokens with per-run/per-runner scoped signed tokens and manifest registration.
#226 Skill bind-mount read-only contractAddressed by runner filesystem contract: read-only skills/config, writes only to approved workspace/output/tmp/log/artifact areas.
#237 Cross-org isolation eval nightlyExpanded to runtime tables/services/artifacts/sandbox/control-plane endpoints and internal-role filters.
#176 Domain tools & integrationsAddressed at architecture level by unified gateway/registry/policy. Provider implementation may remain individual work, but no fake success paths.
#230 Jumio bespoke integrationFolded into canonical gateway/policy/event/artifact path; existing service becomes handler behind gateway.
#511 Agent service HTTP rate limitingAbsorbed by private runner contracts, scoped tokens, per-org/run/user limits, model gateway/tool/artifact/sandbox limit tests.
#172 Agentic executionReplaced by run/approval/tool authority; agentic tasks become product requests, not lifecycle state.
#29 fake execute_node/tool stubsAddressed by fail-closed gateway and explicit test-only fakes.
#248 streamingAddressed by ephemeral streaming plus durable coalesced chunks/checkpoints/artifact transcripts; no one-off streaming authority path.
#166/#156 R2/file artifactsAddressed by run_artifacts + R2 storage + signed access/redaction/retention.
#168 Expert workspace redesign/task detailAddressed by approval inbox and run timeline replacing JSON task cards.
#170 notificationsApproval/run events become notification triggers.
#173 infra hardening/NetworkPolicyPartially addressed by sandbox network policy, no env secrets, runner heartbeat, structured logs; broader infra epic remains.
#625 Separate R2 bucketsAffected. Do not close unless bucket isolation ships; prefix-only requires compensating-control doc and data-class limits.
#608 Agent down fail-open pathMust be preserved/tested with failed run + Expert queue safe escalation when DB is available.
#609 DB migration smoke testRequired validation for schema/cutover with synchronize disabled.
#606/#610 coverage/fixturesHelpful test hardening but not core architecture.
#300/#291 Haystack isolation/provisioningBlocks KB/Haystack run/tool manifest access until scopes are decided/tested.
#541 KB architecture decisionImplementation-lock blocker if any run gets KB/Haystack access.
#614/#615/#619 epicsAdvanced; not closed wholesale.
#507 Slack/Telegram event dedupPattern reusable via durable idempotency; close only if channel dedup is implemented.
#165 dead code cleanupAddressed by deletion/410 of legacy runtime paths at cutover.
#417 AI Draft schema lifecycleIncluded/affected: draft creation and Expert approval must link messages/expert queue to agent_runs; do not close unless lifecycle schema is explicitly reconciled.
#613 ChannelsAffected: channel send/receive, message_send policy, artifacts, ack/retry behavior must be compatible with new run flow.
#559 outbound mediaAffected/out-of-scope unless media artifacts/send policy are implemented. Must not be broken by channel_send changes.
#585 acknowledgement behaviorIncluded in cutover/ingest semantics: do not ack webhooks as processed unless durable capture succeeds.
#587 Google Workspace per Specialist × ClientAffected by Org×Specialist credential/runtime model; do not close unless scoped credential handling is implemented.
#558 Expert queue SLAAffected/included for Ops run duration/SLA surfacing; do not close unless SLA behavior ships.
#505 mock HP credentialsIncluded in “no fake success in prod” guards.
#450 PgBouncer/connection poolingIncluded as RLS/session-context validation requirement.

Implementation sequencing inside one hard-cut release​

This is branch/staging sequencing, not production dual-write or compatibility.

Milestone 0 — immediate safety hotfixes​

  • org-scope or production-disable /agentic/tasks* read/list/approve/reject/SSE authority
  • fail closed for unknown/unimplemented tools outside explicit tests
  • add static guard for no production stubs/mock HP credential success

Milestone 1 — schema/RLS/core ledger​

  • add migrations/entities for runtime tables, indexes, composite org keys/FKs/triggers
  • add Postgres migration-only CI with SYNCHRONIZE=false
  • implement RLS/session context helper for HTTP/workers/webhooks/backfills/callbacks
  • implement RunOrchestrationService and RunEventLedgerService with row-lock sequencing

Milestone 2 — policy/tool/model gateways​

  • implement ActionPolicyService, including default HITL send policy
  • consolidate TypeScript tool registry and convert Nango/Jumio/Shopify/Amazon handlers
  • delete/410 direct /tools/execute and /v1/agent-api/tools/:toolName authority or rewrite as internal run-scoped wrappers
  • implement ModelGatewayService/credential broker
  • generate signed runner tool manifest and contract tests

Milestone 3 — approvals/input/artifacts​

  • implement first-class services/controllers and authorization matrix
  • implement idempotent resume/block semantics
  • wire Expert/client/Ops notifications
  • implement ArtifactService, redaction, signed URLs on read, retention, secret scanning

Milestone 4 — conversation/task/channel wiring​

  • every org-resolved Specialist turn creates agent_runs
  • Expert queue rows/messages link to run ids and preserve always-HITL default
  • side-chat/task/tool-only flows create runs
  • unresolved channel traffic goes to quarantine/remediation, no runtime execution
  • existing fail-open-to-Expert behavior preserved as run failure/escalation

Milestone 5 — runner registry + Hermes adapter​

  • manifest generation/storage/signing/key rotation/revocation
  • runner registration/heartbeat with endpoint/service identity constraints
  • scoped run token issuance
  • Hermes starts from platform run contract, emits signed events, calls model/tool/capability gateways only
  • remove direct provider/vendor/R2/platform secrets from runner deployment assets

Milestone 6 — capability shell + sandbox worker​

  • implement typed protocol and operation schemas
  • env scrub/path allowlists/argv-only exec/output/time caps
  • worker leases/heartbeats/dead-letter/TTL cleanup/orphan reaper
  • wrap in real OS/container sandbox
  • add denial artifacts/events

Milestone 7 — UIs/read models​

  • /ops/agent-runs, detail timeline, /ops/approvals, health views
  • replace /workspace/tasks JSON plan approvals
  • client approval cards/route
  • queue/detail links to run timelines/artifacts
  • assignment filters for AM/Expert/internal roles

Milestone 8 — validators/cutover rehearsal​

  • static guards for removed env/routes/statuses/direct handlers/actions writes
  • runtime validators for manifests, secrets, redaction, sandbox denies, cross-org isolation
  • production assertions that legacy routes are 410/disabled
  • load/capacity rehearsal for run_events/tool_calls/artifacts and rate limits
  • at least one staging rehearsal with channel freeze/drain/backfill/reopen

Milestone 9 — atomic cutover​

  • execute maintenance window protocol below
  • deploy migrations/API/frontend/agent/runner/sandbox workers together
  • no old execution semantics behind feature flags; kill switches may pause new runs or disable specific new runner hashes/tools only

Migration/cutover protocol​

Pre-cutover branch work​

  • Implement all schema/services/controllers/UI/runner changes off production.
  • Write explicit migrations and down/rollback notes.
  • Build dry-run backfill scripts and rehearse in staging with production-like snapshot.
  • Build validators proving no legacy runtime route/path is reachable in production mode except approved 410/read-only traceability.
  • Prepare runner image, manifest signing keys, registration bootstrap, and kill switches.
  • Prepare DB backup/restore runbook and forward-fix containment runbook.
  • Define cutover high-water marks: per channel external event id/timestamp, DB message/conversation ids, queue/job ids, and runner/task ids.

Maintenance window freeze/drain/backfill​

  1. Announce/enter maintenance window.
  2. Freeze inbound channel processing for Slack, Email, Telegram, WhatsApp, Teams, and any channel workers. Prefer durable queueing at edge/provider where possible; otherwise accept webhooks only into durable raw/quarantine queue without runtime execution.
  3. Pause outbound sends/tool writes except approved maintenance-safe operations.
  4. Pause worker queues and in-process async callbacks; drain or terminate with replay-safe idempotency. setImmediate channel work must be drained before high-water snapshot.
  5. Snapshot high-water marks for channel events, messages/conversations, expert queue, agentic tasks, actions, and old event handlers.
  6. Take DB backup.
  7. Run migrations.
  8. Backfill to the fixed boundary only:
    • create agent_runs for active/recent org-resolved conversations and non-terminal agentic_tasks where traceability matters
    • create pending approval/input rows from non-terminal state
    • store original task JSON as legacy_agentic_task_snapshot artifact and legacy.imported event
    • transform valid agent_runtimes into signed runner_manifests/runner_registrations; reject shared/global fallback rows that cannot validate
    • seed policy/tool manifest hashes per org/Specialist
    • quarantine org-less/unresolved channel records; no runs
  9. Verify RLS policies, org consistency triggers/FKs, core indexes, route 410s, manifest signatures, runner registrations, and static guards.
  10. Bootstrap runners and sandbox workers; require ready heartbeats before reopen.
  11. Go/no-go checklist:
    • migration smoke passed with synchronize disabled
    • legacy route guard passed
    • at least one runner ready per enabled org/Specialist/default
    • model/tool/artifact/sandbox gateways healthy
    • fail-open escalation test passed
    • channel replay/dedupe checks passed
    • Ops visibility works for sample run
  12. Reopen channels from high-water mark, replay queued inbound events through new idempotency, resume workers/outbound sends.

Rollback and containment​

Rollback after new runs exist is intentionally limited. Say that plainly; do not pretend dual-write rollback exists.

  • If failure happens before new runs/events are created: restore DB backup and redeploy old stack.
  • After first new run/event: prefer forward fix. Do not reintroduce legacy execution bridge.
  • Containment modes:
    • global new_runs_paused kill switch: accept/durably capture inbound messages but do not start runners; create Expert-visible escalation where possible
    • per-tool/per-runner hash disable
    • outbound-send pause preserving draft/approval state
    • read-only Ops/workspace inspection mode
    • export newly written run/events/artifacts for manual remediation
  • Define rollback decision deadline during maintenance window. If deadline passes with new runs created, move to containment/forward-fix runbook.
  • Customer impact target: no lost inbound messages; no duplicate external sends/tool writes; safe Expert escalation for runtime failures.

Validation/test plan​

DB/RLS/migration​

  • clean Postgres migration smoke with SYNCHRONIZE=false
  • down migration or documented irreversible points
  • RLS policy inspection for all new org-scoped tables
  • composite FK/trigger tests rejecting mismatched org refs
  • cross-org read/write denial tests for run/events/approval/input/tool/artifact endpoints
  • AM/Expert/internal role assignment filtering tests
  • concurrent event append test with distinct callbacks racing for next sequence
  • repeated callback across restart/replica simulation produces exactly one row per (run_id, source, idempotency_key)
  • PgBouncer/transaction-pooling RLS-context behavior test or equivalent

API service tests​

  • run state transition table accepts/rejects correctly
  • generic waiting_approval/waiting_input statuses reflect counters/details
  • approval resolution resumes/blocks exactly once with role matrix
  • input resolution resumes exactly once
  • tool gateway: unknown tool, disabled tool, missing handler, missing credential, policy approval, policy block, success path
  • retry-after-timeout for write/send/submit tools does not duplicate external side effect
  • default HITL: message/channel send requires Expert approval unless explicit auto-send config + policy allow
  • fail-open: runner registry empty/manifest failure/model failure creates failed run + Expert queue safe escalation
  • DB unavailable: webhook/sync behavior is retryable/fail-closed, not fake success
  • redaction: no raw secrets/PII in run_events, audit_log, artifact metadata, WebSocket/SSE payloads
  • runner registry: wrong org/hash/env/expired manifest/revoked signing key/bad endpoint rejected

Agent/runner tests​

  • Hermes adapter starts from platform run contract
  • emits signed model.started, coalesced model.chunk, model.completed/model.failed
  • model requests go through model gateway/credential broker; no broad provider env vars required
  • tool request cannot execute locally; must call gateway
  • local task checkpoint is not platform source of truth
  • signed callbacks required and bound to run/source/payload hash
  • manifest/tool drift contract tests

Sandbox/capability tests​

  • cannot read another org workspace
  • cannot read host secrets (/proc/self/environ, home creds, service env files)
  • cannot write outside allowed dirs
  • raw shell string unsupported
  • network-off blocks egress
  • proxy-only blocks arbitrary domains
  • timeout kills process tree
  • output/memory/PID caps enforced
  • worker unavailable fails closed, not API-process fallback
  • leases/heartbeat/dead-letter/TTL cleanup tested
  • argv/stdout/stderr redaction before events/artifacts
  • denials emit sandbox.exec.denied and artifacts/events as appropriate

Frontend/E2E​

  • Expert approves/rejects pending send/tool approval and run resumes/fails
  • client approval card resolves approval and timeline updates
  • Ops run detail shows timeline, tool calls, policy decisions, artifacts, runner hashes
  • internal users cannot see unrelated org run artifacts/events unless role explicitly permits
  • failed tool call appears with normalized error and no secret leakage
  • conversation draft routes to Expert review by default/threshold 101
  • unresolved channel message appears in remediation/quarantine, not as a running Specialist turn

CI/static guards​

Target commands:

cd api && npm run build && npm test -- --runInBand
cd agent && python3 -m pytest tests/ evals/ -q
cd frontend && npm run build && npm run lint

Add static checks for:

  • no LEGACY_AGENT_API usage
  • no production /tools/execute
  • no direct production /v1/agent-api/tools/:toolName tool execution outside run-scoped gateway wrapper
  • no authoritative /agentic/tasks* mutation/SSE old state machine in production
  • no Python direct SHOPIFY_ACCESS_TOKEN / AMAZON_REFRESH_TOKEN runtime handlers
  • no broad PLATFORM_API_TOKEN tool proxy
  • no broad provider/R2/platform secrets in runner/sandbox deployment assets
  • no legacy runtime statuses as active run statuses
  • no runtime writes/reads to actions for lifecycle/tool execution
  • no raw cmd operation in capability API
  • no persisted signed/public artifact URLs
  • no event/artifact/audit/broadcast payload keys matching secret patterns
  • no /am, /superadmin, /portal routes added
  • no wrong product terminology in product-facing UI/docs/comments

Risks and hard calls​

  • Breaking internal API cut: intentional. One authority is safer than bridge soup.
  • Current /agentic/tasks* IDOR: cannot wait; hotfix/disable before full cutover if production reachable.
  • Sandbox substrate may not fit Railway: do not water it down. Use ECS/Fargate/K8s/microVM-style worker if needed.
  • Historical data will not fully replay: keep old audit as historical record; import active work and store legacy snapshots.
  • Org-less channel history: quarantine/remediate; do not execute runtime against null org.
  • Artifact redaction is load-bearing: rich artifacts are dangerous without redaction/signed access/secret scans/retention.
  • Tool centralization breaks fake demos: good. Use explicit test fakes only.
  • Org×Specialist runtime key is bigger but cleaner: schema must support it now; default requires explicit decision.
  • R2 bucket isolation may be separate: prefix isolation is not equivalent; document data-class limits.
  • Compiled runner can become security theater: do not claim it is enough without OS/container sandbox and no ambient secrets.
  • Hard rollback after new runs is unavailable: use pause/read-only/forward-fix containment, not legacy bridge reanimation.

Non-goals​

  • Do not port old Splinter wholesale.
  • Do not port Rig desktop/TUI/native app/server as Human.work UI.
  • Do not replace Human.work product domain or terminology.
  • Do not build client Specialist workflows around git worktrees/raw file tools.
  • Do not preserve legacy runtime/tool/task endpoints as compatibility surfaces.
  • Do not add /am/*, /superadmin/*, or /portal/* routes.
  • Do not solve all Haystack/KB isolation unless explicitly included.
  • Do not claim all infra/security/testing epics are closed just because runtime control plane improved.

Implementation-lock decisions​

These must be decided before implementation lock / production cutover:

  1. Runtime key: recommendation is support (org_id, specialist_id) now; allow org-level default only by explicit product/ops decision.
  2. Sandbox substrate: ECS/Fargate hardening, Kubernetes, Firecracker/microVM provider, or another worker platform. Railway/shared containers are not enough for documented sandbox goals.
  3. Artifact storage isolation: implement separate R2 buckets (#625) in MVP or use signed-prefix isolation with explicit compensating controls, data-class limits, and retention.
  4. Manifest signing/key rotation: required, not optional. Decide custody, rotation cadence, revocation path, and emergency disable workflow.
  5. packages/agent-harness: either wire it under RunOrchestrationService/RunEventLedgerService with DB-backed state, or retire it from production imports. It cannot keep in-memory/JSONL session state, approval policy, or tool registry as a second authority.
  6. KB/Haystack ACL model (#541/#291/#300): no run manifest/tool manifest may include KB/Haystack access until Org×Specialist data scopes are decided and tested.
  7. Model credential path: default is ModelGatewayService/credential broker. Any runner-side ephemeral provider lease needs security approval and tests proving no broad env secrets.

Definition of done​

  • Every org-resolved Specialist/model execution creates an agent_runs row.
  • Org-less/unresolved channel traffic is quarantined/remediated and never starts runtime execution.
  • Every run has ordered durable run_events with row-lock sequence allocation and DB idempotency.
  • Cross-org references are enforced by composite FKs/triggers plus service assertions.
  • Every tool call goes through the API tool gateway, has a policy decision, and has side-effect idempotency for writes/sends/submits.
  • Every message/channel send preserves default HITL unless explicit auto-send config and policy allow it.
  • Every approval/input pause is a first-class row and can resume/block durably exactly once.
  • Runners cannot use vendor/R2/platform/integration credentials directly and are rejected on manifest/hash/signature/endpoint mismatch.
  • Model calls use platform credential brokerage or approved run-scoped leases, never broad provider env keys.
  • File/process/browser work uses typed capability ops inside sandboxed workers.
  • Ops/workspace/client surfaces enforce assignment/org filters and can inspect active/failed runs, timeline, approvals, policy decisions, tools, artifacts, denials, and runner provenance.
  • Legacy execution paths are deleted or return 410 with tests; no hidden compatibility authority remains.
  • Migration/cutover rehearsal proves freeze/drain/backfill/reopen, high-water replay, and containment behavior.

Final implementation-ready meta-prompt​

Implement the hard-cut Rig-derived Human.work runtime control plane. Use Rig as the architectural reference, not old Splinter and not Rig’s product/server/UI wholesale. Build a Human.work-native NestJS authority with agent_runs, run_events, approval_requests, user_input_requests, tool_calls, policy_decisions, run_artifacts, runner_manifests, runner_registrations, and unresolved-ingress quarantine. Use explicit Postgres migrations, indexes, RLS policies, composite org FKs/triggers, and a migration-only CI path with TypeORM synchronize disabled.

Outcome required: Nest/API is the single authority for org scope, credentials/model brokerage, policy, tool dispatch, approvals/input, event ledger, artifacts, run lifecycle, runner manifests, and sandbox capability dispatch. Hermes/Python is only a signed runner adapter: starts from platform run contracts, emits signed events, calls model/tool/capability gateways, and has no broad provider/R2/platform/vendor integration secrets. All tools go through ToolGatewayService.execute(runId, ...); all model credentials go through ModelGatewayService/credential broker; all file/process/browser actions go through typed capability ops in a real sandbox worker. Replace task JSON approvals with first-class approval/input rows. Preserve Human.work default HITL: no message/channel auto-send unless explicitly configured and policy-allowed.

Hard constraints:

  • No production dual-write/compatibility bridge. Old authority routes are deleted or 410: LEGACY_AGENT_API, /tools/execute, direct /v1/agent-api/tools/:toolName, authoritative /agentic/tasks* mutation/SSE, legacy /v1/tasks*, Python direct vendor tools/fallback proxy, process-local idempotency, production stubs/mock credential success, legacy run statuses, and runtime actions writes.
  • Hotfix or disable current /agentic/tasks* cross-org read/approve/reject risk before waiting for full cutover if production reachable.
  • No runtime execution for org-less traffic; unresolved channel events go to quarantine/remediation.
  • No arbitrary DB-provided runner URLs; use service discovery/outbound polling or strict allowlist/mTLS/DNS/IP protections.
  • No fake sandboxing in the API process. If the platform cannot enforce container/OS isolation, use a worker substrate that can.
  • No KB/Haystack access in manifests until #541/#291/#300 scopes are decided and tested.
  • packages/agent-harness must be wired under DB-backed run/event services or retired from production imports.

Suggested approach: follow the milestones in this plan: immediate /agentic/tasks* safety fix; schema/RLS/core ledger; policy/tool/model gateways; approvals/input/artifacts; conversation/task/channel wiring; runner registry/Hermes adapter; capability shell/sandbox; UIs/read models; validators/rehearsal; atomic cutover. For run creation, allocate run id and compute policy/tool/prompt hashes before insert. For event append, lock the parent run row and increment next_event_sequence; never use max(sequence)+1. For approvals/tool continuation, persist tool_calls, use idempotency keys for side effects, and resume via runtime adapter with the same correlation chain.

Validation before finish:

  • cd api && npm run build && npm test -- --runInBand
  • Postgres migration smoke with SYNCHRONIZE=false, RLS/trigger/index inspection, PgBouncer/transaction-pooling context test or equivalent
  • cd agent && python3 -m pytest tests/ evals/ -q
  • cd frontend && npm run build && npm run lint
  • Cross-org/IDOR tests for run/event/approval/input/tool/artifact endpoints, including internal AM/Expert assignment filters
  • Concurrent event append and callback idempotency tests
  • Tool gateway failure/approval/block/success and retry-after-timeout write idempotency tests
  • Default HITL message/channel send tests
  • Runner manifest/signature/endpoint rejection tests and no broad runner secrets checks
  • Redaction tests for events, audit, artifact metadata, and WebSocket/SSE broadcasts
  • Sandbox denial/lease/cleanup tests
  • E2E Expert approval, client approval, Ops timeline, fail-open escalation, and unresolved channel quarantine flows
  • Static guards for removed routes/env/statuses/direct handlers/actions writes/wrong routes/wrong terminology
  • Staging cutover rehearsal: freeze channels/workers, drain async callbacks, snapshot high-water marks, backfill to boundary, bootstrap runners, reopen/replay, and test containment switches

Escalate before implementation lock for: runtime key Org×Specialist vs org default, sandbox substrate, artifact bucket isolation vs prefix compensating controls, manifest signing key custody/rotation, packages/agent-harness disposition, KB/Haystack ACL model, and any exception to model gateway/no-runner-secrets policy. Stop only when the Definition of Done above is met or a named implementation-lock decision blocks progress.

Last updated on by Degen