Skip to main content

Agent Service Architecture β€” Per-Client Sandboxed Deployments

Author: Brad (Hermes Agent) Date: 2026-05-04 Status: Proposal β€” for founder review Cross-ref: ARCHITECTURE.md, implementation-status.md, docs/features/agentic-tasks.md

⚠️ Runtime Note (2026-05-07, updated 2026-05-12): The current production runtime is Hermes CLI (agent/hermes_client.py). POST /v1/chat shells out to the hermes chat CLI; LangGraph rollback and /graph/invoke have been removed and deferred. The per-client sandboxed deployment model described in this doc is the target architecture (Phase 2+), not the current state. See IMPLEMENTATION_STATUS.md § 15 for current AI agent runtime status.

πŸ›‘ Superseded for /v1/tasks* runtime (2026-05-25, #696 β€” RIG hard-cut): All POST /v1/tasks, POST /v1/tasks/{id}/steps/{n}/go|no|approve|reject, and POST /v1/tasks/{id}/cancel endpoints listed below were removed. The Agent service returns HTTP 410 (LEGACY_TASK_AUTHORITY_REMOVED) for any /v1/tasks* request. The Platform /agentic/tasks/* surface is preserved but is now backed by the RIG runtime control plane (docs/architecture/RIG_HARD_CUT_RUNTIME_PORT_PLAN.md). This doc is retained for the per-client sandbox isolation architecture (sections 0–5, 7+); the task-lifecycle endpoint list in Β§3 and Β§5 is historical, not current API.


0. Design Constraints (non-negotiable)​

  1. No database access. Agent instances never connect to PostgreSQL, Redis, or any platform data store. Zero connection strings, zero query capability.
  2. API-only communication. Agents fetch all context they need from the humanwork Platform API using a scoped auth token. The token restricts the agent to a single org β€” it is structurally impossible for an agent to read another org's data.
  3. Per-client isolation. Each client org gets 1–N dedicated agent instances. Agents are never shared across orgs. Each runs in its own sandbox (container/pod).
  4. Data security is paramount. Even though these are internal systems, the architecture must enforce segregation at every layer β€” not rely on "trust" or "we won't do that."
  5. Audit everything. Every agent action (LLM call, tool execution, callback) must be traceable for diagnosis and compliance.

1. Problem Statement​

Today the agent is a single shared FastAPI service with direct API calls using env-var credentials. This violates all five constraints above:

  • One process serves all orgs β€” no isolation
  • Agent reads Shopify/Amazon credentials from env vars β€” credentials for all orgs in one process
  • Agent calls GET /memory/search on the platform but nothing prevents an agent bug from querying the wrong org
  • No audit trail of what the agent did or why

2. Use Case Enumeration​

Every interaction between the platform and an AI agent, extracted from the codebase:

2.1 Synchronous Requestβ†’Response (Platform β†’ Agent)​

#Use CaseDescription
UC-1Draft a replyClient sends message. Platform calls agent with message + history + specialist persona. Agent returns {reply, confidence, risk_level, flag_for_review}. Platform routes to expert queue or auto-sends.
UC-2Stream a replySame as UC-1, SSE streaming. Frontend renders tokens as they arrive.
UC-3Classify task modeAgent-internal: decides "domain" (bounded tool call) vs "agentic" (multi-step plan).
UC-4Score confidenceAgent-internal: second LLM call to self-rate confidence 1–10 β†’ 0.0–1.0.
UC-5Classify riskAgent-internal: LOW/MEDIUM/HIGH/CRITICAL. Platform uses for routing.

2.2 Tool Execution (Agent β†’ External APIs, via Platform)​

#Use CaseCurrent StateTarget State
UC-6Query ShopifyAgent reads env vars, calls Shopify directlyAgent calls Platform API proxy: GET /v1/agent-api/tools/shopify?action=... β€” platform resolves credentials from encrypted store
UC-7Query AmazonSame β€” direct API callSame pattern β€” platform proxy
UC-8Browse webPlaywright in agent containerStays in agent β€” no credential issue
UC-9Retrieve memory/contextAgent calls GET /memory/searchPlatform retrieves via Haystack and exposes GET /v1/agent-api/context?query=... for org-scoped results only
UC-10Finance/IR toolsFunction signatures only (stubs)Same proxy pattern when implemented

Key change: Agents never hold third-party credentials. They call the Platform API, which resolves credentials from the encrypted store (IntegrationCredential entity, AES-256-GCM) and proxies the request. The agent gets results, never keys.

2.3 Agentic Tasks β€” Multi-Step with HITL Gates (Bidirectional)​

#Use CaseDescription
UC-11Agent proposes a planAgent decomposes complex request into steps with risk levels. Pushes task.plan_ready event to platform.
UC-12Expert approves/rejects stepExpert reviews in UI. Platform sends POST /v1/tasks/{id}/steps/{n}/go or /no to agent.
UC-13Agent executes approved stepAgent calls platform tool proxy to execute. Reports result via task.step_completed callback.
UC-14Agent reports step resultPush task.step_completed or task.step_failed to platform callback.
UC-15Agent requests human inputPush task.needs_input β€” agent pauses until expert responds.

2.4 Agent Lifecycle (Platform manages instances)​

#Use CaseDescription
UC-16Provision agentOnboarding creates org β†’ platform provisions agent container with org-scoped token.
UC-17Configure agentPlatform pushes config (prompt, model, tools, threshold) via POST /v1/configure.
UC-18Health checkPlatform polls GET /v1/health.
UC-19Scale agentPlatform scales container replicas based on load.
UC-20Teardown agentOrg deactivated β†’ platform destroys container + revokes token.

2.5 Learning & Feedback (Platform β†’ Agent)​

#Use CaseDescription
UC-21Expert correctionPlatform pushes corrections via POST /v1/corrections. Agent ingests for prompt refinement.
UC-22Knowledge base updatePlatform ingests documents through /api/kb/documents into Haystack. Agent-side ingest endpoints are deprecated under platform-as-RAG-owner.
UC-23Prompt refinementPlatform pushes few-shot examples extracted from correction patterns.

3. Security Model​

3.1 The Scoped Agent Token​

Each agent instance receives a Platform API token at provisioning time. This token:

{
"sub": "agent-acme-001", // agent instance ID
"org_id": "uuid-of-acme", // the ONLY org this agent can access
"role": "agent", // platform role β€” NOT expert, NOT admin
"scopes": [
"conversations:read", // read message history for its org
"context:read", // search memory/KB for its org
"tools:execute", // call tool proxies for its org
"callbacks:write" // push events back to platform
],
"iat": 1746360000,
"exp": 1746446400 // 24h expiry, auto-rotated
}

Enforcement on Platform side:

  • All /v1/agent-api/* endpoints check req.agent.org_id matches the resource's org_id
  • RLS interceptor applies to agent tokens the same way it applies to user tokens
  • An agent token for Org A literally cannot query Org B's data β€” the WHERE clause filters it out at the DB level

Token lifecycle:

  • Created during agent provisioning
  • Auto-rotated every 24h (agent calls POST /v1/agent-api/token/refresh)
  • Revoked instantly when org is deactivated or agent is torn down
  • Never stored in agent β€” passed as env var PLATFORM_API_TOKEN at container start

3.2 What the Agent CANNOT Do​

ActionBlocked by
Query another org's conversationsToken org_id + RLS
Read another org's integration credentialsToken org_id + service-layer check
Connect to PostgreSQLNo DATABASE_URL β€” not in env, not in config, not discoverable
Connect to RedisSame
Call admin/superadmin endpointsToken role is agent β€” guards reject
Modify org settings, users, billingToken scopes don't include write access to these resources
Read raw credentials (Shopify tokens, etc.)Tool proxy returns results, never keys. Agent sends {tool: "shopify", action: "getOrder", params: {id: "123"}} and gets the order back

3.3 What the Agent CAN Do​

ActionHow
Read conversation history for its orgGET /v1/agent-api/conversations/{id}/messages
Search org knowledge base / memoryGET /v1/agent-api/context?query=... backed by Haystack
Execute tools (Shopify, Amazon, etc.)POST /v1/agent-api/tools/{tool_name}`` β€” platform proxies with credentials
Push events to platformPOST /v1/agent-events with HMAC signature
Call LLM APIs (OpenAI, Anthropic, etc.)Direct β€” agent holds its own LLM API key (or org-specific key)

3.4 Network Isolation​

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”     β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ Platform Network β”‚ β”‚ Agent Sandbox (per-org) β”‚
β”‚ β”‚ β”‚ β”‚
β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
β”‚ β”‚ API β”‚ β”‚ PostgreSQL β”‚ β”‚ β”‚ β”‚ Agent Instance β”‚ β”‚
β”‚ β”‚ Server β”‚ β”‚ + Redis β”‚ β”‚ β”‚ β”‚ β”‚ β”‚
β”‚ β”‚ │◄── β”‚ β”‚ β”‚ β”‚ - No DB access β”‚ β”‚
β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ - Platform API token β”‚ β”‚
β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ - LLM API key β”‚ β”‚
β”‚ β”‚ │◄────────────────┼─────┼──│ - Callback secret β”‚ β”‚
β”‚ β”‚ β”‚ HTTPS only β”‚ β”‚ β”‚ β”‚ β”‚
β”‚ β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”Όβ”€β–Ίβ”‚ (agent-api + events) β”‚ β”‚
β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
β”‚ β”‚ β”‚ β”‚
β”‚ Agent CANNOT reach: β”‚ β”‚ Agent CAN reach: β”‚
β”‚ - PostgreSQL (port 5432) β”‚ β”‚ - Platform API (HTTPS) β”‚
β”‚ - Redis (port 6379) β”‚ β”‚ - LLM providers (HTTPS) β”‚
β”‚ - Internal services β”‚ β”‚ - Tool target APIs (via β”‚
β”‚ β”‚ β”‚ platform proxy) β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

In K8s: NetworkPolicy whitelists only HTTPS egress to Platform API + LLM providers. All other egress blocked.


4. Architecture​

4.1 System Overview​

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ PLATFORM API (NestJS) β”‚
β”‚ β”‚
β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
β”‚ β”‚ Agent Gateway Module β”‚ β”‚
β”‚ β”‚ β”‚ β”‚
β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚
β”‚ β”‚ β”‚ Registry β”‚ β”‚ Router β”‚ β”‚ Lifecycle Mgr β”‚ β”‚ β”‚
│ │ │ (org→agent │ │ (dispatch to │ │ (provision, │ │ │
β”‚ β”‚ β”‚ mapping) β”‚ β”‚ correct β”‚ β”‚ configure, β”‚ β”‚ β”‚
β”‚ β”‚ β”‚ β”‚ β”‚ instance) β”‚ β”‚ scale, teardown) β”‚ β”‚ β”‚
β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚
β”‚ β”‚ β”‚ β”‚ β”‚
β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚
β”‚ β”‚ β”‚ Agent API β”‚ β”‚ Callback Handler β”‚ β”‚ β”‚
β”‚ β”‚ β”‚ /v1/agent-api/* β”‚ β”‚ POST /agent-eventsβ”‚ β”‚ β”‚
β”‚ β”‚ β”‚ (context, tools, β”‚ β”‚ (HMAC verified) β”‚ β”‚ β”‚
β”‚ β”‚ β”‚ conversations) β”‚ β”‚ β”‚ β”‚ β”‚
β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚
β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β”‚ HTTPS (scoped token)
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ β”‚ β”‚
β–Ό β–Ό β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ Agent: β”‚ β”‚ Agent: β”‚ β”‚ Agent: β”‚
β”‚ Acme Corp β”‚ β”‚ Meridian β”‚ β”‚ GlobalTech β”‚
β”‚ β”‚ β”‚ Logistics β”‚ β”‚ β”‚
β”‚ Token: β”‚ β”‚ Token: β”‚ β”‚ Token: β”‚
β”‚ org=acme β”‚ β”‚ org=merid β”‚ β”‚ org=gtech β”‚
β”‚ Model: 4o β”‚ β”‚ Model: 4o β”‚ β”‚ Model: β”‚
β”‚ No DB. β”‚ β”‚ No DB. β”‚ β”‚ Claude β”‚
β”‚ No creds. β”‚ β”‚ No creds. β”‚ β”‚ No DB. β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

4.2 Platform β†’ Agent Interface (Commands)​

POST   /v1/chat                     # UC-1: Draft reply (sync)
POST /v1/chat/stream # UC-2: Draft reply (SSE stream)
POST /v1/tasks # UC-11: Request plan for complex task
POST /v1/tasks/{id}/steps/{n}/go # UC-12: Approve step, resume execution
POST /v1/tasks/{id}/steps/{n}/no # UC-12: Reject step, halt
POST /v1/tasks/{id}/cancel # Cancel running task
POST /v1/configure # UC-17: Hot-reload config (prompt, model, tools, threshold)
POST /v1/knowledge/ingest # UC-22: Push KB documents
POST /v1/knowledge/remove # Remove KB documents
POST /v1/corrections # UC-21: Push expert corrections for learning
GET /v1/health # UC-18: Health + readiness
GET /v1/status # Running tasks, model info, uptime

4.3 Agent β†’ Platform Interface (Data Fetching + Callbacks)​

Data fetching β€” agent calls platform to get context it needs (never DB-direct):

GET  /v1/agent-api/conversations/{id}/messages   # Conversation history
GET /v1/agent-api/context?query=... # Haystack KB search (org-scoped)
POST /v1/agent-api/tools/{tool_name} # Execute tool via platform proxy
POST /v1/agent-api/token/refresh # Rotate token before expiry

Event callbacks β€” agent pushes events to platform:

POST /v1/agent-events

Event types:
task.plan_ready # Agent produced a plan, needs expert approval
task.step_started # Agent began executing step N
task.step_completed # Step N succeeded + results
task.step_failed # Step N failed + error
task.completed # All steps done
task.needs_input # Agent paused, needs expert clarification
tool.called # Audit: agent invoked a tool (name, params, result summary)
llm.called # Audit: agent made an LLM call (model, tokens, cost, latency)
error # Unrecoverable error

Every event is signed (HMAC-SHA256 with per-instance secret) and carries:

{
"event_type": "task.step_completed",
"agent_id": "agent-acme-001",
"org_id": "uuid",
"task_id": "uuid",
"step_id": 2,
"timestamp": "2026-05-04T14:30:00Z",
"payload": { ... },
"idempotency_key": "uuid"
}

4.4 Audit Trail​

Every agent interaction is logged to agent_audit_log on the platform side:

WhatWhenWhat's Logged
Chat requestPlatform β†’ Agentorg_id, conversation_id, message length, timestamp
Chat responseAgent β†’ Platformconfidence, risk_level, flag_for_review, token usage, cost, latency
Tool executionAgent β†’ Platform (proxy)tool_name, parameters (sanitized), result summary, duration
LLM callAgent β†’ callbackmodel, input/output tokens, cost, latency, prompt version
Task lifecycleAgent β†’ callbacktask_id, step transitions, approver_id, execution results
ErrorAgent β†’ callbackerror type, stack trace (sanitized), context
Config changePlatform β†’ Agentwhat changed, who triggered it, before/after
Token rotationPlatform internalold token hash, new token hash, expiry

Platform-side β€” all logged to agent_audit_log table (new entity). Not logged by the agent itself β€” the agent has no DB.

4.5 Tool Proxy Pattern (how agents call tools without holding credentials)​

Agent wants to query Shopify:

Agent Platform API Shopify
β”‚ β”‚ β”‚
β”œβ”€β”€ POST /v1/agent-api/tools/shopify β”‚
β”‚ { action: "getOrder", β”‚ β”‚
β”‚ params: { id: "1234" } } β”‚ β”‚
β”‚ Authorization: Bearer <token> β”‚
β”‚ β”‚ β”‚
β”‚ β”œβ”€β”€ Verify token (org_id=acme) β”‚
β”‚ β”œβ”€β”€ Lookup IntegrationCredential β”‚
β”‚ β”‚ for org=acme, type=shopify β”‚
β”‚ β”œβ”€β”€ Decrypt credentials (AES-256)β”‚
β”‚ β”œβ”€β”€ GET /admin/api/orders/1234 ───
β”‚ β”‚ X-Shopify-Access-Token: xxx β”‚
β”‚ β”‚ β”‚
β”‚ │◄── { order: { ... } } ─────────
β”‚ β”‚ β”‚
│◄── { result: { order: ... } } β”‚ β”‚
β”‚ (no credentials exposed) β”‚ β”‚
β”‚ β”œβ”€β”€ Log to agent_audit_log β”‚
β”‚ β”‚ β”‚

The agent never sees credentials. It sends {tool, action, params} and gets results.


5. Agent Instance Configuration​

Agent receives config at startup (env vars) + hot-reload via POST /v1/configure:

# Injected as env vars at container start
PLATFORM_API_URL: "https://api.h.work"
PLATFORM_API_TOKEN: "<scoped JWT, auto-rotated 24h>"
CALLBACK_URL: "https://api.h.work/v1/agent-events"
CALLBACK_SECRET: "<HMAC signing key>"
LLM_API_KEY: "<OpenRouter or direct provider key>"
AGENT_MODEL: "openai/gpt-4o"

# NO DATABASE_URL. NO REDIS_URL. NO SHOPIFY_TOKEN. NO AMAZON_KEY.

Config pushed via POST /v1/configure (hot-reload without restart):

{
"specialist": {
"name": "Bob",
"system_prompt": "You are Bob, a senior e-commerce operations specialist..."
},
"model": "openai/gpt-4o",
"temperature": 0.3,
"confidence_threshold": 70,
"enabled_tools": ["shopify_query", "amazon_query"],
"max_concurrent_tasks": 3
}

5.1 Agent Internal Architecture​

Agent Instance (per-org sandbox)
β”‚
β”œβ”€β”€ /v1/chat β†’ ChatHandler
β”‚ β”œβ”€β”€ PromptBuilder (specialist persona, injected via configure)
β”‚ β”œβ”€β”€ ContextFetcher (calls Platform: /v1/agent-api/context)
β”‚ β”œβ”€β”€ LLMClient (OpenRouter/direct β€” agent holds LLM key only)
β”‚ β”œβ”€β”€ ToolCaller (calls Platform: /v1/agent-api/tools/*)
β”‚ β”œβ”€β”€ ConfidenceScorer
β”‚ β”œβ”€β”€ RiskClassifier
β”‚ └── AuditEmitter (pushes llm.called, tool.called events)
β”‚
β”œβ”€β”€ /v1/tasks β†’ TaskHandler
β”‚ β”œβ”€β”€ Planner (decompose β†’ steps)
β”‚ β”œβ”€β”€ Checkpointer (local SQLite or file β€” NOT platform DB)
β”‚ β”œβ”€β”€ StepExecutor (calls Platform tool proxy per step)
β”‚ └── CallbackEmitter (pushes task.* events)
β”‚
β”œβ”€β”€ /v1/configure β†’ ConfigHandler (hot-reload model/prompt/tools)
β”œβ”€β”€ /v1/knowledge/* β†’ Deprecated under platform-as-RAG-owner; NestJS owns Haystack ingestion
β”œβ”€β”€ /v1/corrections β†’ LearningHandler (ingest β†’ local few-shot example store)
β”œβ”€β”€ /v1/health β†’ HealthHandler
└── /v1/status β†’ StatusHandler

Local state: Agent maintains Hermes session mappings and workspace files. KB retrieval is platform-owned. NestJS injects retrieved KB context into conversation history and also serves /v1/agent-api/context for agent callers. Haystack credentials never leave the platform.


6. Deployment Model​

6.1 Container per Org (launch default)​

Each org gets a dedicated container:

Railway / ECS Fargate / K8s:
β”œβ”€β”€ agent-acme-001 (256MB RAM, 0.25 vCPU)
β”œβ”€β”€ agent-meridian-001 (256MB RAM, 0.25 vCPU)
β”œβ”€β”€ agent-globaltech-001 (512MB RAM, 0.5 vCPU) ← enterprise tier
└── ...

Substrate note (2026-06-01): Railway hosts prod today; staging migrating to ECS Fargate (ADR-003 update, #1101/#1303). See ADR-022 for the longer-term sandbox substrate decision.

One universal Docker image. Config-driven specialization via env vars + /v1/configure.

Cost: ~$5–15/mo per agent at idle. 100 orgs = $500–1,500/mo. Acceptable given $500+/mo plan pricing.

6.2 Multiple Agents per Org​

Enterprise clients can have multiple specialized agents:

Acme Corp:
β”œβ”€β”€ agent-acme-ecommerce (Shopify + Amazon, ecommerce_ops persona)
β”œβ”€β”€ agent-acme-finance (QuickBooks + Xero, finance_ops persona)
└── agent-acme-ir (HKEX, ir_reporting persona)

Each gets its own scoped token (same org_id, different agent_id). The registry supports 1:N org→agent mapping. Router picks the right agent based on conversation role or specialist assignment.

6.3 Future: Serverless (low-volume orgs)​

For orgs with < 100 messages/month, run agent as serverless function:

  • Cold start: 2–5s (acceptable β€” client is waiting for AI reply)
  • Trade-off: no persistent in-agent planner state β€” must re-hydrate from platform on each invocation

7. Migration Path​

Phase 1: Define the interface (1–2 weeks)​

No deployment changes. Keep single shared agent. But:

  1. Write OpenAPI spec for ACP (/v1/* endpoints)
  2. Build agent-api/ endpoints on platform (context fetch, tool proxy)
  3. Replace AgentClient β†’ AgentGateway (initially returns shared URL for all orgs)
  4. Add POST /v1/agent-events callback handler on platform
  5. Add agent_audit_log entity and logging

Result: Clean interface defined. Agent still shared, but protocol ready.

Phase 2: Per-org provisioning (2–3 weeks)​

  1. Build agent_instances table + lifecycle management
  2. Implement scoped token issuance (JWT with org_id claim)
  3. During onboarding, auto-provision agent container
  4. Migrate agent to fetch context via platform API instead of direct calls
  5. Migrate tool execution to platform proxy pattern
  6. Remove all DB connection code from agent

Result: New orgs get dedicated sandboxed agents. Existing orgs migrated one at a time.

Phase 3: Agentic execution (3–4 weeks)​

  1. Implement Hermes-backed tool execution using the platform tool proxy
  2. Persist Hermes session state through the workspace/session mapping
  3. Step-level approval flow via callback protocol
  4. Expert task UI (approve/reject steps, view results, chat with agent)

Result: Full plan→gate→execute with real tool execution, HITL approval, complete audit trail.


8. Decision Points​

#QuestionRecommendation
1Container runtimeRailway containers (Phase 1, prod today), ECS Fargate (Phase 2, staging in flight per #1101/#1303), K8s pods (Phase 3+) β€” see ADR-003 update and ADR-022
2Agent state storageHermes session mapping and workspace state (persistent volume, re-hydratable from platform)
3Credential delivery to agentNone β€” agent never holds third-party credentials. Tool proxy pattern.
4LLM API keyAgent holds its own LLM key (or per-org key for cost tracking). Only key the agent gets.
5Knowledge base storageHaystack via NestJS (api/src/haystack/*, /api/kb/*) with pgvector on Postgres. Agent-side local index is deprecated.
6Callback securityHMAC-SHA256 per instance (consistent with channel webhook pattern)
7Token rotation24h auto-rotation. Agent calls /v1/agent-api/token/refresh before expiry.
8Agent imageOne universal image, config-driven. No per-domain images.

9. Files to Create/Modify​

Status update (2026-05-25): The aspirational agent-gateway/ module described below was implemented under different (shipped) directory names: api/src/agent-api/ and api/src/agent-events/. The "gateway" naming was never adopted in code. The AgentClient deletion is deferred β€” the file still exists and is still used by ConversationsService and ExpertAgentThreadService. Treat the table below as the original design intent; consult the column on the right for the current state.

ActionOriginal target pathDescriptionCurrent state
Createdapi/src/agent-api/HTTP routes the agent calls (context fetch, tool proxy)βœ… Shipped (agent-api.controller.ts, agent-api.guard.ts, agent-api.service.ts, agent-lifecycle.service.ts, tool-registry.ts)
Createdapi/src/agent-events/HMAC-verified callback receiverβœ… Shipped (agent-events.controller.ts, agent-events.service.ts)
Createdapi/src/agent-api/agent-runtime.module.tsModule wiringβœ… Shipped
Plannedapi/src/agent-gateway/agent-gateway.service.tsReplaces AgentClient β€” org-aware routing⏸️ Deferred (Phase 2) β€” AgentClient still used in ConversationsService
Plannedapi/src/agent-gateway/agent-registry.tsagent_instances table CRUD⏸️ Deferred (Phase 2)
Plannedapi/src/agent-gateway/agent-lifecycle.tsProvision/configure/scale/teardown⏸️ Partially in agent-lifecycle.service.ts
Plannedapi/src/agent-gateway/agent-audit.service.tsagent_audit_log entity + logging⏸️ Deferred
Createdagent/config.pyConfig loaderβœ… Shipped
Createdagent/platform_client.pyHTTP client for Platform APIβœ… Shipped (via hermes_client.py)
Createdagent/callbacks.pyEvent emitter to platformβœ… Shipped
Createdagent/handlers/Extracted handlersβœ… Shipped (chat.py, tasks.py, etc.)
Createdagent/AGENT_API_SPEC.mdOpenAPI spec for ACP (formerly proposed as docs/design/AGENT_CONTROL_PROTOCOL.md)βœ… Shipped in agent/
Modifyapi/src/conversations/conversations.service.tsReplace AgentClient with new gateway⏸️ Deferred (still uses AgentClient)
Modifyapi/src/common/entities.tsAdd AgentInstance, AgentAuditLog entities⏸️ Deferred
Deleteapi/src/common/agent.client.tsReplace with the new gateway⏸️ Deferred β€” file still present and in use
DeleteAgent env vars: DATABASE_URL, REDIS_URL, SHOPIFY_*, AMAZON_*Agent must not have these⏸️ Partially β€” agent still holds some of these