Agent Service Architecture β Per-Client Sandboxed Deployments
Author: Brad (Hermes Agent) Date: 2026-05-04 Status: Proposal β for founder review Cross-ref: ARCHITECTURE.md, implementation-status.md, docs/features/agentic-tasks.md
β οΈ Runtime Note (2026-05-07, updated 2026-05-12): The current production runtime is Hermes CLI (
agent/hermes_client.py).POST /v1/chatshells out to thehermes chatCLI; LangGraph rollback and/graph/invokehave been removed and deferred. The per-client sandboxed deployment model described in this doc is the target architecture (Phase 2+), not the current state. See IMPLEMENTATION_STATUS.md Β§ 15 for current AI agent runtime status.
π Superseded for
/v1/tasks*runtime (2026-05-25, #696 β RIG hard-cut): AllPOST /v1/tasks,POST /v1/tasks/{id}/steps/{n}/go|no|approve|reject, andPOST /v1/tasks/{id}/cancelendpoints listed below were removed. The Agent service returns HTTP 410 (LEGACY_TASK_AUTHORITY_REMOVED) for any/v1/tasks*request. The Platform/agentic/tasks/*surface is preserved but is now backed by the RIG runtime control plane (docs/architecture/RIG_HARD_CUT_RUNTIME_PORT_PLAN.md). This doc is retained for the per-client sandbox isolation architecture (sections 0β5, 7+); the task-lifecycle endpoint list in Β§3 and Β§5 is historical, not current API.
0. Design Constraints (non-negotiable)β
- No database access. Agent instances never connect to PostgreSQL, Redis, or any platform data store. Zero connection strings, zero query capability.
- API-only communication. Agents fetch all context they need from the humanwork Platform API using a scoped auth token. The token restricts the agent to a single org β it is structurally impossible for an agent to read another org's data.
- Per-client isolation. Each client org gets 1βN dedicated agent instances. Agents are never shared across orgs. Each runs in its own sandbox (container/pod).
- Data security is paramount. Even though these are internal systems, the architecture must enforce segregation at every layer β not rely on "trust" or "we won't do that."
- Audit everything. Every agent action (LLM call, tool execution, callback) must be traceable for diagnosis and compliance.
1. Problem Statementβ
Today the agent is a single shared FastAPI service with direct API calls using env-var credentials. This violates all five constraints above:
- One process serves all orgs β no isolation
- Agent reads Shopify/Amazon credentials from env vars β credentials for all orgs in one process
- Agent calls
GET /memory/searchon the platform but nothing prevents an agent bug from querying the wrong org - No audit trail of what the agent did or why
2. Use Case Enumerationβ
Every interaction between the platform and an AI agent, extracted from the codebase:
2.1 Synchronous RequestβResponse (Platform β Agent)β
| # | Use Case | Description |
|---|---|---|
| UC-1 | Draft a reply | Client sends message. Platform calls agent with message + history + specialist persona. Agent returns {reply, confidence, risk_level, flag_for_review}. Platform routes to expert queue or auto-sends. |
| UC-2 | Stream a reply | Same as UC-1, SSE streaming. Frontend renders tokens as they arrive. |
| UC-3 | Classify task mode | Agent-internal: decides "domain" (bounded tool call) vs "agentic" (multi-step plan). |
| UC-4 | Score confidence | Agent-internal: second LLM call to self-rate confidence 1β10 β 0.0β1.0. |
| UC-5 | Classify risk | Agent-internal: LOW/MEDIUM/HIGH/CRITICAL. Platform uses for routing. |
2.2 Tool Execution (Agent β External APIs, via Platform)β
| # | Use Case | Current State | Target State |
|---|---|---|---|
| UC-6 | Query Shopify | Agent reads env vars, calls Shopify directly | Agent calls Platform API proxy: GET /v1/agent-api/tools/shopify?action=... β platform resolves credentials from encrypted store |
| UC-7 | Query Amazon | Same β direct API call | Same pattern β platform proxy |
| UC-8 | Browse web | Playwright in agent container | Stays in agent β no credential issue |
| UC-9 | Retrieve memory/context | Agent calls GET /memory/search | Platform retrieves via Haystack and exposes GET /v1/agent-api/context?query=... for org-scoped results only |
| UC-10 | Finance/IR tools | Function signatures only (stubs) | Same proxy pattern when implemented |
Key change: Agents never hold third-party credentials. They call the Platform API, which resolves credentials from the encrypted store (IntegrationCredential entity, AES-256-GCM) and proxies the request. The agent gets results, never keys.
2.3 Agentic Tasks β Multi-Step with HITL Gates (Bidirectional)β
| # | Use Case | Description |
|---|---|---|
| UC-11 | Agent proposes a plan | Agent decomposes complex request into steps with risk levels. Pushes task.plan_ready event to platform. |
| UC-12 | Expert approves/rejects step | Expert reviews in UI. Platform sends POST /v1/tasks/{id}/steps/{n}/go or /no to agent. |
| UC-13 | Agent executes approved step | Agent calls platform tool proxy to execute. Reports result via task.step_completed callback. |
| UC-14 | Agent reports step result | Push task.step_completed or task.step_failed to platform callback. |
| UC-15 | Agent requests human input | Push task.needs_input β agent pauses until expert responds. |
2.4 Agent Lifecycle (Platform manages instances)β
| # | Use Case | Description |
|---|---|---|
| UC-16 | Provision agent | Onboarding creates org β platform provisions agent container with org-scoped token. |
| UC-17 | Configure agent | Platform pushes config (prompt, model, tools, threshold) via POST /v1/configure. |
| UC-18 | Health check | Platform polls GET /v1/health. |
| UC-19 | Scale agent | Platform scales container replicas based on load. |
| UC-20 | Teardown agent | Org deactivated β platform destroys container + revokes token. |
2.5 Learning & Feedback (Platform β Agent)β
| # | Use Case | Description |
|---|---|---|
| UC-21 | Expert correction | Platform pushes corrections via POST /v1/corrections. Agent ingests for prompt refinement. |
| UC-22 | Knowledge base update | Platform ingests documents through /api/kb/documents into Haystack. Agent-side ingest endpoints are deprecated under platform-as-RAG-owner. |
| UC-23 | Prompt refinement | Platform pushes few-shot examples extracted from correction patterns. |
3. Security Modelβ
3.1 The Scoped Agent Tokenβ
Each agent instance receives a Platform API token at provisioning time. This token:
{
"sub": "agent-acme-001", // agent instance ID
"org_id": "uuid-of-acme", // the ONLY org this agent can access
"role": "agent", // platform role β NOT expert, NOT admin
"scopes": [
"conversations:read", // read message history for its org
"context:read", // search memory/KB for its org
"tools:execute", // call tool proxies for its org
"callbacks:write" // push events back to platform
],
"iat": 1746360000,
"exp": 1746446400 // 24h expiry, auto-rotated
}
Enforcement on Platform side:
- All
/v1/agent-api/*endpoints checkreq.agent.org_idmatches the resource's org_id - RLS interceptor applies to agent tokens the same way it applies to user tokens
- An agent token for Org A literally cannot query Org B's data β the WHERE clause filters it out at the DB level
Token lifecycle:
- Created during agent provisioning
- Auto-rotated every 24h (agent calls
POST /v1/agent-api/token/refresh) - Revoked instantly when org is deactivated or agent is torn down
- Never stored in agent β passed as env var
PLATFORM_API_TOKENat container start
3.2 What the Agent CANNOT Doβ
| Action | Blocked by |
|---|---|
| Query another org's conversations | Token org_id + RLS |
| Read another org's integration credentials | Token org_id + service-layer check |
| Connect to PostgreSQL | No DATABASE_URL β not in env, not in config, not discoverable |
| Connect to Redis | Same |
| Call admin/superadmin endpoints | Token role is agent β guards reject |
| Modify org settings, users, billing | Token scopes don't include write access to these resources |
| Read raw credentials (Shopify tokens, etc.) | Tool proxy returns results, never keys. Agent sends {tool: "shopify", action: "getOrder", params: {id: "123"}} and gets the order back |
3.3 What the Agent CAN Doβ
| Action | How |
|---|---|
| Read conversation history for its org | GET /v1/agent-api/conversations/{id}/messages |
| Search org knowledge base / memory | GET /v1/agent-api/context?query=... backed by Haystack |
| Execute tools (Shopify, Amazon, etc.) | POST /v1/agent-api/tools/{tool_name}`` β platform proxies with credentials |
| Push events to platform | POST /v1/agent-events with HMAC signature |
| Call LLM APIs (OpenAI, Anthropic, etc.) | Direct β agent holds its own LLM API key (or org-specific key) |
3.4 Network Isolationβ
βββββββββββββββββββββββββββββββ βββββββββββββββββββββββββββββββ
β Platform Network β β Agent Sandbox (per-org) β
β β β β
β ββββββββββ ββββββββββββββ β β ββββββββββββββββββββββββ β
β β API β β PostgreSQL β β β β Agent Instance β β
β β Server β β + Redis β β β β β β
β β ββββ€ β β β β - No DB access β β
β β β ββββββββββββββ β β β - Platform API token β β
β β β β β β - LLM API key β β
β β βββββββββββββββββββΌββββββΌβββ - Callback secret β β
β β β HTTPS only β β β β β
β β βββββββββββββββββββΌββββββΌββΊβ (agent-api + events) β β
β ββββββββββ β β ββββββββββββββββββββββββ β
β β β β
β Agent CANNOT reach: β β Agent CAN reach: β
β - PostgreSQL (port 5432) β β - Platform API (HTTPS) β
β - Redis (port 6379) β β - LLM providers (HTTPS) β
β - Internal services β β - Tool target APIs (via β
β β β platform proxy) β
βββββββββββββββββββββββββββββββ ββββ βββββββββββββββββββββββββββ
In K8s: NetworkPolicy whitelists only HTTPS egress to Platform API + LLM providers. All other egress blocked.
4. Architectureβ
4.1 System Overviewβ
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β PLATFORM API (NestJS) β
β β
β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β Agent Gateway Module β β
β β β β
β β βββββββββββββββ ββββββββββββββββ βββββββββββββββββββββ β β
β β β Registry β β Router β β Lifecycle Mgr β β β
β β β (orgβagent β β (dispatch to β β (provision, β β β
β β β mapping) β β correct β β configure, β β β
β β β β β instance) β β scale, teardown) β β β
β β βββββββββββββββ ββββββββ¬ββββββββ βββββββββββββββββββββ β β
β β β β β
β β ββββββββββββββββββββ ββ΄ββββββββββββββββββ β β
β β β Agent API β β Callback Handler β β β
β β β /v1/agent-api/* β β POST /agent-eventsβ β β
β β β (context, tools, β β (HMAC verified) β β β
β β β conversations) β β β β β
β β ββββββββββββββββββββ βββββββββββββββββββββ β β
β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
βββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββββ
β HTTPS (scoped token)
βββββββββββββββΌββββββββββββββ
β β β
βΌ βΌ βΌ
ββββββββββββββ ββββββββββββββ ββββββββββββββ
β Agent: β β Agent: β β Agent: β
β Acme Corp β β Meridian β β GlobalTech β
β β β Logistics β β β
β Token: β β Token: β β Token: β
β org=acme β β org=merid β β org=gtech β
β Model: 4o β β Model: 4o β β Model: β
β No DB. β β No DB. β β Claude β
β No creds. β β No creds. β β No DB. β
ββββββββββββββ ββββββββββββββ ββββββββββββββ
4.2 Platform β Agent Interface (Commands)β
POST /v1/chat # UC-1: Draft reply (sync)
POST /v1/chat/stream # UC-2: Draft reply (SSE stream)
POST /v1/tasks # UC-11: Request plan for complex task
POST /v1/tasks/{id}/steps/{n}/go # UC-12: Approve step, resume execution
POST /v1/tasks/{id}/steps/{n}/no # UC-12: Reject step, halt
POST /v1/tasks/{id}/cancel # Cancel running task
POST /v1/configure # UC-17: Hot-reload config (prompt, model, tools, threshold)
POST /v1/knowledge/ingest # UC-22: Push KB documents
POST /v1/knowledge/remove # Remove KB documents
POST /v1/corrections # UC-21: Push expert corrections for learning
GET /v1/health # UC-18: Health + readiness
GET /v1/status # Running tasks, model info, uptime
4.3 Agent β Platform Interface (Data Fetching + Callbacks)β
Data fetching β agent calls platform to get context it needs (never DB-direct):
GET /v1/agent-api/conversations/{id}/messages # Conversation history
GET /v1/agent-api/context?query=... # Haystack KB search (org-scoped)
POST /v1/agent-api/tools/{tool_name} # Execute tool via platform proxy
POST /v1/agent-api/token/refresh # Rotate token before expiry
Event callbacks β agent pushes events to platform:
POST /v1/agent-events
Event types:
task.plan_ready # Agent produced a plan, needs expert approval
task.step_started # Agent began executing step N
task.step_completed # Step N succeeded + results
task.step_failed # Step N failed + error
task.completed # All steps done
task.needs_input # Agent paused, needs expert clarification
tool.called # Audit: agent invoked a tool (name, params, result summary)
llm.called # Audit: agent made an LLM call (model, tokens, cost, latency)
error # Unrecoverable error
Every event is signed (HMAC-SHA256 with per-instance secret) and carries:
{
"event_type": "task.step_completed",
"agent_id": "agent-acme-001",
"org_id": "uuid",
"task_id": "uuid",
"step_id": 2,
"timestamp": "2026-05-04T14:30:00Z",
"payload": { ... },
"idempotency_key": "uuid"
}
4.4 Audit Trailβ
Every agent interaction is logged to agent_audit_log on the platform side:
| What | When | What's Logged |
|---|---|---|
| Chat request | Platform β Agent | org_id, conversation_id, message length, timestamp |
| Chat response | Agent β Platform | confidence, risk_level, flag_for_review, token usage, cost, latency |
| Tool execution | Agent β Platform (proxy) | tool_name, parameters (sanitized), result summary, duration |
| LLM call | Agent β callback | model, input/output tokens, cost, latency, prompt version |
| Task lifecycle | Agent β callback | task_id, step transitions, approver_id, execution results |
| Error | Agent β callback | error type, stack trace (sanitized), context |
| Config change | Platform β Agent | what changed, who triggered it, before/after |
| Token rotation | Platform internal | old token hash, new token hash, expiry |
Platform-side β all logged to agent_audit_log table (new entity). Not logged by the agent itself β the agent has no DB.
4.5 Tool Proxy Pattern (how agents call tools without holding credentials)β
Agent wants to query Shopify:
Agent Platform API Shopify
β β β
βββ POST /v1/agent-api/tools/shopify β
β { action: "getOrder", β β
β params: { id: "1234" } } β β
β Authorization: Bearer <token> β
β β β
β βββ Verify token (org_id=acme) β
β βββ Lookup IntegrationCredential β
β β for org=acme, type=shopify β
β βββ Decrypt credentials (AES-256)β
β βββ GET /admin/api/orders/1234 βββ€
β β X-Shopify-Access-Token: xxx β
β β β
β ββββ { order: { ... } } βββββββββ€
β β β
ββββ { result: { order: ... } } β β
β (no credentials exposed) β β
β βββ Log to agent_audit_log β
β β β
The agent never sees credentials. It sends {tool, action, params} and gets results.
5. Agent Instance Configurationβ
Agent receives config at startup (env vars) + hot-reload via POST /v1/configure:
# Injected as env vars at container start
PLATFORM_API_URL: "https://api.h.work"
PLATFORM_API_TOKEN: "<scoped JWT, auto-rotated 24h>"
CALLBACK_URL: "https://api.h.work/v1/agent-events"
CALLBACK_SECRET: "<HMAC signing key>"
LLM_API_KEY: "<OpenRouter or direct provider key>"
AGENT_MODEL: "openai/gpt-4o"
# NO DATABASE_URL. NO REDIS_URL. NO SHOPIFY_TOKEN. NO AMAZON_KEY.
Config pushed via POST /v1/configure (hot-reload without restart):
{
"specialist": {
"name": "Bob",
"system_prompt": "You are Bob, a senior e-commerce operations specialist..."
},
"model": "openai/gpt-4o",
"temperature": 0.3,
"confidence_threshold": 70,
"enabled_tools": ["shopify_query", "amazon_query"],
"max_concurrent_tasks": 3
}
5.1 Agent Internal Architectureβ
Agent Instance (per-org sandbox)
β
βββ /v1/chat β ChatHandler
β βββ PromptBuilder (specialist persona, injected via configure)
β βββ ContextFetcher (calls Platform: /v1/agent-api/context)
β βββ LLMClient (OpenRouter/direct β agent holds LLM key only)
β βββ ToolCaller (calls Platform: /v1/agent-api/tools/*)
β βββ ConfidenceScorer
β βββ RiskClassifier
β βββ AuditEmitter (pushes llm.called, tool.called events)
β
βββ /v1/tasks β TaskHandler
β βββ Planner (decompose β steps)
β βββ Checkpointer (local SQLite or file β NOT platform DB)
β βββ StepExecutor (calls Platform tool proxy per step)
β βββ CallbackEmitter (pushes task.* events)
β
βββ /v1/configure β ConfigHandler (hot-reload model/prompt/tools)
βββ /v1/knowledge/* β Deprecated under platform-as-RAG-owner; NestJS owns Haystack ingestion
βββ /v1/corrections β LearningHandler (ingest β local few-shot example store)
βββ /v1/health β HealthHandler
βββ /v1/status β StatusHandler
Local state: Agent maintains Hermes session mappings and workspace files. KB retrieval is platform-owned. NestJS injects retrieved KB context into conversation history and also serves /v1/agent-api/context for agent callers. Haystack credentials never leave the platform.
6. Deployment Modelβ
6.1 Container per Org (launch default)β
Each org gets a dedicated container:
Railway / ECS Fargate / K8s:
βββ agent-acme-001 (256MB RAM, 0.25 vCPU)
βββ agent-meridian-001 (256MB RAM, 0.25 vCPU)
βββ agent-globaltech-001 (512MB RAM, 0.5 vCPU) β enterprise tier
βββ ...
Substrate note (2026-06-01): Railway hosts prod today; staging migrating to ECS Fargate (ADR-003 update, #1101/#1303). See ADR-022 for the longer-term sandbox substrate decision.
One universal Docker image. Config-driven specialization via env vars + /v1/configure.
Cost: ~$5β15/mo per agent at idle. 100 orgs = $500β1,500/mo. Acceptable given $500+/mo plan pricing.
6.2 Multiple Agents per Orgβ
Enterprise clients can have multiple specialized agents:
Acme Corp:
βββ agent-acme-ecommerce (Shopify + Amazon, ecommerce_ops persona)
βββ agent-acme-finance (QuickBooks + Xero, finance_ops persona)
βββ agent-acme-ir (HKEX, ir_reporting persona)
Each gets its own scoped token (same org_id, different agent_id). The registry supports 1:N orgβagent mapping. Router picks the right agent based on conversation role or specialist assignment.
6.3 Future: Serverless (low-volume orgs)β
For orgs with < 100 messages/month, run agent as serverless function:
- Cold start: 2β5s (acceptable β client is waiting for AI reply)
- Trade-off: no persistent in-agent planner state β must re-hydrate from platform on each invocation
7. Migration Pathβ
Phase 1: Define the interface (1β2 weeks)β
No deployment changes. Keep single shared agent. But:
- Write OpenAPI spec for ACP (
/v1/*endpoints) - Build
agent-api/endpoints on platform (context fetch, tool proxy) - Replace
AgentClientβAgentGateway(initially returns shared URL for all orgs) - Add
POST /v1/agent-eventscallback handler on platform - Add
agent_audit_logentity and logging
Result: Clean interface defined. Agent still shared, but protocol ready.
Phase 2: Per-org provisioning (2β3 weeks)β
- Build
agent_instancestable + lifecycle management - Implement scoped token issuance (JWT with
org_idclaim) - During onboarding, auto-provision agent container
- Migrate agent to fetch context via platform API instead of direct calls
- Migrate tool execution to platform proxy pattern
- Remove all DB connection code from agent
Result: New orgs get dedicated sandboxed agents. Existing orgs migrated one at a time.
Phase 3: Agentic execution (3β4 weeks)β
- Implement Hermes-backed tool execution using the platform tool proxy
- Persist Hermes session state through the workspace/session mapping
- Step-level approval flow via callback protocol
- Expert task UI (approve/reject steps, view results, chat with agent)
Result: Full planβgateβexecute with real tool execution, HITL approval, complete audit trail.
8. Decision Pointsβ
| # | Question | Recommendation |
|---|---|---|
| 1 | Container runtime | Railway containers (Phase 1, prod today), ECS Fargate (Phase 2, staging in flight per #1101/#1303), K8s pods (Phase 3+) β see ADR-003 update and ADR-022 |
| 2 | Agent state storage | Hermes session mapping and workspace state (persistent volume, re-hydratable from platform) |
| 3 | Credential delivery to agent | None β agent never holds third-party credentials. Tool proxy pattern. |
| 4 | LLM API key | Agent holds its own LLM key (or per-org key for cost tracking). Only key the agent gets. |
| 5 | Knowledge base storage | Haystack via NestJS (api/src/haystack/*, /api/kb/*) with pgvector on Postgres. Agent-side local index is deprecated. |
| 6 | Callback security | HMAC-SHA256 per instance (consistent with channel webhook pattern) |
| 7 | Token rotation | 24h auto-rotation. Agent calls /v1/agent-api/token/refresh before expiry. |
| 8 | Agent image | One universal image, config-driven. No per-domain images. |
9. Files to Create/Modifyβ
Status update (2026-05-25): The aspirational
agent-gateway/module described below was implemented under different (shipped) directory names:api/src/agent-api/andapi/src/agent-events/. The "gateway" naming was never adopted in code. TheAgentClientdeletion is deferred β the file still exists and is still used byConversationsServiceandExpertAgentThreadService. Treat the table below as the original design intent; consult the column on the right for the current state.
| Action | Original target path | Description | Current state |
|---|---|---|---|
| Created | api/src/agent-api/ | HTTP routes the agent calls (context fetch, tool proxy) | β
Shipped (agent-api.controller.ts, agent-api.guard.ts, agent-api.service.ts, agent-lifecycle.service.ts, tool-registry.ts) |
| Created | api/src/agent-events/ | HMAC-verified callback receiver | β
Shipped (agent-events.controller.ts, agent-events.service.ts) |
| Created | api/src/agent-api/agent-runtime.module.ts | Module wiring | β Shipped |
| Planned | api/src/agent-gateway/agent-gateway.service.ts | Replaces AgentClient β org-aware routing | βΈοΈ Deferred (Phase 2) β AgentClient still used in ConversationsService |
| Planned | api/src/agent-gateway/agent-registry.ts | agent_instances table CRUD | βΈοΈ Deferred (Phase 2) |
| Planned | api/src/agent-gateway/agent-lifecycle.ts | Provision/configure/scale/teardown | βΈοΈ Partially in agent-lifecycle.service.ts |
| Planned | api/src/agent-gateway/agent-audit.service.ts | agent_audit_log entity + logging | βΈοΈ Deferred |
| Created | agent/config.py | Config loader | β Shipped |
| Created | agent/platform_client.py | HTTP client for Platform API | β
Shipped (via hermes_client.py) |
| Created | agent/callbacks.py | Event emitter to platform | β Shipped |
| Created | agent/handlers/ | Extracted handlers | β
Shipped (chat.py, tasks.py, etc.) |
| Created | agent/AGENT_API_SPEC.md | OpenAPI spec for ACP (formerly proposed as docs/design/AGENT_CONTROL_PROTOCOL.md) | β
Shipped in agent/ |
| Modify | api/src/conversations/conversations.service.ts | Replace AgentClient with new gateway | βΈοΈ Deferred (still uses AgentClient) |
| Modify | api/src/common/entities.ts | Add AgentInstance, AgentAuditLog entities | βΈοΈ Deferred |
| Delete | api/src/common/agent.client.ts | Replace with the new gateway | βΈοΈ Deferred β file still present and in use |
| Delete | Agent env vars: DATABASE_URL, REDIS_URL, SHOPIFY_*, AMAZON_* | Agent must not have these | βΈοΈ Partially β agent still holds some of these |