Feature: Expert Queue (HITL)

The Expert Queue is the core human-in-the-loop review surface. When a Specialist's confidence falls below the Confidence Threshold, the draft response is routed to the queue for an Expert to review before it is sent to the client.

Default behavior: autoRespondThreshold = 101 (Never auto-send) AND autoReplyMode = 'none' (Issue #346) for new orgs. All AI responses queue for Expert review until the AM explicitly opts the org into system_reply or agent_reply and (if agent_reply) lowers the threshold. This ensures no AI draft or template is delivered to a client without explicit AM/SuperAdmin opt-in.

See docs/features/auto-reply.md for the full auto-reply mode matrix.

Conversation Lifecycle

Client sends message (via Email, WhatsApp, Slack, or portal)
         │
         ▼
Channel adapter normalizes → POST /conversations/:id/messages
         │
         ▼
ConversationsService calls Agent Service (POST /v1/chat → Hermes CLI)
         │
         ▼
Agent returns { reply, confidence, flag_for_review, risk_level }
         │
         ├── confidence >= threshold AND risk_level NOT in (HIGH, CRITICAL)
         │         → Auto-send reply as Specialist
         │         (default threshold 101 means this path is never taken)
         │
         └── confidence < threshold OR flag_for_review OR risk_level HIGH/CRITICAL
                   → Create ExpertQueueItem (status: unread)
                   → Emit expert_queue_item_added via WebSocket
                   → Reconnect banner on Socket.io disconnect (no polling fallback — #843, closes #804)

Queue Item Lifecycle

2026-06-05 update: vocab is now the 4-state set pending | awaiting_client | resolved | snoozed. awaiting_client covers items where the Expert has replied and is waiting on the client; snoozed covers Expert-side defer. Per-draft release / discard endpoints land via #1878; per-thread Slack queue items via #1877. The pre-#1878 unread → in_review → resolved shape is preserved below for historical reference.

pending → awaiting_client → resolved
              ↑    ↓
              └─ snoozed

Status	Meaning
`pending`	New item; Expert has not released or sent a reply
`awaiting_client`	Expert sent / released a reply; waiting on the client
`resolved`	Thread closed (resolved or auto-resolved)
`snoozed`	Expert deferred the item to a later time

Legacy shape (pre-#1878), retained for historical reference:

unread → in_review → resolved

Expert Actions

From the queue item detail view (/workspace/queue/[id]), an Expert can:

Action	Endpoint	Description
Approve & Send	`POST /expert-queue/:id/respond`	Send the AI draft verbatim as Specialist
Edit & Send	`POST /expert-queue/:id/respond`	Modify draft, then send as Specialist
Write & Send	`POST /expert-queue/:id/respond`	Discard draft, write own response
Resolve	`POST /expert-queue/:id/resolve`	Mark resolved with optional satisfaction score
Defer	`POST /expert-queue/:id/defer`	Defer the item by X hours
Add Note	`POST /expert-queue/:id/add-note`	Add internal note (never visible to client)
Get AI suggestion	`POST /expert-queue/:id/suggest`	Re-fetch AI draft on demand

Reassign: previously routed to an expert pool. Pool model was removed 2026-05-03. The reassign UX is being re-introduced on top of expert_access (ADR-007) — reassigning a thread means routing to a different Expert who already holds an expert_access grant for the same (org) or (org × specialist) slice.

Expert ↔ Data Access

Expert routing is grant-based via expert_access (ADR-007). Each Expert has one or more expert_access rows scoping them to either:

scope='org' — visibility into every thread on a given Org (broad grant), or
scope='specialist' — visibility into a single (Org × Specialist) slice (e.g. Acme's KYC Specialist threads only — Finance threads stay hidden).

Queue items are routed based on conversation.org_id + conversation.specialist_id ↔ the Expert's grants, with the application enforcing the union of all grants the Expert holds.

caution

Expert Pools Removed (2026-05-03) — org_experts Deprecated (2026-05-22)

Expert pools and the intermediate org_experts table are both deprecated. All new code must use ExpertAccessService. The legacy org_experts table remains read-only during soak (#540) and is dropped after.

Assignment is managed by SuperAdmin via /ops/team and the org detail /ops/clients/[id].

Confidence Threshold

Each Specialist has a confidenceThreshold (integer 0–101) set per Org via OrgSpecialistAssignment:

Value	Behavior
`0`	Always auto-send (any agent score ≥ 0 qualifies)
`1–100`	Auto-send if agent confidence ≥ threshold
`101`	Never auto-send — every response routes to Expert

Default is 101 (Never autoreply) until the AM explicitly changes it. AM configures this in wizard Step 2; Experts can adjust it post-onboarding. Clients never see or configure the threshold.

Risk Classification

The agent service classifies every response with a risk level. HIGH and CRITICAL override the confidence threshold — the item always routes to Expert review regardless of confidence score.

Risk Level	Auto-Send Eligible?
`low`	Yes (if confidence ≥ threshold)
`medium`	Yes (if confidence ≥ threshold)
`high`	No — always routes to Expert
`critical`	No — always routes to Expert

Current implementation: keyword heuristic classifier. Finance/IR roles have elevated base risk. LLM-based classification is planned for P2.

Real-Time Updates

WebSocket (primary): Socket.io NotificationsGateway
- Room experts — all experts platform-wide
- Room org:<id> — per-org experts
- Events: expert_queue_item_added, queue_item_resolved, agent_message_sent
Reconnect banner (fallback): when the Socket.io connection drops, the frontend renders a reconnect banner and re-fetches on reconnect. The previous 8-second HTTP polling fallback was removed in #843 (closes #804).

PR #244 (2026-05-07): ExpertQueueService.respond() now emits agent_message_sent after saving the expert reply, so the client portal receives the message in real-time over Socket.io.

Specialist Prompt Template Key

Each Specialist carries a prompt_template_key field (added in #1211, closes #1203) — passed into /v1/chat so the agent loads the correct role template (e.g. kyc_ops vs ecommerce_ops). Default role inheritance happens at conversation create time. This removes the previous hardcoded _DEFAULT_ROLE = 'ecommerce_ops' cross-org bleed where KYC clients were getting e-commerce-flavoured drafts.

The field, its allowed values (ecommerce_ops, finance_ops, ir_reporting, kyc_ops), default (NULL → role-chain fallback), isolation classification, and migration story are specified in ADR-034 — Specialist Prompt-Template Scoping.

Audit Fields on Every Write

Every message write — inbound from a channel, AI draft, expert reply via /respond, expert internal note via /add-note, queue /resolve — now carries the ADR-020 audit columns on the unified messages table:

actor_role — who wrote the row (client / agent / expert / system)
direction — inbound / outbound / internal
osa_id — the org_specialist_assignments FK; the pivot key for per-OSA billing rollup
cost_cents, token_input, token_output, model_name — populated when the row is LLM-derived
refines_queue_item_id, refinement_index — link back to the originating queue item for refinement chains

The expert_agent_messages shadow table was dropped in #1153 — there is now a single unified messages table with actor_role. See ADR-020 and the #1119 PR series (#1148–#1153). Per-OSA cost/billing rollup is via LlmCostRollupService (#1151). The audit write paths in ExpertQueueService and ConversationsService were retrofitted in #1185 / #1187 / #1150.

Expert Waitlist Promote / Decline

Marketing-funnel Expert applicants land in marketing_leads. SuperAdmin can promote or decline them from the SuperAdmin marketing surface via:

POST /admin/marketing/leads/experts/:id/promote — creates the Expert record + sends invite (#1200, closes #1084)
POST /admin/marketing/leads/experts/:id/decline — marks the lead declined

Queue Filtering

Experts can filter the queue by:

Status: pending / awaiting_client / resolved / snoozed (legacy filters in_progress map to pending)
Risk: critical / high / medium / low
Org: specific client org
Sort: Newest first (no toggle — always newest-first)

Agentic Task Approval

For multi-step agentic tasks, Experts must approve each step before execution. See agentic-tasks.md.

Endpoints:

GET /agentic/tasks — list tasks pending approval
POST /agentic/tasks/:id/steps/:stepId/approve
POST /agentic/tasks/:id/steps/:stepId/reject

Correction Capture

When an Expert sends a corrected response:

POST /expert-queue/:id/corrections
{
  "correctionType": "factual_error | tone | missing_step | wrong_action",
  "originalResponse": "...",
  "correctedResponse": "...",
  "notes": "..."
}

Stored in corrections table. Reviewed by SuperAdmin via GET /learning/corrections.

Per-Org SLA + Breach Alerting (#1508)

#1508 introduces a per-org SLA configuration for Expert response time, plus Slack-based breach alerting:

Configuration: AMs set a per-org SLA (response-time target in minutes) on the client detail page. Stored on organizations.sla_config (jsonb).
Tracking: the queue worker computes due_at = item.created_at + sla_minutes per queue item. The Expert workspace surfaces a per-item SLA countdown.
Breach alerting: when now() > due_at and the item is still unread or in_review, an alert is posted to the configured Slack channel (organizations.sla_config.alert_slack_channel). One alert per breach, deduped via a sla_breach_alerts audit row.
Surfacing in Ops: breaches show in /ops/clients/[id] (per-org) and in /ops/queue (cross-org) for SuperAdmin/AM triage.

SLA breach alerting is independent of the confidenceThreshold / risk classifier — it fires on time-to-touch, not on content properties.

API Reference

Method	Path	Auth	Description
GET	`/expert-queue`	Expert	List queue items (status, orgId filters)
GET	`/expert/orgs`	Expert	Get Expert's available orgs
GET	`/expert-queue/:id`	Expert	Fetch single queue item
POST	`/expert-queue/:id/suggest`	Expert	Get fresh AI suggestion
POST	`/expert-queue/:id/respond`	Expert	Send response (approve/edit/write)
POST	`/expert-queue/:id/resolve`	Expert	Resolve with satisfaction score
PATCH	`/expert-queue/:id/status`	Expert	Update status
POST	`/expert-queue/:id/defer`	Expert	Defer X hours
POST	`/expert-queue/:id/add-note`	Expert	Add internal note
POST	`/expert-queue/:id/corrections`	Expert	Submit correction

Conversation Lifecycle​

Queue Item Lifecycle​

Expert Actions​

Expert ↔ Data Access​

Confidence Threshold​

Risk Classification​

Real-Time Updates​

Specialist Prompt Template Key​

Audit Fields on Every Write​

Expert Waitlist Promote / Decline​

Queue Filtering​

Agentic Task Approval​

Correction Capture​

Per-Org SLA + Breach Alerting (#1508)​

API Reference​