Consolidate Specialist Draft Retrieval onto the Hybrid Engine — Design

Date: 2026-06-25 Status: Approved design (Option A′), pre-implementation Issue: #3166 (P1) · Related: eval harness #3091, event collection #3189, KB-role hardening #3190

Problem

When a Specialist reply is drafted, two KB retrievals run and the weaker one reaches the model with system privilege:

• NestJS runs the hybrid + Cohere rerank retriever (HaystackRetrievalService.search(), conversations.service.ts:2300) — MRR 0.99 (rag/evals/retrieval/BASELINE.md) — and (since #3190) injects it into the prompt as a user-role history turn (prompt-injection hardening).
• The agent then also self-fetches keyword-only (agent/main.py:_load_kb_context → agent/kb_client.py → GET /kb/retrieval, KbRetrievalService) — MRR 0.47 — and renders it via format_kb_context into the system prompt (agent/chat_helpers.py:232).

Net: redundant double-retrieval per draft, and a weaker-engine KB block sitting in the system prompt.

What #3190 already fixed (changes the scope)

#3190 (merged after #3166 was filed) moved the NestJS hybrid result into a security-hardened user-role turn. So the hybrid result is already delivered to the agent in a good slot. The remaining bug is narrower than the original framing: the agent additionally self-fetches keyword and puts it into the system prompt.

Goal (Option A′ — honor #3190)

Make NestJS's existing hybrid retrieval the single source of the draft's KB context (it already is, in the user-role history turn), and stop the agent from self-fetching keyword. Keep the keyword path as an outage-only fallback. Do not move KB content into the system prompt (that would regress #3190).

This is the minimal change that satisfies the evidence-backed ACs:

• hybrid is the only KB content the model sees (keyword no longer fetched/injected),
• redundant retrieval removed,
• keyword retained as outage fallback,
• #3190's user-role privilege treatment preserved.

The prompt-slot/answer-quality question (#3166 hypothesis 3) remains a deferred follow-up needing an end-to-end eval that does not exist yet.

Approach chosen: A′ (vs original A / B / C)

• A′ (chosen): keep NestJS's hybrid→user-turn injection; agent skips its keyword self-fetch when NestJS signals KB was provided. Smallest change; preserves #3190; removes redundancy and the weak engine.
• A (original): pass hybrid content in a kb_context field that the agent renders as the labeled system-prompt block — rejected: reintroduces KB into system privilege, regressing #3190.
• B: agent calls the hybrid /v1/agent-api/context endpoint — rejected: leaves NestJS's fetch in place (redundancy) and adds an embed+rerank call on the agent's critical path.
• C: agent sniffs KB out of history — rejected: brittle string detection.

Wire contract — kb_context_provided flag

A new boolean on the agent chat request: kb_context_provided: bool = false.

Value	Meaning	Agent behavior
true	Hybrid retrieval genuinely completed — incl. a real 0-results hit	Skip _load_kb_context (trust hybrid; no keyword self-fetch)
false / absent	Outage (paused, or circuit-open / failOpen which search() tags degraded) — or a direct non-NestJS agent call	Fall back: self-fetch keyword via _load_kb_context

Gate on ctx.paused !== true && ctx.degraded !== true — i.e. genuine completion, not results.length. The subtlety: HaystackRetrievalService.search() fails open by returning { results: [], total: 0 } on circuit-open / caught errors, which is otherwise indistinguishable from a genuine 0-results hit. So search() now tags those fail-open returns with degraded: true (and the pause path keeps paused: true); a genuine empty success has neither. This lets us honor both halves of the contract: a real empty hybrid → true (trust it; don't resurrect the weak keyword engine), an outage → false (fall back to keyword). Gating on results.length > 0 would wrongly fall back on real empties; gating on !paused alone would wrongly not fall back on degraded outages.

Data flow (after)

inbound → conversations.service.ts
  ctx = haystackRetrieval.search(orgId, query, { specialistId: conv.specialistId, source:"kb", failOpen:true })
     completed (not paused, not degraded) → kbContextProvided = true
                          (results>0 also injected into history as a user turn, #3190)
     paused / degraded outage → kbContextProvided = false
  POST /v1/chat  { ..., kb_context_provided: kbContextProvided }
       ▼
  agent /chat | /chat/stream:
     req.kb_context_provided == true  → SKIP _load_kb_context  (hybrid already in history)
     req.kb_context_provided == false → _load_kb_context (keyword self-fetch)  // fallback

Components & changes

NestJS — api/src/conversations/conversations.service.ts

1. Add specialistId: conv.specialistId to the haystackRetrieval.search() call (~:2300) so the hybrid result is K1/K2-scoped (ADR-008), matching the keyword path's scoping. (Today the call omits it → org-wide; consolidating without this would widen what the Specialist sees.)
2. Compute kbContextProvided: true when the search was attempted and not paused (!ctx.paused); false on pause/failure. Keep the existing history.unshift({ role:"user", ... }) injection unchanged (#3190).
3. Pass kbContextProvided to the orgAgentClient.chat(...) / chatStream(...) call(s) on the draft path. Apply at each draft-producing call site that performs this retrieval (inventory in the plan).
4. No change to the kb_retrieval_event capture (#3189) — engine:"hybrid" is now truly the served engine; drop the pre-#3166 caveat comment if present.

NestJS — api/src/common/agent.client.ts

5. Add kbContextProvided?: boolean to AgentChatRequest, kb_context_provided?: boolean to AgentV1ChatPayload, and set it in buildV1ChatPayload from req.kbContextProvided (omit/false-default when unset, so non-draft callers are unaffected).

Agent — agent/models/chat.py, agent/routers/chat.py

6. Add kb_context_provided: bool = Field(default=False, validation_alias=AliasChoices("kb_context_provided","kbContextProvided")) to the v1 ChatRequest; map it through _to_legacy_request onto the legacy request.

Agent — agent/main.py

7. Add kb_context_provided: bool = False to the legacy ChatRequest. At the two _load_kb_context call sites (/chat ~:521 and /chat/stream ~:847): call it only when not req.kb_context_provided. When skipped, req.kb_context stays empty → format_kb_context renders nothing into the system prompt → no keyword content; the hybrid content already lives in the user-role history turn. chat_helpers.format_kb_context is unchanged (already returns "" for empty/None).

Error handling

• NestJS retrieval is already failOpen:true; on pause/failure it sends kb_context_provided:false → agent falls back to keyword (degraded but functional). Never surfaces a 500.
• Agent fallback path is the existing fail-open _load_kb_context.

Testing

• NestJS unit: specialistId is passed to search(); the chat payload carries kb_context_provided:true on a normal retrieval and false when the search reports paused. (History injection behavior is already covered by existing tests / #3190.)
• Agent unit: _load_kb_context is not called when kb_context_provided is true, and is called when false; format_kb_context still renders nothing extra into the system prompt in the provided case.
• Eval (AC#4): re-run the rag-eval-baseline workflow after the change to confirm the served engine.
• Answer-quality eval (AC#5) is an explicit follow-up, not built here.

Out of scope / deferred

• End-to-end answer-quality eval (faithfulness/answer-relevancy) — #3166 hypothesis 3.
• Removing KbRetrievalService / /kb/retrieval — retained as the fallback path.
• The unused /v1/agent-api/context endpoint — left as-is.
• Moving KB content between prompt slots — explicitly not done (preserves #3190).

Acceptance criteria (from #3166)

• The KB content the model uses is produced by the hybrid retriever — keyword self-fetch removed; hybrid (already in the user turn) is the only KB content.
• No redundant second retrieval — agent skips self-fetch when kb_context_provided.
• Keyword retrieval retained as documented outage fallback — kb_context_provided:false path.
• Re-run rag-eval-baseline after the change.
• Answer-quality impact tracked as follow-up (not a blocker).

Note on AC#1 wording: the original AC said "labeled block". Under A′ the hybrid content stays in the #3190-hardened user-role turn (still produced by formatKbContext, still labeled "Relevant knowledge base context:"), rather than the system-prompt block — a deliberate, more-secure placement. The substance (hybrid is the KB the model sees) is met.