ADR-032 — KB retrieval measurement and engine selection

Status: Accepted for retrieval evaluation and isolation; turn integration superseded by ADR-046 Originally accepted: 2026-06-25

Decision that remains

Haystack hybrid retrieval is the supported Humanwork KB search engine. Retrieval quality is measured with an offline judged-query harness, and first-party retrieval events may be stored for quality analysis under the existing Org × Specialist isolation boundary. Event collection and evaluation do not run as a mandatory pre-model stage.

The useful data rules remain:

• K1/K2 retrieval is always Specialist-scoped as well as org-scoped;
• customer query text stays inside the platform PII boundary;
• low-cardinality operational metrics belong in Prometheus;
• high-cardinality judged retrieval evidence belongs in the protected first-party data plane; and
• retrieval relevance and final-answer quality are evaluated separately.

Superseded turn integration

The former implementation ran hybrid retrieval in NestJS before every draft, injected a synthetic user-role KB block, carried kb_context_provided, and kept a second keyword self-fetch as a fallback. That design is retired.

Humanwork now sends Hermes only the actual inbound message and attachments. Hermes decides when knowledge is needed and invokes the available retrieval capability on demand. There is no pre-turn KB query, injected kb_context block, outage fallback fetch, or prompt-slot decision in the managed turn path.

Historical implementation detail remains in Git history; it is not live architecture or a compatibility path.