KB Retrieval Event Collection — Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Persist a kb_retrieval_event row (query, engine, per-doc scores/ranks, correlation keys) for every drafted reply, written async via BullMQ, plus Prometheus health metrics — so retrieval quality can be monitored and joined to Expert outcomes.

Architecture: A dedicated Postgres table is the system of record. At draft-save in NestJS (where message_id + the scored ContextResult[] coexist), we synchronously bump low-cardinality Prometheus counters and enqueue a BullMQ job; the processor does an idempotent INSERT … ON CONFLICT DO NOTHING. Outcomes are derived later by joining on message_id/conversation_id — no outcome event is emitted.

Tech Stack: NestJS 11, TypeORM (Postgres 16 + pgvector; sqlite in local unit tests), BullMQ, prom-client.

Design spec: docs/superpowers/specs/2026-06-23-kb-retrieval-events-design.md

Conventions

All commands run from api/. Tests: npm test -- <pattern> (Jest; sqlite in-memory locally, Postgres in CI).
Commit after every green step. Messages: feat(kb-events): … / test(kb-events): ….
Local npm test uses sqlite; Postgres-only behavior (partial unique index ON CONFLICT) is asserted in a spec that self-skips on sqlite and runs in CI (per CLAUDE.md DB-driver note).

File structure (locked)

api/migrations/1799000000007-CreateKbRetrievalEvent.ts   # table + indexes
api/src/common/entities.ts                                # + KbRetrievalEvent entity
api/src/kb/kb-retrieval-event.builder.ts                  # pure mapping ContextResult[] -> row (TDD)
api/src/kb/kb-retrieval-event.builder.spec.ts
api/src/kb/kb-retrieval-event.service.ts                  # idempotent persist + Prometheus emit
api/src/kb/kb-retrieval-event.service.spec.ts
api/src/kb/kb-retrieval-event.processor.ts               # BullMQ worker
api/src/kb/kb-retrieval-event.types.ts                    # KbRetrievalEventInput + queue/job names
api/src/common/metrics.service.ts                         # + retrieval counters/histograms
api/src/conversations/conversations.module.ts             # register entity + queue
api/src/conversations/conversations.service.ts            # capture point: emit metrics + enqueue
api/test/kb-retrieval-event.pg.spec.ts                    # Postgres-only dedup test (CI)

Task 1: Migration — `kb_retrieval_event` table

Files:

Create: api/migrations/1799000000007-CreateKbRetrievalEvent.ts

This is a pure additive (expand) migration — safe in a single PR.

Step 1: Write the migration

import { MigrationInterface, QueryRunner } from "typeorm";

export class CreateKbRetrievalEvent1799000000007 implements MigrationInterface {
  name = "CreateKbRetrievalEvent1799000000007";

  public async up(queryRunner: QueryRunner): Promise<void> {
    await queryRunner.query(`
      CREATE TABLE IF NOT EXISTS "kb_retrieval_event" (
        "id" uuid PRIMARY KEY DEFAULT gen_random_uuid(),
        "org_id" uuid NOT NULL,
        "specialist_id" uuid NOT NULL,
        "conversation_id" uuid NOT NULL,
        "message_id" uuid NULL,
        "engine" text NOT NULL,
        "query" text NOT NULL,
        "top_k" integer NOT NULL,
        "result_count" integer NOT NULL,
        "latency_ms" integer NULL,
        "retrieved" jsonb NOT NULL DEFAULT '[]'::jsonb,
        "created_at" timestamptz NOT NULL DEFAULT now()
      )
    `);
    await queryRunner.query(
      `CREATE INDEX IF NOT EXISTS "kb_retrieval_event_org_spec_time" ON "kb_retrieval_event" ("org_id", "specialist_id", "created_at" DESC)`,
    );
    await queryRunner.query(
      `CREATE INDEX IF NOT EXISTS "kb_retrieval_event_conversation" ON "kb_retrieval_event" ("conversation_id")`,
    );
    await queryRunner.query(
      `CREATE UNIQUE INDEX IF NOT EXISTS "kb_retrieval_event_message_uq" ON "kb_retrieval_event" ("message_id") WHERE "message_id" IS NOT NULL`,
    );
  }

  public async down(queryRunner: QueryRunner): Promise<void> {
    await queryRunner.query(`DROP TABLE IF EXISTS "kb_retrieval_event"`);
  }
}

Step 2: Verify it compiles

Run: npm run build Expected: no TypeScript errors.

Step 3: Commit

git add api/migrations/1799000000007-CreateKbRetrievalEvent.ts
git commit -m "feat(kb-events): migration for kb_retrieval_event table"

Task 2: TypeORM entity

Files:

Modify: api/src/common/entities.ts (append a new entity; follow the existing @Entity style in this file)
Step 1: Add the entity

Append to api/src/common/entities.ts:

/**
 * KB retrieval event — one row per drafted reply's KB retrieval.
 *
 * Isolation (ADR-020): Org × Specialist. org_id + specialist_id NOT NULL;
 * service-layer writes always set both from the conversation context.
 * Append-only telemetry; joined to Message/Correction by message_id for
 * retrieval-quality analysis. See docs/superpowers/specs/2026-06-23-kb-retrieval-events-design.md
 */
@Entity("kb_retrieval_event")
export class KbRetrievalEvent {
  @PrimaryGeneratedColumn("uuid")
  id!: string;

  @Column({ name: "org_id", type: "uuid" })
  orgId!: string;

  @Column({ name: "specialist_id", type: "uuid" })
  specialistId!: string;

  @Column({ name: "conversation_id", type: "uuid" })
  conversationId!: string;

  @Column({ name: "message_id", type: "uuid", nullable: true })
  messageId!: string | null;

  @Column({ type: "text" })
  engine!: string;

  @Column({ type: "text" })
  query!: string;

  @Column({ name: "top_k", type: "int" })
  topK!: number;

  @Column({ name: "result_count", type: "int" })
  resultCount!: number;

  @Column({ name: "latency_ms", type: "int", nullable: true })
  latencyMs!: number | null;

  @Column({ type: "jsonb", default: () => "'[]'" })
  retrieved!: Array<{ doc_id: string; chunk_id: string | null; rank: number; score: number }>;

  @CreateDateColumn({ name: "created_at", type: "timestamptz" })
  createdAt!: Date;
}

Note: ensure PrimaryGeneratedColumn, CreateDateColumn, Column, Entity are already imported at the top of entities.ts (they are — used by other entities).

Step 2: Verify it compiles

Run: npm run build Expected: no errors.

Step 3: Commit

git add api/src/common/entities.ts
git commit -m "feat(kb-events): KbRetrievalEvent entity (Org × Specialist isolation)"

Task 3: Pure event-builder (TDD)

Files:

Create: api/src/kb/kb-retrieval-event.types.ts
Create: api/src/kb/kb-retrieval-event.builder.ts
Test: api/src/kb/kb-retrieval-event.builder.spec.ts

Maps the in-scope retrieval results + ids into a row. Pure, deterministic, no DB — the must-be-correct core. ContextResult (from api/src/haystack/haystack.types.ts) has: document_id, doc_id?, similarity, citation.hash?.

Step 1: Write the input type — api/src/kb/kb-retrieval-event.types.ts

import { ContextResult } from "../haystack/haystack.types";

export const KB_RETRIEVAL_EVENT_QUEUE = "kb-retrieval-event";
export const KB_RETRIEVAL_EVENT_JOB = "persist";

export interface KbRetrievalEventInput {
  orgId: string;
  specialistId: string;
  conversationId: string;
  messageId: string | null;
  engine: string;
  query: string;
  topK: number;
  latencyMs: number | null;
  results: ContextResult[];
}

export interface KbRetrievalEventRow {
  orgId: string;
  specialistId: string;
  conversationId: string;
  messageId: string | null;
  engine: string;
  query: string;
  topK: number;
  resultCount: number;
  latencyMs: number | null;
  retrieved: Array<{ doc_id: string; chunk_id: string | null; rank: number; score: number }>;
}

Step 2: Write the failing test — api/src/kb/kb-retrieval-event.builder.spec.ts

import { buildKbRetrievalEventRow } from "./kb-retrieval-event.builder";
import { KbRetrievalEventInput } from "./kb-retrieval-event.types";

function ctx(partial: Partial<any>): any {
  return {
    source: "kb",
    content: "",
    document_id: "doc-a",
    dataset_id: "ds",
    similarity: 0.9,
    citation: {},
    metadata: {},
    ...partial,
  };
}

const base: KbRetrievalEventInput = {
  orgId: "o1", specialistId: "s1", conversationId: "c1", messageId: "m1",
  engine: "hybrid", query: "how do refunds work?", topK: 5, latencyMs: 42, results: [],
};

test("maps results to ranked retrieved entries", () => {
  const row = buildKbRetrievalEventRow({
    ...base,
    results: [
      ctx({ document_id: "doc-a", similarity: 0.91, citation: { hash: "h1" } }),
      ctx({ document_id: "doc-b", similarity: 0.55, citation: {} }),
    ],
  });
  expect(row.resultCount).toBe(2);
  expect(row.retrieved).toEqual([
    { doc_id: "doc-a", chunk_id: "h1", rank: 1, score: 0.91 },
    { doc_id: "doc-b", chunk_id: null, rank: 2, score: 0.55 },
  ]);
});

test("prefers doc_id over document_id when present", () => {
  const row = buildKbRetrievalEventRow({ ...base, results: [ctx({ doc_id: "real", document_id: "fallback" })] });
  expect(row.retrieved[0].doc_id).toBe("real");
});

test("empty results -> resultCount 0, empty retrieved", () => {
  const row = buildKbRetrievalEventRow({ ...base, results: [] });
  expect(row.resultCount).toBe(0);
  expect(row.retrieved).toEqual([]);
});

test("passes through correlation keys + facts", () => {
  const row = buildKbRetrievalEventRow({ ...base, messageId: null });
  expect(row.orgId).toBe("o1");
  expect(row.specialistId).toBe("s1");
  expect(row.conversationId).toBe("c1");
  expect(row.messageId).toBeNull();
  expect(row.engine).toBe("hybrid");
  expect(row.query).toBe("how do refunds work?");
  expect(row.topK).toBe(5);
  expect(row.latencyMs).toBe(42);
});

Step 3: Run to confirm FAIL

Run: npm test -- kb-retrieval-event.builder Expected: FAIL (module not found).

Step 4: Implement — api/src/kb/kb-retrieval-event.builder.ts

import { KbRetrievalEventInput, KbRetrievalEventRow } from "./kb-retrieval-event.types";

export function buildKbRetrievalEventRow(input: KbRetrievalEventInput): KbRetrievalEventRow {
  const retrieved = input.results.map((r, index) => ({
    doc_id: r.doc_id ?? r.document_id,
    chunk_id: r.citation?.hash ?? null,
    rank: index + 1,
    score: r.similarity,
  }));
  return {
    orgId: input.orgId,
    specialistId: input.specialistId,
    conversationId: input.conversationId,
    messageId: input.messageId,
    engine: input.engine,
    query: input.query,
    topK: input.topK,
    resultCount: input.results.length,
    latencyMs: input.latencyMs,
    retrieved,
  };
}

Step 5: Run to confirm PASS

Run: npm test -- kb-retrieval-event.builder Expected: 4 passed.

Step 6: Commit

git add api/src/kb/kb-retrieval-event.types.ts api/src/kb/kb-retrieval-event.builder.ts api/src/kb/kb-retrieval-event.builder.spec.ts
git commit -m "feat(kb-events): pure event-builder mapping ContextResult -> row (TDD)"

Task 4: Persist service (idempotent insert + metrics emit)

Files:

Create: api/src/kb/kb-retrieval-event.service.ts
Test: api/src/kb/kb-retrieval-event.service.spec.ts
Step 1: Write the failing test — api/src/kb/kb-retrieval-event.service.spec.ts

import { Test } from "@nestjs/testing";
import { getRepositoryToken } from "@nestjs/typeorm";
import { KbRetrievalEvent } from "../common/entities";
import { KbRetrievalEventService } from "./kb-retrieval-event.service";
import { MetricsService } from "../common/metrics.service";
import { KbRetrievalEventRow } from "./kb-retrieval-event.types";

const row: KbRetrievalEventRow = {
  orgId: "o1", specialistId: "s1", conversationId: "c1", messageId: "m1",
  engine: "hybrid", query: "q", topK: 5, resultCount: 0, latencyMs: 10, retrieved: [],
};

test("persist() inserts ignoring conflicts and records metrics", async () => {
  const orIgnore = jest.fn().mockReturnThis();
  const execute = jest.fn().mockResolvedValue({});
  const qb = { insert: () => qb, into: () => qb, values: () => qb, orIgnore, execute };
  const repo = { createQueryBuilder: () => qb };
  const metrics = { recordKbRetrieval: jest.fn() };

  const moduleRef = await Test.createTestingModule({
    providers: [
      KbRetrievalEventService,
      { provide: getRepositoryToken(KbRetrievalEvent), useValue: repo },
      { provide: MetricsService, useValue: metrics },
    ],
  }).compile();

  const svc = moduleRef.get(KbRetrievalEventService);
  await svc.persist(row);

  expect(orIgnore).toHaveBeenCalled();
  expect(execute).toHaveBeenCalled();
  expect(metrics.recordKbRetrieval).toHaveBeenCalledWith("hybrid", 0, null, 10);
});

Step 2: Run to confirm FAIL

Run: npm test -- kb-retrieval-event.service Expected: FAIL (module not found).

Step 3: Implement — api/src/kb/kb-retrieval-event.service.ts

import { Injectable, Logger } from "@nestjs/common";
import { InjectRepository } from "@nestjs/typeorm";
import { Repository } from "typeorm";
import { KbRetrievalEvent } from "../common/entities";
import { MetricsService } from "../common/metrics.service";
import { KbRetrievalEventRow } from "./kb-retrieval-event.types";

@Injectable()
export class KbRetrievalEventService {
  private readonly logger = new Logger(KbRetrievalEventService.name);

  constructor(
    @InjectRepository(KbRetrievalEvent)
    private readonly repo: Repository<KbRetrievalEvent>,
    private readonly metrics: MetricsService,
  ) {}

  async persist(row: KbRetrievalEventRow): Promise<void> {
    const topScore = row.retrieved.length > 0 ? row.retrieved[0].score : null;
    this.metrics.recordKbRetrieval(row.engine, row.resultCount, topScore, row.latencyMs);
    // Idempotent under DLQ replay / multi-pod: partial unique index on message_id.
    await this.repo
      .createQueryBuilder()
      .insert()
      .into(KbRetrievalEvent)
      .values({
        orgId: row.orgId,
        specialistId: row.specialistId,
        conversationId: row.conversationId,
        messageId: row.messageId,
        engine: row.engine,
        query: row.query,
        topK: row.topK,
        resultCount: row.resultCount,
        latencyMs: row.latencyMs,
        retrieved: row.retrieved,
      })
      .orIgnore()
      .execute();
  }
}

Note: the metrics emit lives here so it fires once per persisted event (sync emit at the capture point is also added in Task 7 for the request-path counter; keep the call name consistent — recordKbRetrieval). To avoid double counting, emit metrics only at the capture point (Task 7), not in the service. Remove the this.metrics.recordKbRetrieval(...) line and the MetricsService dependency from this service if Task 7 emits synchronously. Decision for this plan: emit in the service (single emit point, fired when the event is actually persisted) and Task 7 does NOT emit. The test above asserts the service emits.

Step 4: Run to confirm PASS

Run: npm test -- kb-retrieval-event.service Expected: 1 passed.

Step 5: Commit

git add api/src/kb/kb-retrieval-event.service.ts api/src/kb/kb-retrieval-event.service.spec.ts
git commit -m "feat(kb-events): idempotent persist service + metrics emit (TDD)"

Task 5: Prometheus metrics

Files:

Modify: api/src/common/metrics.service.ts

Follow the existing pattern in this file (private readonly xTotal = new Counter({...}) + a public method).

Step 1: Add metrics + method

Add fields alongside the existing counters in MetricsService:

  private readonly kbRetrievalTotal = new Counter({
    name: "kb_retrieval_total",
    help: "KB retrievals performed, by engine",
    labelNames: ["engine"],
    registers: [this.registry],
  });

  private readonly kbRetrievalEmptyTotal = new Counter({
    name: "kb_retrieval_empty_total",
    help: "KB retrievals that returned zero documents, by engine",
    labelNames: ["engine"],
    registers: [this.registry],
  });

  private readonly kbRetrievalTopScore = new Histogram({
    name: "kb_retrieval_top_score",
    help: "Top relevance score per KB retrieval",
    labelNames: ["engine"],
    buckets: [0, 0.1, 0.3, 0.5, 0.7, 0.85, 0.95, 1],
    registers: [this.registry],
  });

  private readonly kbRetrievalLatencyMs = new Histogram({
    name: "kb_retrieval_latency_ms",
    help: "KB retrieval latency in milliseconds",
    labelNames: ["engine"],
    buckets: [50, 100, 250, 500, 1000, 2500, 5000, 10000],
    registers: [this.registry],
  });

  recordKbRetrieval(
    engine: string,
    resultCount: number,
    topScore: number | null,
    latencyMs: number | null,
  ): void {
    this.kbRetrievalTotal.inc({ engine });
    if (resultCount === 0) this.kbRetrievalEmptyTotal.inc({ engine });
    if (topScore !== null) this.kbRetrievalTopScore.observe({ engine }, topScore);
    if (latencyMs !== null) this.kbRetrievalLatencyMs.observe({ engine }, latencyMs);
  }

Use the registry reference the existing counters use (match registers: [...] to the pattern already present in the file — if existing counters omit registers, omit it here too and rely on the default registry).

Step 2: Verify build + existing metrics tests

Run: npm run build && npm test -- metrics Expected: builds; existing metrics tests pass.

Step 3: Commit

git add api/src/common/metrics.service.ts
git commit -m "feat(kb-events): Prometheus retrieval health metrics"

Task 6: BullMQ processor

Files:

Create: api/src/kb/kb-retrieval-event.processor.ts

Follow the existing processor pattern (e.g. api/src/conversations/*processor* or api/src/channels/*). Uses @Processor(KB_RETRIEVAL_EVENT_QUEUE).

Step 1: Implement

import { Processor, WorkerHost } from "@nestjs/bullmq";
import { Logger } from "@nestjs/common";
import { Job } from "bullmq";
import { KbRetrievalEventService } from "./kb-retrieval-event.service";
import { KbRetrievalEventRow, KB_RETRIEVAL_EVENT_QUEUE } from "./kb-retrieval-event.types";

@Processor(KB_RETRIEVAL_EVENT_QUEUE)
export class KbRetrievalEventProcessor extends WorkerHost {
  private readonly logger = new Logger(KbRetrievalEventProcessor.name);

  constructor(private readonly service: KbRetrievalEventService) {
    super();
  }

  async process(job: Job<KbRetrievalEventRow>): Promise<void> {
    try {
      await this.service.persist(job.data);
    } catch (err) {
      // Fail-open telemetry: log and swallow so a bad event never wedges the queue.
      this.logger.warn(`kb_retrieval_event persist failed: ${(err as Error).message}`);
    }
  }
}

Step 2: Verify build

Run: npm run build Expected: no errors.

Step 3: Commit

git add api/src/kb/kb-retrieval-event.processor.ts
git commit -m "feat(kb-events): BullMQ processor for async event persistence"

Task 7: Wire module + capture point

Files:

Modify: api/src/conversations/conversations.module.ts
Modify: api/src/conversations/conversations.service.ts
Step 1: Register entity, queue, providers in conversations.module.ts
Add KbRetrievalEvent to the existing TypeOrmModule.forFeature([...]) array.
Add a second BullModule.registerQueue({ name: KB_RETRIEVAL_EVENT_QUEUE }) (import the const from ../kb/kb-retrieval-event.types).
Add KbRetrievalEventService and KbRetrievalEventProcessor to providers.
Import MetricsService's module/provider if not already available to this module (it is global in most setups; if injection fails, add MetricsModule/CommonModule to imports).
Step 2: Inject the queue in conversations.service.ts constructor

// add import
import { getQueueToken } from "@nestjs/bullmq"; // already imported in module; service uses @InjectQueue
import { InjectQueue } from "@nestjs/bullmq";
import { Queue } from "bullmq";
import {
  KB_RETRIEVAL_EVENT_QUEUE,
  KB_RETRIEVAL_EVENT_JOB,
} from "../kb/kb-retrieval-event.types";
import { buildKbRetrievalEventRow } from "../kb/kb-retrieval-event.builder";

Add to the constructor parameters:

    @InjectQueue(KB_RETRIEVAL_EVENT_QUEUE)
    private readonly kbRetrievalEventQueue: Queue,

Step 3: Enqueue at draft-save — in conversations.service.ts, immediately after the draft agentMsg is saved (the await this.msgRepo.save(...) that sets metadata.retrievedKb, ~line 2454), add:

    // KB retrieval event (async, fail-open): correlate retrieval -> this draft.
    // jobId = message_id => cross-pod idempotency; processor INSERTs ON CONFLICT DO NOTHING.
    try {
      const eventRow = buildKbRetrievalEventRow({
        orgId: conv.orgId,
        specialistId: conv.specialistId,
        conversationId: id,
        messageId: agentMsg.id,
        engine: "hybrid", // NestJS-resolved Haystack retrieval; single engine post-#3166
        query: dto.content ?? "",
        topK: Number(process.env.HAYSTACK_TOP_K ?? process.env.RAGFLOW_TOP_K ?? 5),
        latencyMs: null,
        results: retrievedKbContext,
      });
      await this.kbRetrievalEventQueue.add(KB_RETRIEVAL_EVENT_JOB, eventRow, {
        jobId: `kbre:${agentMsg.id}`,
        removeOnComplete: true,
        removeOnFail: 1000,
      });
    } catch (err) {
      this.logger.warn(`failed to enqueue kb_retrieval_event: ${(err as Error).message}`);
    }

(retrievedKbContext, conv, id, dto are already in scope at this point.)

Step 4: Build + run conversations tests

Run: npm run build && npm test -- conversations Expected: builds; existing conversations specs pass (the new enqueue is wrapped in try/catch and uses an injected queue — provide a mock queue in any failing spec's providers if needed: { provide: getQueueToken(KB_RETRIEVAL_EVENT_QUEUE), useValue: { add: jest.fn() } }).

Step 5: Commit

git add api/src/conversations/conversations.module.ts api/src/conversations/conversations.service.ts
git commit -m "feat(kb-events): capture point — enqueue retrieval event at draft-save"

Task 8: Postgres-only idempotency test (CI)

Files:

Create: api/test/kb-retrieval-event.pg.spec.ts

Verifies the partial unique index dedups duplicate message_id inserts. Self-skips on sqlite (local), runs on the Postgres CI job.

Step 1: Write the test

import { DataSource } from "typeorm";
import { KbRetrievalEvent } from "../src/common/entities";
import { buildTestDataSource } from "./test-db.config";

const isPg = (process.env.DATABASE_TYPE ?? "") === "postgres";
const d = isPg ? describe : describe.skip;

d("kb_retrieval_event idempotency (Postgres)", () => {
  let ds: DataSource;
  beforeAll(async () => {
    ds = await buildTestDataSource([KbRetrievalEvent]);
  });
  afterAll(async () => {
    await ds?.destroy();
  });

  it("ON CONFLICT DO NOTHING dedups duplicate message_id", async () => {
    const repo = ds.getRepository(KbRetrievalEvent);
    const base = {
      orgId: "00000000-0000-0000-0000-000000000001",
      specialistId: "00000000-0000-0000-0000-000000000002",
      conversationId: "00000000-0000-0000-0000-000000000003",
      messageId: "00000000-0000-0000-0000-000000000004",
      engine: "hybrid", query: "q", topK: 5, resultCount: 0, latencyMs: 1, retrieved: [],
    };
    const insert = () =>
      repo.createQueryBuilder().insert().into(KbRetrievalEvent).values(base).orIgnore().execute();
    await insert();
    await insert();
    const count = await repo.count({ where: { messageId: base.messageId } });
    expect(count).toBe(1);
  });
});

Note: match buildTestDataSource to the actual helper in api/test/test-db.config.ts; if the helper signature differs, adapt the setup to how other *.pg.spec.ts/postgres specs in api/test/ build their datasource. If no such helper exists, follow the pattern an existing Postgres-only spec uses to obtain a DataSource.

Step 2: Run locally (expect skip)

Run: npm test -- kb-retrieval-event.pg Expected: skipped (sqlite local). It runs in the CI test-api Postgres job.

Step 3: Commit

git add api/test/kb-retrieval-event.pg.spec.ts
git commit -m "test(kb-events): Postgres partial-unique-index dedup (CI)"

Self-Review

Spec coverage:

Dedicated kb_retrieval_event Postgres table → Task 1/2. ✓
Correlation keys (message_id nullable + conversation_id + org/specialist) → entity + builder + capture (Tasks 2/3/7). ✓
Per-doc scores/ranks in retrieved jsonb → builder (Task 3). ✓
Capture at NestJS draft-save → Task 7. ✓
Async via BullMQ, after save, fail-open → Tasks 6/7. ✓
Idempotent ON CONFLICT DO NOTHING on partial unique index → service (Task 4) + migration (Task 1) + CI test (Task 8). ✓
Prometheus low-cardinality (engine only) → Task 5. ✓
query stored raw, result_count typed column → entity/migration. ✓
Tenancy docstring (ADR-020) → Task 2. ✓
Goal B/C join queries → documented in spec; no code needed in v1 (queries are SQL run against the table). Deferred items (frontend, QuerySource, dashboard, ClickHouse) intentionally not tasked. ✓

Placeholder scan: Tasks reference buildTestDataSource and existing processor/module patterns with a "match the actual helper" instruction — these are real existing-codebase hooks the implementer verifies, not invented APIs. No TBD/TODO.

Type consistency: recordKbRetrieval(engine, resultCount, topScore, latencyMs) defined in Task 5, called in Task 4 with the same signature. KbRetrievalEventRow shape consistent across builder (3), service (4), processor (6), capture (7). KB_RETRIEVAL_EVENT_QUEUE/KB_RETRIEVAL_EVENT_JOB defined in Task 3 types, used in 6/7. Metrics emitted once (service only; Task 7 enqueues, does not emit) — explicitly resolved in Task 4's note.

Decision locked: metrics are emitted in the service (Task 4), when the event is persisted — not at the capture point — to avoid double counting. Task 7 only enqueues.

Conventions​

File structure (locked)​

Task 1: Migration — kb_retrieval_event table​

Task 2: TypeORM entity​

Task 3: Pure event-builder (TDD)​

Task 4: Persist service (idempotent insert + metrics emit)​

Task 5: Prometheus metrics​

Task 6: BullMQ processor​

Task 7: Wire module + capture point​

Task 8: Postgres-only idempotency test (CI)​

Self-Review​

Conventions

File structure (locked)

Task 1: Migration — `kb_retrieval_event` table

Task 2: TypeORM entity

Task 3: Pure event-builder (TDD)

Task 4: Persist service (idempotent insert + metrics emit)

Task 5: Prometheus metrics

Task 6: BullMQ processor

Task 7: Wire module + capture point

Task 8: Postgres-only idempotency test (CI)

Self-Review