Skip to main content

P2.4 Golden-Answer Retrieval Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Published kb_golden_answer config-assets become retrievable by /chat's existing Haystack hybrid search (semantic match), are identified in the result set, and pinned to the top of the KB context block — with the config-asset base staying the sole truth source (Haystack holds only a derived index copy).

Architecture: On publish/rollback/override/discard/archive of a kb_golden_answer asset, the config-assets services enqueue a best-effort, post-commit BullMQ job carrying the golden's synthesized retrieval text + scope + marker meta in the job payload itself (so the processor never reverse-queries config-assets — dependency stays one-way config-assets → haystack, no cycle). A processor in HaystackModule calls the existing HaystackIngestionService.upload() (A1 path: writes an org_documents shadow row with status="approved" + parseStatus="ingested" so the golden goes through the retrieval fail-closed authorization branch, not the fail-open fallback) with metadata.kind="golden_answer". The rag _serialize_document allowlist is extended two keys so the marker survives retrieval serialization; conversations reads it and pins golden hits. Consistency is eventual/best-effort per ADR-019.

Tech Stack: NestJS 11 + TypeORM 0.3, BullMQ (@nestjs/bullmq WorkerHost), Haystack (pgvector) via HaystackIngestionService/HaystackRetrievalService, Python rag service (rag/pipelines/retrieval/pipeline_wrapper.py), Jest (better-sqlite3 local / Postgres CI).

Design source of truth: docs/superpowers/specs/2026-07-08-p2.4-golden-answer-retrieval-design.md (§4 定案 A1+B1; §3 约束 1 fail-open 安全语义; §7 Q1 影子行选型 status="approved"+parseStatus="ingested"+新 source golden_answer).

Scope note: This plan targets a long-lived branch feat/specialist-value-output-p2plus (base 490517fb, already contains the KB line PR #4066). It is NOT merged to dev per-phase — all remaining Phases accumulate into one PR (user decision 2026-07-08). Commit per task; do not open a PR at plan end.

Pre-existing test baseline: verify failures against the branch base point (490517fb), not against a clean tree — local npm test has ~21 known pre-existing failures unrelated to this work. Only NEW failures introduced by a task are blocking.

File Structure​

Create:

  • api/src/haystack/jobs/golden-answer-sync.processor.ts — BullMQ WorkerHost; queue name constant GOLDEN_ANSWER_SYNC_QUEUE; job payload type GoldenAnswerSyncJobData; process() does upload() on kind:"upsert" and deleteByMeta() on kind:"delete". Injects ONLY HaystackIngestionService (no config-assets dep).
  • api/src/haystack/jobs/golden-answer-sync.processor.spec.ts — processor unit test (hand-new, mocked ingestion, fake Job), incl. fail-open.
  • api/src/config-assets/golden-answer-sync.enqueuer.ts — pure helper buildGoldenSyncUpsertJob(...) / buildGoldenSyncDeleteJob(...) that turns a published version + scope into a GoldenAnswerSyncJobData (synthesizes retrieval text, computes contentHash). Kept pure + separately unit-tested so the enqueue sites stay thin.
  • api/src/config-assets/golden-answer-sync.enqueuer.spec.ts — pure-function unit test for the payload builder.

Modify:

  • rag/pipelines/retrieval/pipeline_wrapper.py:203-220 — add "kind" + "golden_asset_id" to the _serialize_document allowlist (2 lines).
  • api/src/common/entities.ts:1357-1367 — add "golden_answer" to OrgDocumentSource union (expand-only).
  • api/src/haystack/haystack.types.ts:65-86 — add optional isGolden?: boolean + goldenAssetId?: string to ContextResult.
  • api/src/haystack/haystack-retrieval.service.ts:338-364 — toContextResult reads metadata.kind === "golden_answer" → sets isGolden/goldenAssetId.
  • api/src/haystack/haystack.module.ts:73-83, 97-118 — register GOLDEN_ANSWER_SYNC_QUEUE + add GoldenAnswerSyncProcessor to providers.
  • api/src/config-assets/config-publish.service.ts:64-67, 154, 218, 293, 336 — @Optional() @InjectQueue(GOLDEN_ANSWER_SYNC_QUEUE); enqueue at the 4 post-commit points (publish=upsert, rollback=re-upsert the now-current published version OR delete if none, override=upsert, discard=no-op unless it discarded a published version — see Task 5).
  • api/src/config-assets/config-assets.service.ts:426-456 — same @Optional() @InjectQueue; enqueue delete at the archive post-commit audit point.
  • api/src/config-assets/config-assets.module.ts — (no import of HaystackModule needed — HaystackModule is @Global() and exports BullModule; the @InjectQueue resolves via the global module graph. Integration specs must import HaystackModule explicitly since @Global() doesn't apply in isolated TestingModules.)
  • api/src/conversations/conversations.service.ts:3006-3027 — after retrievedKbContext = ctx.results, stable-sort golden hits (isGolden === true) to the front before unshift.
  • api/src/conversations/CLAUDE.md, api/src/config-assets/CLAUDE.md, api/src/haystack/CLAUDE.md (create if absent) — document the golden retrieval path.

Dependency direction (locked): config-assets enqueues jobs onto a queue that HaystackModule owns; the processor lives in HaystackModule and depends only on HaystackIngestionService. The job payload is self-contained (carries synthesized text + scope + marker), so the processor never calls back into config-assets. No config-assets → haystack service import, no cycle.

Task 1: rag _serialize_document allowlist — carry golden markers through retrieval​

Why first: the rag change is backward-compatible (returning two extra meta keys never breaks existing consumers), so it can land ahead of everything else with zero risk. Without it, metadata.kind="golden_answer" is stripped at serialization (design §3 约束 2) and the NestJS side can never see the marker.

Files:

  • Modify: rag/pipelines/retrieval/pipeline_wrapper.py:203-220

  • Test: rag/tests/ (mirror an existing _serialize_document test if present; else add one)

  • Step 1: Read the current allowlist to confirm exact shape

Run: sed -n '182,221p' rag/pipelines/retrieval/pipeline_wrapper.py Expected: an 8-key allowlist (org_id, dataset, specialist_id, audience, jurisdiction, effective_date, version, filename).

  • Step 2: Write/extend the failing test

Add a test that a document whose meta contains kind and golden_asset_id serializes with those keys present. If rag/tests/ has an existing serialize test, extend it; otherwise create rag/tests/test_serialize_document.py:

def test_serialize_document_carries_golden_markers():
    doc = _make_doc(meta={
        "org_id": "o1", "dataset": "default-kb",
        "kind": "golden_answer", "golden_asset_id": "a1",
    })
    out = _serialize_document(doc)
    assert out["meta"]["kind"] == "golden_answer"
    assert out["meta"]["golden_asset_id"] == "a1"

(Adapt _make_doc / import path to the repo's existing test helpers — read a neighboring rag test first.)

  • Step 3: Run test to verify it fails

Run: cd rag && python -m pytest tests/test_serialize_document.py -v (or the repo's rag test command — check rag/ for pytest.ini/Makefile) Expected: FAIL — kind / golden_asset_id missing from serialized meta (stripped by allowlist).

  • Step 4: Add the two keys to the allowlist

In _serialize_document (pipeline_wrapper.py:203-220), add to the allowlisted meta keys:

        "kind",
        "golden_asset_id",

(Place them alongside the existing 8 keys — the exact list/set literal to edit is the one returning filename etc.)

  • Step 5: Run test to verify it passes

Run: cd rag && python -m pytest tests/test_serialize_document.py -v Expected: PASS.

  • Step 6: Commit
git add rag/pipelines/retrieval/pipeline_wrapper.py rag/tests/
git commit -m "feat(rag): carry golden_answer markers through retrieval serialization (P2.4)"

Task 2: golden-answer sync job payload type + pure enqueuer helper​

Files:

  • Create: api/src/config-assets/golden-answer-sync.enqueuer.ts
  • Test: api/src/config-assets/golden-answer-sync.enqueuer.spec.ts

Design: The job payload is fully self-contained so the processor (Task 3) needs no config-assets access. contentHash is computed here (not in the processor) so it is deterministic from the published version content and drives the idempotent haystack_doc_id = ${orgId}:default-kb:${contentHash}.

  • Step 1: Write the failing test
import { computeContentHash } from "./config-assets.util";
import {
  buildGoldenSyncUpsertJob,
  buildGoldenSyncDeleteJob,
  type GoldenAnswerSyncJobData,
} from "./golden-answer-sync.enqueuer";

describe("golden-answer-sync enqueuer", () => {
  const base = {
    orgId: "00000000-0000-0000-0000-0000000000a1",
    specialistId: "00000000-0000-0000-0000-0000000000b1",
    assetId: "00000000-0000-0000-0000-0000000000d1",
    versionId: "00000000-0000-0000-0000-0000000000e1",
  };
  const content = { question: "Can I get a refund?", answer: "Yes, within 30 days.", sources: [] };

  test("upsert job synthesizes question+answer text and carries scope + markers", () => {
    const job = buildGoldenSyncUpsertJob({ ...base, content });
    expect(job.kind).toBe("upsert");
    expect(job.orgId).toBe(base.orgId);
    expect(job.specialistId).toBe(base.specialistId);
    expect(job.text).toContain("Can I get a refund?");
    expect(job.text).toContain("Yes, within 30 days.");
    expect(job.metadata.kind).toBe("golden_answer");
    expect(job.metadata.golden_asset_id).toBe(base.assetId);
    expect(job.metadata.golden_version_id).toBe(base.versionId);
    // deterministic hash over the synthesized text
    expect(job.contentHash).toBe(computeContentHash({ text: job.text }));
  });

  test("delete job carries orgId + asset marker only", () => {
    const job = buildGoldenSyncDeleteJob({ orgId: base.orgId, assetId: base.assetId });
    expect(job.kind).toBe("delete");
    expect(job.orgId).toBe(base.orgId);
    expect(job.metadata.golden_asset_id).toBe(base.assetId);
  });

  test("idempotency key is stable for identical content", () => {
    const a = buildGoldenSyncUpsertJob({ ...base, content });
    const b = buildGoldenSyncUpsertJob({ ...base, content });
    expect(a.contentHash).toBe(b.contentHash);
  });
});
  • Step 2: Run test to verify it fails

Run: cd api && npx jest src/config-assets/golden-answer-sync.enqueuer.spec.ts -v Expected: FAIL — module ./golden-answer-sync.enqueuer not found.

  • Step 3: Write the implementation
// api/src/config-assets/golden-answer-sync.enqueuer.ts
import { computeContentHash } from "./config-assets.util";

/** Self-contained payload for the golden-answer Haystack sync queue (P2.4).
 *  Carries everything the processor needs so it never reverse-queries
 *  config-assets — keeps the dependency direction one-way. */
export interface GoldenAnswerSyncJobData {
  kind: "upsert" | "delete";
  orgId: string;
  specialistId: string | null;
  /** Synthesized retrieval text (question + answer). Empty for delete. */
  text: string;
  /** Deterministic hash over the synthesized text; drives haystack_doc_id. */
  contentHash: string;
  metadata: {
    kind: "golden_answer";
    golden_asset_id: string;
    golden_version_id?: string;
  };
}

/** Minimal shape of KbGoldenAnswerContent this helper reads (avoid importing
 *  the Zod type — the payload only needs question/answer text). */
interface GoldenContentLike {
  question: string;
  answer: string;
}

function synthesizeRetrievalText(content: GoldenContentLike): string {
  // Question drives recall (matches the user query); answer text also
  // contributes semantic recall (design §7 Q2). Presentation later uses the
  // full golden content from the config-asset truth source, not this text.
  return `${content.question}\n\n${content.answer}`.trim();
}

export function buildGoldenSyncUpsertJob(params: {
  orgId: string;
  specialistId: string | null;
  assetId: string;
  versionId: string;
  content: GoldenContentLike;
}): GoldenAnswerSyncJobData {
  const text = synthesizeRetrievalText(params.content);
  return {
    kind: "upsert",
    orgId: params.orgId,
    specialistId: params.specialistId,
    text,
    contentHash: computeContentHash({ text }),
    metadata: {
      kind: "golden_answer",
      golden_asset_id: params.assetId,
      golden_version_id: params.versionId,
    },
  };
}

export function buildGoldenSyncDeleteJob(params: {
  orgId: string;
  assetId: string;
}): GoldenAnswerSyncJobData {
  return {
    kind: "delete",
    orgId: params.orgId,
    specialistId: null,
    text: "",
    contentHash: "",
    metadata: { kind: "golden_answer", golden_asset_id: params.assetId },
  };
}
  • Step 4: Run test to verify it passes

Run: cd api && npx jest src/config-assets/golden-answer-sync.enqueuer.spec.ts -v Expected: PASS (all 3).

  • Step 5: Commit
git add api/src/config-assets/golden-answer-sync.enqueuer.ts api/src/config-assets/golden-answer-sync.enqueuer.spec.ts
git commit -m "feat(config-assets): golden-answer Haystack sync job payload + pure enqueuer (P2.4)"

Task 3: GoldenAnswerSyncProcessor — consumes the queue, calls upload()/deleteByMeta()​

Files:

  • Create: api/src/haystack/jobs/golden-answer-sync.processor.ts
  • Test: api/src/haystack/jobs/golden-answer-sync.processor.spec.ts

Design: Mirrors CorrectionRefinementProcessor (WorkerHost base). Injects ONLY HaystackIngestionService. On upsert → upload(orgId, "default-kb", syntheticFile, { source:"golden_answer", status:"approved", contentHash, specialistId, metadata }). On delete → deleteByMeta(orgId, { golden_asset_id }). Fail-open: an error is logged, never thrown (best-effort, ADR-019). upload() writes parseStatus="ingested" on success, so the shadow row passes the retrieval fail-closed authorization branch (design §3 约束 1).

  • Step 1: Write the failing test (mirror kb-retrieval-event.processor.spec.ts)
import { Job } from "bullmq";
import { GoldenAnswerSyncProcessor } from "./golden-answer-sync.processor";
import type { GoldenAnswerSyncJobData } from "../../config-assets/golden-answer-sync.enqueuer";
import type { HaystackIngestionService } from "../haystack-ingestion.service";

function processorWith(overrides: Partial<Record<"upload" | "deleteByMeta", jest.Mock>>) {
  const upload = overrides.upload ?? jest.fn().mockResolvedValue({ id: "doc1" });
  const deleteByMeta = overrides.deleteByMeta ?? jest.fn().mockResolvedValue(undefined);
  const svc = { upload, deleteByMeta } as unknown as HaystackIngestionService;
  return { proc: new GoldenAnswerSyncProcessor(svc), upload, deleteByMeta };
}

const upsertJob: GoldenAnswerSyncJobData = {
  kind: "upsert",
  orgId: "00000000-0000-0000-0000-0000000000a1",
  specialistId: "00000000-0000-0000-0000-0000000000b1",
  text: "Can I get a refund?\n\nYes, within 30 days.",
  contentHash: "hash123",
  metadata: { kind: "golden_answer", golden_asset_id: "asset1", golden_version_id: "ver1" },
};

const deleteJob: GoldenAnswerSyncJobData = {
  kind: "delete",
  orgId: "00000000-0000-0000-0000-0000000000a1",
  specialistId: null,
  text: "",
  contentHash: "",
  metadata: { kind: "golden_answer", golden_asset_id: "asset1" },
};

describe("GoldenAnswerSyncProcessor", () => {
  test("upsert calls upload with default-kb dataset, approved status, golden markers", async () => {
    const { proc, upload } = processorWith({});
    await proc.process({ data: upsertJob } as Job<GoldenAnswerSyncJobData>);
    expect(upload).toHaveBeenCalledTimes(1);
    const [orgId, dataset, file, attrs] = upload.mock.calls[0];
    expect(orgId).toBe(upsertJob.orgId);
    expect(dataset).toBe("default-kb");
    expect(file.buffer.toString("utf-8")).toBe(upsertJob.text);
    expect(attrs.source).toBe("golden_answer");
    expect(attrs.status).toBe("approved");
    expect(attrs.contentHash).toBe("hash123");
    expect(attrs.specialistId).toBe(upsertJob.specialistId);
    expect(attrs.metadata.kind).toBe("golden_answer");
    expect(attrs.metadata.golden_asset_id).toBe("asset1");
  });

  test("delete calls deleteByMeta with golden_asset_id filter", async () => {
    const { proc, deleteByMeta } = processorWith({});
    await proc.process({ data: deleteJob } as Job<GoldenAnswerSyncJobData>);
    expect(deleteByMeta).toHaveBeenCalledWith(deleteJob.orgId, { golden_asset_id: "asset1" });
  });

  test("fail-open: an upload error does not throw", async () => {
    const { proc } = processorWith({ upload: jest.fn().mockRejectedValue(new Error("haystack down")) });
    await expect(proc.process({ data: upsertJob } as Job<GoldenAnswerSyncJobData>)).resolves.toBeUndefined();
  });
});
  • Step 2: Run test to verify it fails

Run: cd api && npx jest src/haystack/jobs/golden-answer-sync.processor.spec.ts -v Expected: FAIL — module not found.

  • Step 3: Write the implementation
// api/src/haystack/jobs/golden-answer-sync.processor.ts
import { Processor, WorkerHost } from "@nestjs/bullmq";
import { Logger } from "@nestjs/common";
import type { Job } from "bullmq";
import type { GoldenAnswerSyncJobData } from "../../config-assets/golden-answer-sync.enqueuer";
import { HaystackIngestionService } from "../haystack-ingestion.service";

export const GOLDEN_ANSWER_SYNC_QUEUE = "golden-answer-sync";

/**
 * Syncs published kb_golden_answer config-assets into a Haystack derived index
 * (P2.4, design A1). Best-effort / fail-open per ADR-019 — a Haystack outage
 * leaves the golden temporarily unretrievable (next publish/retry backfills),
 * it NEVER rolls back the publish. The job payload is self-contained so this
 * processor never reverse-queries config-assets (one-way dependency).
 */
@Processor(GOLDEN_ANSWER_SYNC_QUEUE)
export class GoldenAnswerSyncProcessor extends WorkerHost {
  private readonly logger = new Logger(GoldenAnswerSyncProcessor.name);

  constructor(private readonly ingestionService: HaystackIngestionService) {
    super();
  }

  async process(job: Job<GoldenAnswerSyncJobData>): Promise<void> {
    const data = job.data;
    try {
      if (data.kind === "delete") {
        await this.ingestionService.deleteByMeta(data.orgId, {
          golden_asset_id: data.metadata.golden_asset_id,
        });
        return;
      }
      const buffer = Buffer.from(data.text, "utf-8");
      await this.ingestionService.upload(
        data.orgId,
        "default-kb",
        {
          originalname: `golden-${data.metadata.golden_asset_id}.md`,
          buffer,
          mimetype: "text/markdown",
          size: buffer.byteLength,
        } as Express.Multer.File,
        {
          source: "golden_answer",
          status: "approved",
          contentHash: data.contentHash,
          specialistId: data.specialistId,
          metadata: data.metadata,
        },
      );
    } catch (err) {
      // Fail-open (ADR-019): best-effort derived-index sync. Never throw —
      // that would fail the BullMQ job into retries but must never surface to
      // the publish path (which already committed).
      this.logger.warn(
        `golden-answer sync failed (kind=${data.kind}, asset=${data.metadata.golden_asset_id}): ${(err as Error).message}`,
      );
    }
  }
}
  • Step 4: Run test to verify it passes

Run: cd api && npx jest src/haystack/jobs/golden-answer-sync.processor.spec.ts -v Expected: PASS (all 3).

  • Step 5: Commit
git add api/src/haystack/jobs/golden-answer-sync.processor.ts api/src/haystack/jobs/golden-answer-sync.processor.spec.ts
git commit -m "feat(haystack): golden-answer sync processor (upload/deleteByMeta, fail-open) (P2.4)"

Task 4: add golden_answer to OrgDocumentSource (expand-only enum)​

Files:

  • Modify: api/src/common/entities.ts:1357-1367

  • Step 1: Read the current union

Run: cd api && sed -n '1357,1367p' src/common/entities.ts Expected: a 10-value string union type OrgDocumentSource.

  • Step 2: Add the value

Append | "golden_answer" to the OrgDocumentSource union (expand-only — no migration needed, the column is text; existing rows unaffected). Add an inline comment: // P2.4: golden-answer derived-index shadow row.

  • Step 3: Verify it compiles

Run: cd api && npx tsc --noEmit Expected: no new errors (the processor in Task 3 uses source: "golden_answer" which now type-checks).

  • Step 4: Commit
git add api/src/common/entities.ts
git commit -m "feat(config-assets): add golden_answer OrgDocumentSource value (P2.4)"

Task 5: enqueue sync at the 5 config-asset lifecycle transitions​

Files:

  • Modify: api/src/config-assets/config-publish.service.ts (constructor + 4 post-commit points)
  • Modify: api/src/config-assets/config-assets.service.ts (constructor + archive post-commit point)
  • Test: extend api/src/config-assets/config-publish.service.spec.ts (verify enqueue is called for a golden asset, NOT for a soul asset)

Design decisions (from Explore anchors + design §5):

  • Enqueue ONLY when asset.assetType === "golden_answer" (no-op for soul/skill/domain-rule).

  • Inject the queue as @Optional() @InjectQueue(GOLDEN_ANSWER_SYNC_QUEUE) (per run-artifact.service.ts:148 — so hand-wired specs and Redis-less test modules still construct; undefined queue → enqueue is a no-op).

  • Enqueue is post-commit, best-effort (.catch swallow, mirror run-artifact.service.ts:495).

  • Transition → job mapping:

    • requestPublish → upsert (the version just published; content in hand at L146-148).
    • overridePublish → upsert (same).
    • rollback → upsert the version that is now published after rollback (rollback switches the pointer to targetVersionId; sync the new current published content). If rollback leaves no published version, delete.
    • discardDraft → no-op (discard only ever removes a draft, never a published version — a published golden is untouched, so its Haystack copy stays correct). Add a code comment stating this so a future reader doesn't "fix" it.
    • archiveAsset → delete (archived asset must not remain retrievable).

  • Step 1: Write the failing test (add to config-publish.service.spec.ts)

The existing spec hand-news ConfigPublishService(audit, ds). Add a mock queue and a golden asset, and assert enqueue behavior. Add near the existing describe blocks:

import { buildGoldenSyncUpsertJob } from "./golden-answer-sync.enqueuer";
import { GOLDEN_ANSWER_SYNC_QUEUE } from "../haystack/jobs/golden-answer-sync.processor";

// a minimal valid KbGoldenAnswerContent (mirror validSoul in this file)
const validGolden = {
  question: "Can I get a refund?",
  answer: "Yes, within 30 days of purchase.",
  sources: [],
  applicability: {},
};

describe("ConfigPublishService golden-answer Haystack sync enqueue", () => {
  let goldenQueue: { add: jest.Mock };
  let pubWithQueue: ConfigPublishService;

  beforeEach(() => {
    goldenQueue = { add: jest.fn().mockResolvedValue(undefined) };
    // constructor gains an optional 3rd arg (the queue); see Step 3
    pubWithQueue = new ConfigPublishService(
      audit as unknown as AuditService,
      ds,
      goldenQueue as unknown as any,
    );
  });

  async function seedGoldenDraft() {
    return assetsSvc.createAsset({
      assetType: "kb_golden_answer",
      scope: "org_instance",
      orgId: ORG,
      specialistId: SPEC,
      slug: "refund-policy",
      displayName: "Refund policy",
      content: validGolden,
      actor: USER,
    });
  }

  test("publish enqueues an upsert for a golden asset", async () => {
    const a = await seedGoldenDraft();
    await pubWithQueue.requestPublish(a.id, a.draftVersionId!, USER, CTX);
    expect(goldenQueue.add).toHaveBeenCalledTimes(1);
    const [jobName, data] = goldenQueue.add.mock.calls[0];
    expect(data.kind).toBe("upsert");
    expect(data.orgId).toBe(ORG);
    expect(data.metadata.golden_asset_id).toBe(a.id);
  });

  test("publish does NOT enqueue for a soul asset", async () => {
    const a = await seedAssetWithDraft(); // soul, existing helper
    await pubWithQueue.requestPublish(a.id, a.draftVersionId!, USER, CTX);
    expect(goldenQueue.add).not.toHaveBeenCalled();
  });

  test("enqueue is best-effort: a queue.add rejection does not fail publish", async () => {
    goldenQueue.add.mockRejectedValue(new Error("redis down"));
    const a = await seedGoldenDraft();
    await expect(
      pubWithQueue.requestPublish(a.id, a.draftVersionId!, USER, CTX),
    ).resolves.toBeDefined(); // publish still succeeds
  });
});
  • Step 2: Run test to verify it fails

Run: cd api && npx jest src/config-assets/config-publish.service.spec.ts -t "golden-answer Haystack sync" -v Expected: FAIL — ConfigPublishService constructor takes 2 args; enqueue not wired.

  • Step 3: Wire the queue into ConfigPublishService

Constructor (config-publish.service.ts:64-67) — add optional queue:

  constructor(
    private readonly audit: AuditService,
    private readonly dataSource: DataSource,
    @Optional()
    @InjectQueue(GOLDEN_ANSWER_SYNC_QUEUE)
    private readonly goldenSyncQueue?: Queue<GoldenAnswerSyncJobData>,
  ) {}

Add imports at top of file:

import { Optional } from "@nestjs/common";
import { InjectQueue } from "@nestjs/bullmq";
import type { Queue } from "bullmq";
import {
  GOLDEN_ANSWER_SYNC_QUEUE,
  type GoldenAnswerSyncProcessor, // (type-only, not used at runtime)
} from "../haystack/jobs/golden-answer-sync.processor";
import {
  buildGoldenSyncUpsertJob,
  buildGoldenSyncDeleteJob,
  type GoldenAnswerSyncJobData,
} from "./golden-answer-sync.enqueuer";

Add a private best-effort helper:

  /** Post-commit, best-effort golden-answer Haystack sync (ADR-019). No-op
   *  for non-golden assets and when no queue is wired (tests / Redis-less). */
  private enqueueGoldenSync(
    asset: SpecialistConfigAsset,
    version: SpecialistConfigVersion,
    op: "upsert" | "delete",
  ): void {
    if (asset.assetType !== "kb_golden_answer" || !this.goldenSyncQueue) return;
    const job =
      op === "delete"
        ? buildGoldenSyncDeleteJob({ orgId: asset.orgId!, assetId: asset.id })
        : buildGoldenSyncUpsertJob({
            orgId: asset.orgId!,
            specialistId: asset.specialistId,
            assetId: asset.id,
            versionId: version.id,
            content: version.content as { question: string; answer: string },
          });
    void this.goldenSyncQueue
      .add(op, job)
      .catch((err) =>
        this.logger.warn(
          `golden sync enqueue failed (asset=${asset.id}): ${(err as Error).message}`,
        ),
      );
  }

(If ConfigPublishService has no logger, add private readonly logger = new Logger(ConfigPublishService.name); and import Logger.)

  • Step 4: Call the helper at the 4 post-commit points

  • requestPublish L154 (after await this.emitAudits(...)): this.enqueueGoldenSync(asset, version, "upsert");

  • overridePublish L218 (after emitAudits): this.enqueueGoldenSync(asset, version, "upsert");

  • rollback L293 (after emitAudits): this.enqueueGoldenSync(asset, version, "upsert"); — version here is the now-current published version returned by rollback.

  • discardDraft L336: add a comment // no golden sync — discard only removes a draft; a published golden's Haystack copy is unaffected. (no call)

  • Step 5: Run test to verify publish/soul/best-effort pass

Run: cd api && npx jest src/config-assets/config-publish.service.spec.ts -t "golden-answer Haystack sync" -v Expected: PASS (3).

  • Step 6: Wire archive enqueue in ConfigAssetsService

Add the same @Optional() @InjectQueue(GOLDEN_ANSWER_SYNC_QUEUE) constructor param to ConfigAssetsService (config-assets.service.ts) and, in archiveAsset after the this.audit.tryLog({ action: "config.archive", ... }) at L444, enqueue a delete when asset.assetType === "kb_golden_answer":

    if (asset.assetType === "kb_golden_answer" && this.goldenSyncQueue) {
      void this.goldenSyncQueue
        .add("delete", buildGoldenSyncDeleteJob({ orgId: asset.orgId!, assetId: asset.id }))
        .catch((err) =>
          this.logger.warn(`golden sync (archive) enqueue failed: ${(err as Error).message}`),
        );
    }

Add a matching test to config-assets.service.spec.ts (hand-new ConfigAssetsService with a mock queue; archive a golden asset; assert add("delete", ...) called with the asset id; archive a soul asset; assert not called).

  • Step 7: Run the full config-assets spec suite

Run: cd api && npx jest src/config-assets/ -v Expected: PASS — all golden-sync tests + no regression in existing config-assets/config-publish tests.

  • Step 8: Commit
git add api/src/config-assets/config-publish.service.ts api/src/config-assets/config-assets.service.ts api/src/config-assets/config-publish.service.spec.ts api/src/config-assets/config-assets.service.spec.ts
git commit -m "feat(config-assets): enqueue golden-answer Haystack sync at publish/override/rollback/archive (P2.4)"

Task 6: register the queue + processor in HaystackModule​

Files:

  • Modify: api/src/haystack/haystack.module.ts:73-83 (registerQueue) + :97-118 (providers)

  • Step 1: Register the queue

After the two existing BullModule.registerQueue({...}) calls (haystack.module.ts:76-83), add:

    BullModule.registerQueue({
      name: GOLDEN_ANSWER_SYNC_QUEUE,
      connection: buildHaystackRedisConnection(),
    }),

Import the constant: import { GOLDEN_ANSWER_SYNC_QUEUE, GoldenAnswerSyncProcessor } from "./jobs/golden-answer-sync.processor";

  • Step 2: Add the processor to providers

In the providers array (L97-118), add GoldenAnswerSyncProcessor alongside CorrectionRefinementProcessor and ArtifactIngestionProcessor.

  • Step 3: Verify the app module graph boots (compile + a smoke spec)

Run: cd api && npx tsc --noEmit && npx jest src/haystack/ -v Expected: compiles; haystack specs pass. If there is an app-level boot spec (app.module.spec.ts / main smoke), run it to confirm no DI resolution error from the new provider.

  • Step 4: Commit
git add api/src/haystack/haystack.module.ts
git commit -m "feat(haystack): register golden-answer-sync queue + processor (P2.4)"

Task 7: ContextResult.isGolden + toContextResult reads the marker​

Files:

  • Modify: api/src/haystack/haystack.types.ts:65-86

  • Modify: api/src/haystack/haystack-retrieval.service.ts:338-364

  • Test: api/src/haystack/haystack-retrieval.service.spec.ts (or a focused toContextResult test)

  • Step 1: Write the failing test

Find the existing retrieval-service spec; add a toContextResult case (if toContextResult is private, test it through the public search() with a mocked Haystack client returning a chunk whose meta.kind === "golden_answer", OR make the test drive a small exported helper). Minimal shape:

test("toContextResult flags a golden chunk", () => {
  const chunk = {
    doc_id: "d1", chunk_id: "c1", content: "Refund policy...",
    score: 0.9,
    meta: { dataset: "default-kb", kind: "golden_answer", golden_asset_id: "asset1" },
  };
  const r = (svc as any).toContextResult(chunk);
  expect(r.isGolden).toBe(true);
  expect(r.goldenAssetId).toBe("asset1");
});

test("toContextResult leaves a normal KB chunk unflagged", () => {
  const chunk = { doc_id: "d2", chunk_id: "c2", content: "FAQ", score: 0.5, meta: { dataset: "default-kb" } };
  const r = (svc as any).toContextResult(chunk);
  expect(r.isGolden).toBe(false);
});
  • Step 2: Run test to verify it fails

Run: cd api && npx jest src/haystack/haystack-retrieval.service.spec.ts -t "golden" -v Expected: FAIL — isGolden undefined.

  • Step 3: Add the typed fields to ContextResult

In haystack.types.ts ContextResult interface (L65-86), add:

  /** P2.4: true when this hit is a golden-answer derived-index copy. */
  isGolden?: boolean;
  goldenAssetId?: string;
  • Step 4: Set them in toContextResult

In haystack-retrieval.service.ts toContextResult (L338-364), before the return {:

    const isGolden = metadata.kind === "golden_answer";

and add to the returned object:

      isGolden,
      goldenAssetId: isGolden ? (metadata.golden_asset_id as string | undefined) : undefined,
  • Step 5: Run test to verify it passes

Run: cd api && npx jest src/haystack/haystack-retrieval.service.spec.ts -t "golden" -v Expected: PASS (2).

  • Step 6: Commit
git add api/src/haystack/haystack.types.ts api/src/haystack/haystack-retrieval.service.ts api/src/haystack/haystack-retrieval.service.spec.ts
git commit -m "feat(haystack): flag golden-answer hits on ContextResult (P2.4)"

Task 8: pin golden hits to the top of the KB context block in /chat​

Files:

  • Modify: api/src/conversations/conversations.service.ts:3006-3027
  • Test: api/src/conversations/ — add a focused unit test for the sort (extract the sort into a testable pure helper if the block is hard to unit-test directly).

Design: golden and hybrid come back in ONE retrieval (design §6 ⑦ = in-retrieval ordering, not two retrievals). A stable sort moves isGolden hits to the front, preserving relative order within each group, then the existing unshift/formatKbContext path is unchanged.

  • Step 1: Write the failing test (pure sort helper)

Add an exported pure helper + test:

// test
import { pinGoldenFirst } from "./kb-context.util"; // new pure helper
test("pinGoldenFirst moves golden hits ahead, stable within groups", () => {
  const input = [
    { document_id: "a", isGolden: false },
    { document_id: "b", isGolden: true },
    { document_id: "c", isGolden: false },
    { document_id: "d", isGolden: true },
  ] as any[];
  const out = pinGoldenFirst(input);
  expect(out.map((r) => r.document_id)).toEqual(["b", "d", "a", "c"]);
});
  • Step 2: Run test to verify it fails

Run: cd api && npx jest src/conversations/kb-context.util.spec.ts -v Expected: FAIL — module not found.

  • Step 3: Implement the pure helper
// api/src/conversations/kb-context.util.ts
import type { ContextResult } from "../haystack/haystack.types";

/** P2.4: stable-pin golden-answer hits to the front of the KB context list
 *  (design §6 ⑦ "golden 优先 → hybrid" = one-retrieval ordering). Stable so
 *  relevance order within each group is preserved. */
export function pinGoldenFirst(results: ContextResult[]): ContextResult[] {
  const golden = results.filter((r) => r.isGolden === true);
  const rest = results.filter((r) => r.isGolden !== true);
  return [...golden, ...rest];
}
  • Step 4: Run test to verify it passes

Run: cd api && npx jest src/conversations/kb-context.util.spec.ts -v Expected: PASS.

  • Step 5: Wire it into the retrieval block

In conversations.service.ts at L3013 (right after retrievedKbContext = ctx.results;), insert:

        retrievedKbContext = pinGoldenFirst(retrievedKbContext);

Import: import { pinGoldenFirst } from "./kb-context.util";

  • Step 6: Run conversations specs to confirm no regression

Run: cd api && npx jest src/conversations/ -v Expected: PASS — no regression; the block still formats/unshifts as before, now golden-first.

  • Step 7: Commit
git add api/src/conversations/kb-context.util.ts api/src/conversations/kb-context.util.spec.ts api/src/conversations/conversations.service.ts
git commit -m "feat(conversations): pin golden-answer hits to top of KB context (P2.4)"

Task 9: documentation​

Files:

  • Modify: api/src/config-assets/CLAUDE.md (golden retrieval now BUILT, not just designed)

  • Modify: api/src/conversations/CLAUDE.md (golden-first ordering in the KB block)

  • Create/Modify: api/src/haystack/CLAUDE.md (golden-answer sync queue + shadow-row semantics + the fail-open authorization note)

  • Step 1: Update config-assets/CLAUDE.md

Change the line that says golden-answer retrieval is "designed but NOT built" (module CLAUDE.md line ~3 and the Pitfalls note ~L76) to state it is now built: publish → BullMQ golden-answer-sync → upload() shadow row (source="golden_answer", status="approved", parseStatus="ingested") with metadata.kind="golden_answer"; truth source stays in config-asset. Note the 5 lifecycle transitions (publish/override/rollback/archive enqueue; discard is a deliberate no-op) and that enqueue is post-commit best-effort (ADR-019).

  • Step 2: Update conversations/CLAUDE.md

Document that the KB retrieval block (conversations.service.ts:3006-3027) stable-pins isGolden hits first via pinGoldenFirst before formatKbContext.

  • Step 3: Create/update haystack/CLAUDE.md

Document: (a) the golden-answer-sync queue + processor; (b) the fail-open authorization subtlety — allowedDocumentIds fail-opens for doc_ids with NO org_documents row, so golden MUST write an approved+ingested shadow row to be governed by status/specialist/effectiveDate/audience (design §3 约束 1); (c) the rag allowlist now carries kind+golden_asset_id (10 keys) — removing them re-breaks golden identification.

  • Step 4: Self-check no machine paths

Run: grep -rnE '/Users/|/home/|/private/tmp' api/src/config-assets/CLAUDE.md api/src/conversations/CLAUDE.md api/src/haystack/CLAUDE.md; echo "exit=$?" Expected: exit=1 (no matches).

  • Step 5: Commit
git add api/src/config-assets/CLAUDE.md api/src/conversations/CLAUDE.md api/src/haystack/CLAUDE.md
git commit -m "docs: golden-answer retrieval path (P2.4)"

Task 10: full-suite verification against branch base​

  • Step 1: tsc + lint

Run: cd api && npx tsc --noEmit && npm run lint Expected: no new errors/warnings introduced by P2.4 (lint has a hard max-warnings budget — do NOT raise it; fix any new warning at source).

  • Step 2: Run the touched suites

Run: cd api && npx jest src/config-assets/ src/haystack/ src/conversations/ -v Expected: PASS. Any failure that also reproduces on the branch base (490517fb) is pre-existing (see baseline note); a NEW failure is blocking.

  • Step 3: rag test

Run: cd rag && python -m pytest tests/ -q (or repo rag command) Expected: PASS incl. the Task 1 serialize test.

  • Step 4: No commit — verification only. Proceed to branch-level review (subagent-driven-development final review).

Self-Review Checklist (run before handoff)​

Spec coverage:

  • design §4 A1 (upload shadow row) → Task 3 (upload with approved/ingested) ✅
  • design §4 B1 (meta flag + rag allowlist) → Task 1 (rag allowlist) + Task 3 (metadata.kind) + Task 7 (read marker) ✅
  • design §5 5 transitions → Task 5 (publish/override/rollback/archive enqueue; discard no-op) ✅
  • design §5 consistency best-effort BullMQ → Task 3 fail-open + Task 5 post-commit .catch ✅
  • design §7 Q1 shadow-row selection → Task 3 (status="approved") + Task 4 (source="golden_answer") ✅
  • design §6 ⑦ one-retrieval ordering → Task 8 pinGoldenFirst ✅

Type consistency: GoldenAnswerSyncJobData defined in Task 2, imported unchanged in Tasks 3 & 5. GOLDEN_ANSWER_SYNC_QUEUE defined in Task 3 processor file, imported in Tasks 5 & 6. isGolden/goldenAssetId defined in Task 7, consumed in Task 8. ✅

Placeholder scan: every code step shows the actual code; every run step shows the command + expected result. ✅

Isolation (ADR-020): golden shadow rows are scoped via specialistId in the upload attrs (Task 3) → retrieval's specialist arm enforces cross-specialist isolation exactly as normal KB docs. The enqueuer never crosses org (orgId flows from the asset). Branch-level review must include an isolation check (golden of specialist A never retrievable by specialist B).

Last updated on by Degen