Skip to main content

P3.3 — Retrieval Cascade Convergence Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development. Steps use checkbox (- [ ]) syntax.

Goal: On the assembler path (flag prompt_assembler_enabled ON), converge the correction few-shot into the assembler's correction layer (today it is dropped when assembling) and make the assembler's token budget actually enforce (today it is dead code), so KB + correction share one deterministic budget source with the priority-ordered truncation the design (§6.3) specifies — WITHOUT splitting the single Haystack retrieval or changing the flag-OFF path.

Architecture: P3.3 is the "converge retrieval/injection into the assembler" step. Reality (verified by Explore) reshaped the scope vs the design doc's framing — see the DECISION POINT below. The real, valuable, non-high-risk work is: (1) run correction retrieval on the assembler path and feed it into the correction layer (currently correction: null at both assemble() sites → correction is silently absent whenever assembling); (2) pass a real maxTokens so the assembler's already-built priority-ordered truncation (assembler.service.ts:107-119, currently inert because maxTokens is never passed → 4M-char default) becomes the single budget authority for the assembled body. Golden is already pinned within the KB layer by pinGoldenFirst (one retrieval, golden in-band — P2.4), so no retrieval split is needed or done.

⚠️ DECISION POINT (safe default during user absence — reversible): The design doc §6.5 frames P3.3 as "golden→hybrid→correction 逐级可跳过 (per-level skippable cascade)". Explore verified this framing does not match reality: golden and hybrid are NOT two retrieval stages — they come back in ONE HaystackRetrievalService.search() call and are only reordered by pinGoldenFirst. Only correction is a genuinely separate retrieval (pgvector over the corrections table, learning.service.ts). A literal "skip hybrid if golden fills the budget" would require SPLITTING the single Haystack call into two — which (a) does not exist today, (b) touches the production retrieval path the user was highly protective of in P2.4, and (c) is a performance regression (two calls where one suffices) for speculative benefit (top-K=5 rarely fills the 16k cap). This plan takes the pragmatic reading that matches the design's INTENT (§6.3 "预算在 assembler 侧执行" + one budget source) without the high-risk split: converge correction + activate the assembler budget; leave the single KB retrieval intact with golden pinned inside it. If the user specifically wants a true two-call golden/hybrid cascade with per-level skip, that is a separate, larger, retrieval-path change to scope explicitly — flag it and stop. The pragmatic version ships as the reversible default.

Tech Stack: NestJS 11 (AssemblerService budget, ConversationsService correction resolution reusing retrieveSimilarCorrections + formatCorrectionFewShot), Jest.

Scope note: Long-lived branch feat/specialist-value-output-p2plus. Depends on P3.1 (assembler) + P3.2 (skill layer). Commit per sub-task.

Boundary (P3.3 does NOT do):

  • Does NOT split the single Haystack search() into golden-only + hybrid calls (DECISION POINT — pragmatic default).
  • Does NOT change the flag-OFF path — OFF stays byte-identical (KB history unshift + separate correction unshift exactly as today).
  • Does NOT change the correction retrieval subsystem (retrieveSimilarCorrections / correction_fewshot_enabled flag) — it reuses them.
  • Does NOT unify MAX_KB_CONTEXT_CHARS (runtime, sanitize-llm.ts) with config-asset-budgets.ts (save-time validation). Those stay separate; P3.3's assembler budget is a THIRD, explicit total-body budget (see Task 2). Per-layer formatKbContext 16k cap stays as the inner KB cap.
  • Does NOT add KB retrieval to the regenerate path (kb stays null there by construction — a per-level KB cascade on regenerate would be a no-op).
  • Does NOT touch skill.redLines (P3.5) or crisis-layer relocation.

Pre-existing baseline: verify against branch base 490517fb. Prompt-shape change on the assembler path → re-run agent/evals/test_prompt_echo_guard.py + test_chat_helpers.py + test_prompt_assembly_meta.py.

File Structure​

Modify:

  • api/src/conversations/conversations.service.ts:
    • Add private async resolveCorrectionFewShot(ctx): Promise<string | null> — a sibling of maybeInjectCorrectionFewShot that RETURNS the formatted correction string (or null) instead of unshifting it into history. Reuses retrieveSimilarCorrections + formatCorrectionFewShot. Same correction_fewshot_enabled flag gate, same fail-open (returns null on error).
    • At the main dispatch assemble({...}) (~:3242): pass correction: <resolved> (instead of hard null) and maxTokens: ASSEMBLER_BODY_MAX_TOKENS.
    • At the regenerate assemble({...}) (~:6544): pass correction: <resolved> and maxTokens: ASSEMBLER_BODY_MAX_TOKENS (regenerate CAN have corrections even though it has no KB — the correction subsystem is independent of KB retrieval).
  • api/src/conversations/assembler.service.ts: export a documented ASSEMBLER_BODY_MAX_TOKENS constant (single source for the total assembled-body budget) and use it as the default when maxTokens is unset (replacing the current 1_000_000 magic default), so the budget is real even if a call site forgets to pass it.
  • api/src/conversations/CLAUDE.md, design doc §6.5 note — document the pragmatic convergence + the budget activation + the DECISION POINT (no retrieval split).

Reuse (no change): pinGoldenFirst, formatKbContext, formatCorrectionFewShot, retrieveSimilarCorrections, the single haystackRetrieval.search() call.

Task 1: resolveCorrectionFewShot — correction as a returned string (assembler path)​

Files:

  • Modify: api/src/conversations/conversations.service.ts
  • Test: api/src/conversations/prompt-assembler-wiring.spec.ts (extend) or a focused correction spec

Context: maybeInjectCorrectionFewShot (:464-497) unshifts correction few-shot into history on the OFF path. On the assembler ON path it's gated OFF (if (!assembling) at :3109/:6479) and assemble() gets correction: null — so correction is silently absent whenever assembling. Task 1 adds a return-a-string sibling so the assembler correction layer gets populated.

  • Step 1: Read maybeInjectCorrectionFewShot (:464-497), formatCorrectionFewShot (:504-523), and this.learning.retrieveSimilarCorrections signature (learning.service.ts:331). Note the flag gate correction_fewshot_enabled and the fail-open try/catch.

  • Step 2: Write the failing test (extend prompt-assembler-wiring.spec.ts)

The harness already has featureFlags.resolve (extend the flagResolver to also enable correction_fewshot_enabled when needed) and needs a LearningService mock with retrieveSimilarCorrections. Add it to the providers if absent (mock returns [{ original: "draft text", corrected: "better text" }]).

it("flag ON + corrections enabled: correction rides the assembler correction layer, NOT history", async () => {
  featureFlags.resolve.mockImplementation((key: string) => {
    if (key === "prompt_assembler_enabled") return { key, enabled: true, reason: null, scopeType: "org_specialist", scopeId: "osa-1", flagId: "f1" };
    if (key === "correction_fewshot_enabled") return { key, enabled: true, reason: null, scopeType: "org_specialist", scopeId: "osa-1", flagId: "f2" };
    return undefined;
  });
  await driveSendMessage();
  const arg = chatArg(orgAgentClient);
  // correction is inside the assembled body...
  expect(arg.agentContext.assembled_prompt.assembled_context).toContain("Expert-corrected: better text");
  const corrLayer = arg.agentContext.assembled_prompt.layer_manifest.find((l: any) => l.id === "correction");
  expect(corrLayer).toBeDefined();
  // ...and NOT ALSO as a history turn (mutual-exclusion, like KB)
  expect(arg.history.some((t: any) => t.content?.includes("<correction_example>"))).toBe(false);
});

it("flag ON + corrections DISABLED: no correction layer (correction flag still respected)", async () => {
  featureFlags.resolve.mockImplementation((key: string) =>
    key === "prompt_assembler_enabled" ? { key, enabled: true, reason: null, scopeType: "org_specialist", scopeId: "osa-1", flagId: "f1" } : undefined);
  await driveSendMessage();
  const arg = chatArg(orgAgentClient);
  const corrLayer = arg.agentContext.assembled_prompt.layer_manifest.find((l: any) => l.id === "correction");
  expect(corrLayer).toBeUndefined(); // correction layer absent when correction flag off
});

Confirm the real mock/helper names in the spec first (the harness may already stub LearningService — grep the providers). Adapt the flagResolver extension to the existing flagResolver(bool) shape (it currently keys only prompt_assembler_enabled; you'll generalize it or add a per-test mockImplementation).

Run → FAIL (correction is null in the assembled body today).

  • Step 3: Implement resolveCorrectionFewShot (sibling of maybeInjectCorrectionFewShot, returns the string):
/**
 * P3.3 — assembler-path correction few-shot. Same retrieval + formatting as
 * maybeInjectCorrectionFewShot, but RETURNS the formatted string (or null)
 * for the assembler's `correction` layer instead of unshifting a history turn.
 * On the assembler path, correction lives inside assembled_context (mutual-
 * exclusion with the OFF-path history unshift, like KB). Same flag gate
 * (correction_fewshot_enabled, default OFF) and fail-open (null on error).
 */
private async resolveCorrectionFewShot(ctx: {
  orgId?: string | null;
  specialistAssignmentId?: string | null;
  specialistId?: string | null;
  role?: string | null;
  queryText: string;
}): Promise<string | null> {
  if (!ctx.orgId || !ctx.specialistId || !this.learning) return null;
  const flag = await this.featureFlags?.resolve("correction_fewshot_enabled", {
    orgId: ctx.orgId,
    specialistAssignmentId: ctx.specialistAssignmentId ?? undefined,
  });
  if (flag?.enabled !== true) return null;
  try {
    const examples = await this.learning.retrieveSimilarCorrections({
      specialistId: ctx.specialistId,
      role: ctx.role,
      queryText: ctx.queryText,
    });
    return examples.length > 0 ? this.formatCorrectionFewShot(examples) : null;
  } catch (err) {
    this.logger.warn(`correction few-shot (assembler) failed (non-fatal): ${(err as Error).message}`);
    return null;
  }
}
  • Step 4: Wire it at BOTH assembler sites. In the if (assembling) block, before assemble({...}):
    const correctionFewShot = await this.resolveCorrectionFewShot({
      orgId: dispatchOrgId,            // conv.orgId ?? undefined at the regenerate site
      specialistAssignmentId: specialistAssignmentId, // regenAssignmentId at regenerate
      specialistId: conv.specialistId,
      role: conv.role,
      queryText: dto.content,          // inboundText at the regenerate site
    });

Then change correction: null → correction: correctionFewShot in the assemble({...}) object at both sites.

The OFF-path maybeInjectCorrectionFewShot call (inside if (!assembling)) stays EXACTLY as-is — this only adds the ON-path resolution. Mutual-exclusion holds: if (!assembling) unshifts to history; if (assembling) feeds the layer. Never both.

  • Step 5: Run tests — new tests pass; ALL existing wiring tests (flag-OFF byte-equal, skill layer, assembler-absent edge, Fix-2 dedup) stay green; persona-override-mode / specialist-template-key / full conversations suite green. npx tsc --noEmit clean.

  • Step 6: Commit

git add api/src/conversations/conversations.service.ts api/src/conversations/prompt-assembler-wiring.spec.ts
git commit -m "feat(conversations): converge correction few-shot into assembler correction layer (P3.3)"

Task 2: activate the assembler token budget (single body-budget source)​

Files:

  • Modify: api/src/conversations/assembler.service.ts
  • Modify: api/src/conversations/conversations.service.ts (pass maxTokens at both sites)
  • Test: api/src/conversations/assembler.service.spec.ts + wiring spec

Context: the assembler's truncation (:107-119) is correct but DEAD — maxTokens is never passed, defaulting to 1_000_000 (4M chars). Task 2 gives it a real default constant and passes it explicitly, so the priority-ordered, neverTruncate-respecting truncation (§6.3) actually governs the assembled body.

  • Step 1: Read assembler.service.ts:63-119 (the APPROX_CHARS_PER_TOKEN, budgetChars, truncation loop). Confirm neverTruncate layers (red_lines/crisis/soul) are exempt and cutting is lowest-priority-first.

  • Step 2: Write the failing test (in assembler.service.spec.ts) — a real budget now truncates low-priority layers but never red_lines/soul:

test("ASSEMBLER_BODY_MAX_TOKENS is the default budget when maxTokens unset", () => {
  // A giant kb layer under the DEFAULT budget must get truncated (proves the default is finite, not 1M tokens)
  const { assembled_context, layer_manifest } = svc.assemble({
    ...baseInput,
    kb: "x".repeat(1_000_000),
  });
  const kbLayer = layer_manifest.find((l) => l.id === "kb")!;
  expect(kbLayer.charCount).toBeLessThan(1_000_000);       // truncated by the real default
  expect(assembled_context).toContain(baseInput.redLines);  // red_lines survives
  expect(assembled_context).toContain(baseInput.soul);      // soul survives (neverTruncate)
});

Read the existing assembler.service.spec.ts to confirm baseInput has redLines/soul/kb and the svc fixture. The existing "when over budget..." test passes an explicit maxTokens: 200; this new one proves the DEFAULT is now finite.

Run → FAIL (today the default is 1M tokens = 4M chars, so 1M chars is NOT truncated).

  • Step 3: Implement. In assembler.service.ts, add near APPROX_CHARS_PER_TOKEN:
/**
 * Default total budget for the assembled body (P3.3). The runtime tail
 * (inbound files / extracted docs / web_context) is appended by the agent
 * and NOT counted here, so this reserves headroom below the model context
 * window (design §6.3). A conservative default so the priority-ordered
 * truncation is always live — call sites may override via `maxTokens`.
 */
export const ASSEMBLER_BODY_MAX_TOKENS = 24_000; // ~96k chars @ 4 chars/token

Change the default in assemble from 1_000_000 to ASSEMBLER_BODY_MAX_TOKENS:

const budgetChars = (input.maxTokens ?? ASSEMBLER_BODY_MAX_TOKENS) * APPROX_CHARS_PER_TOKEN;

The 24_000-token value: KB is already capped at 16k CHARS by formatKbContext before it arrives, skill at 16k chars, soul at 8k chars — the aggregate of all layers is well under 96k chars in practice, so the default won't truncate real traffic; it's a safety ceiling that makes the mechanism live and testable. Document this reasoning inline.

  • Step 4: Pass maxTokens explicitly at both call sites — add maxTokens: ASSEMBLER_BODY_MAX_TOKENS to both assemble({...}) objects (import the const). This makes the budget source explicit at the call site (not relying on the default), matching design §6.3 "single budget source".

  • Step 5: Run tests — new + existing assembler tests green (the explicit maxTokens: 200 test still truncates; the new default test truncates 1M chars; the layer-order tests unaffected); wiring tests green; npx tsc --noEmit clean.

  • Step 6: Commit

git add api/src/conversations/assembler.service.ts api/src/conversations/conversations.service.ts api/src/conversations/assembler.service.spec.ts
git commit -m "feat(conversations): activate assembler body token budget (ASSEMBLER_BODY_MAX_TOKENS, single source) (P3.3)"

Task 3: docs + full verification​

Files:

  • Modify: api/src/conversations/CLAUDE.md

  • Step 1: Docs — add a Constraint bullet to conversations/CLAUDE.md:

    • P3.3 converges correction few-shot into the assembler correction layer on the assembler path (resolveCorrectionFewShot returns the string; correction: null at the assemble() sites is now the resolved few-shot). Mutual-exclusion with the OFF-path maybeInjectCorrectionFewShot history unshift — same correction_fewshot_enabled flag, same fail-open. Correction now works on the regenerate assembler path too (correction retrieval is independent of KB retrieval).
    • ASSEMBLER_BODY_MAX_TOKENS (assembler.service.ts) is the single total-body budget; the assembler's priority-ordered truncation (neverTruncate red_lines/soul exempt) is now live (was dead — maxTokens never passed). This is separate from MAX_KB_CONTEXT_CHARS (the inner per-KB 16k cap in formatKbContext, unchanged) and from config-asset-budgets.ts (save-time validation).
    • Note the DECISION POINT: no golden/hybrid retrieval split — golden is pinned within the single KB retrieval by pinGoldenFirst; a true two-call cascade is out of scope (design §6.5 framing corrected to reality).

  • Step 2: Self-check no machine paths — grep -rnE '/Users/|/home/|/private/tmp' api/src/conversations/CLAUDE.md; echo exit=$? → 1.

  • Step 3: Full verification

    • cd api && npx tsc --noEmit && npm run lint → clean (hard lint budget).
    • npx jest src/conversations/ src/config-assets/ src/feature-flags/ --runInBand → all pass; flag-OFF byte-equality intact.
    • cd agent && <venv>/bin/python -m pytest evals/test_prompt_echo_guard.py evals/test_chat_helpers.py evals/test_prompt_assembly_meta.py -q → pass (correction in the assembled body must not trip echo/CoT guard).

  • Step 4: Commit docs

git add api/src/conversations/CLAUDE.md
git commit -m "docs(P3.3): correction convergence + assembler budget activation + no-split decision"
  • Step 5: No further commit — proceed to Phase-3-task-level review (P3.3 range). Dispatch code-reviewer: verify flag-OFF byte-equality (correction/budget changes never touch OFF path), correction mutual-exclusion (layer XOR history unshift, never both, never neither when enabled), budget truncation never cuts red_lines/soul, and the regenerate-site correction wiring is correct.

Self-Review Checklist​

Spec coverage (design §6.5 P3.3, pragmatic reading): correction converged into assembler correction layer ✅ (Task 1); assembler budget activated as the single body-budget source with priority-ordered truncation ✅ (Task 2); golden already pinned in the single KB retrieval (no split — DECISION POINT) ✅.

Type consistency: resolveCorrectionFewShot returns string | null → assemble({ correction }) accepts string | null | undefined (assembler AssembleInput.correction). ASSEMBLER_BODY_MAX_TOKENS exported from assembler, imported at both call sites + used as the assemble default.

Safety net: flag OFF → neither change runs (correction resolution + maxTokens are inside if (assembling); the assembler default only affects the ON path since OFF never calls assemble). Correction respects its own correction_fewshot_enabled flag independently. Fail-open: resolveCorrectionFewShot → null on any error. Rollback = leave prompt_assembler_enabled OFF.

Boundary respected: no retrieval split; no OFF-path change; no correction-subsystem change; no budget-constant unification; no regenerate KB retrieval; no redLines/crisis work.

DECISION POINT surfaced: the "no golden/hybrid split" pragmatic reading is flagged for user confirmation; a true two-call cascade is scoped as a separate larger change if wanted.

Last updated on by Degen