CI/CD: the 4-lane policy

Short team guide. Full rationale and decision record: ADR-039.

The idea in one sentence: every change gets a fast, cheap check the moment it's a PR; the expensive full suite runs automatically only at pivot moments — post-merge (scoped), nightly, and promotion — plus manually whenever you want it.

Why: we were running the full suite (~55 runner-minutes + a 10×-billed macOS job) on every push to dev — 172 times a week — while only ~38% of merges touched api/, 6% touched the connector, and 29% touched no service at all.

The four lanes

#	When	What runs	Blocking?
1. PR gate	every PR → dev (pr-checks.yml)	lint, compile, unit tests (no DB/Redis containers), migrations check if you touched api/migrations/**. Plus Greptile review.	✅ PR Gate is a required check
2. Post-merge	push to dev = your merge (ci.yml)	integration suites, but only for the services your merge touched (frontend-only merge ⇒ no API shards; connector untouched ⇒ no macOS)	dev is fix-forward; a red run = fix it now
3. Promotion	PR dev→staging / staging→main (ci.yml + e2e-smoke.yml)	the FULL unscoped suite + live LLM evals (advisory) + trufflehog secret scan + e2e smoke	✅ CI Success + e2e gate
4. Nightly	02:00 UTC (nightly.yml → calls ci.yml)	FULL suite incl. coverage thresholds (the ratchet lives here now) + live evals + full connector run. Safety net for anything Lane 2's scoping skipped.	Slack digest on failure

Manual full run, any branch, any time:

gh workflow run ci.yml --ref <branch> [-f coverage=true] [-f live_evals=true]

What this means for you day-to-day

• Your PR feedback is ~10 minutes and doesn't spin up Postgres/Redis or macOS runners.
• Unit vs integration tests are separate Jest lanes (api/jest.classify.js decides automatically by scanning for infra tokens — DataSource, DATABASE_URL, REDIS_URL, BullModule.forRoot, …):
  • npx jest --config jest.config.unit.js — what your PR runs (parallel, no containers)
  • npx jest --config jest.config.integration.js — what the post-merge run uses (--runInBand + containers)
• If your new spec needs real PG/Redis but the classifier can't tell (e.g. it connects through a helper), the PR gate will fail it loudly with a connection error. Fix = add one comment line to the spec:

  // @jest-lane integration
• Coverage regressions no longer block your merge — they surface in the next morning's nightly digest instead. Don't treat that as license to skip tests; the ratchet still fires.
• Integration breakage in a service you didn't touch (e.g. an API change breaking an agent contract) surfaces at nightly or promotion, not on your merge. That's the accepted trade for the budget cut. If you know your change has cross-service blast radius, dispatch a manual full run on your branch before merging.
• Touching .github/workflows/ or .github/actions/ always triggers the full suite — CI changes never get scoped CI.

Release cadence (context)

Weekly: staging cut Monday 06:00 UTC → testing Mon–Wed → prod Wednesday 06:00 UTC + smoke. Motto: keep prod stable; stability derives from proper testing on staging. Unfinished features ship dark behind feature flags. (A release-branch model was considered and deferred — the dev → staging → main flow stays for now.)

Ownership / tuning

Shard counts, path patterns, and lane budgets are re-tuned against real measured minutes — see the standing measurement issue #4579. Baseline to beat: ~13,000 Linux-min-equivalents/week (week of 2026-07-08).