Skip to main content

ADR-038: Railway deprecation and ECS-canonical production

Status: Accepted (2026-07-13) Authors: Degen (formal write-up); decision confirmed by Eusden/Emanuele per ADR-003 addenda and the #701 Kaito-launch P0 cutover. Supersedes: ADR-003 — Railway + Vercel Split Deployment (partially, per below)

Context​

ADR-003 (2026-04-22) established Railway for all backend services (api, agent, Postgres, Redis) plus Vercel for the frontend, as the platform's canonical deployment target. That was correct at the time — Railway's managed Postgres/Redis and simple deploy flow suited an early-stage single-region product.

As the platform moved toward its first paying client (Kaito) and multi-region ambitions, the Railway posture became insufficient:

  • Railway is single-region (US-West); Asia-Pacific latency for the Kaito launch was a known trade-off, tracked as BLK-037 and folded into the ECS cutover.
  • #701 (P0, "Migrate humanwork production from Railway to ECS — Kaito-launch P0") framed the decision plainly: "Railway is fine for vibe-coded dev iteration; it is not the production environment we hand a real client a WhatsApp number on."
  • ADR-028 (2026-06-03) had already retired the per-Org Railway agent-runtime provisioner (railway_per_org) in favor of ECS-only per-Org isolation — a narrower, code-level decommission that anticipated this broader infrastructure move.

#701 closed on 2026-06-15. Its acceptance criteria — ECS infrastructure provisioned, API+Agent+Frontend on ECS (not Railway) for production, secrets in AWS Secrets Manager, RDS migrated, Twilio/Cloudflare/Slack webhooks repointed — were all met and smoke-tested (WhatsApp round-trip via Kaito's number, end to end on ECS).

This ADR is the formal supersession that ADR-003's addendum said would land "once prod also cuts over."

Decision​

ADR-003 is superseded for staging and production. The current and going-forward canonical posture, per docs/ops/deploy.md:

EnvironmentRuntimeDatabaseConnection pooler
dev (api.h852.work)RailwayRailway PostgresNone — direct connection
staging (api.h853.work)AWS ECS FargateRDS PostgreSQLAWS RDS Proxy
production (api.h.work)AWS ECS FargateRDS PostgreSQLAWS RDS Proxy
  • Railway remains canonical for dev only — this is a deliberate choice (see #1233, 2026-06-05), not an oversight or a lagging migration. Dev's low load profile doesn't justify AWS's operational overhead; the Railway PgBouncer plugin remains available unwired if that ever changes.
  • Staging and production are AWS ECS Fargate, with RDS PostgreSQL fronted by RDS Proxy, Terraform-managed (infra/terraform/modules/{ecs-cluster,ecs-service,alb,monitoring,data}), and deployed via the dev → staging → main CI-enforced promotion flow (#1559).
  • Frontend stays on Vercel — ADR-003's frontend decision is unaffected by this supersession. Vercel's CDN-first Next.js hosting was never part of the Railway-vs-ECS question.
  • Vercel deployment protection is per-environment: dev/staging preview deployments, production on the main branch.

What ADR-003 got right (preserved)​

  • The Vercel-for-frontend / backend-services-elsewhere split.
  • The reasoning for rejecting Render and Fly.io at the time (limited WebSocket support, no managed Postgres/Redis) — still valid; those options were never revisited because ECS solved the actual problem (multi-region readiness, AWS-native secrets/observability, no per-instance connection-limit surprises).
  • The env-driven DB pool sizing addendum (DATABASE_POOL_MAX etc., #917) — the mechanism is unchanged; only the pooler in front of it differs by environment (RDS Proxy on AWS, none on Railway dev).

What changed​

  • Railway is no longer a production concern. Production incidents, on-call runbooks, and rollback procedures should default to the ECS/AWS path first (see docs/runbooks/incident-response.md, which already leads with ECS/AWS and treats Railway as "dev only").
  • The per-Org Railway agent-runtime provisioner is deleted (ADR-028) — there is no Railway per-Org isolation escape hatch anymore. shared_railway remains as the dev-only shared runtime path.
  • Deploy credentials for staging/prod are AWS IAM + Terraform state, not Railway tokens. Railway tokens remain only for dev.

Consequences​

Positive:

  • Multi-region readiness (AWS AP-Southeast is now a real option for the Asia-Pacific latency concern BLK-037 raised) — not yet exercised, but no longer architecturally blocked by Railway's single-region limitation.
  • One production incident-response playbook (ECS-first), not two parallel paths to reason about for staging+prod.
  • AWS Secrets Manager for production secrets, closing the "plain Railway env vars" gap #701 flagged as unacceptable for a paying client's credentials.

Negative / trade-offs:

  • Two infrastructure stacks to maintain (Railway for dev, AWS for staging+prod) rather than one. Accepted deliberately — dev's low load profile and fast-iteration needs don't match production's isolation/compliance needs, and unifying them would either over-provision dev or under-provision prod.
  • Contributors need to know which environment they're debugging before reaching for railway logs vs. aws ecs/aws logs. Mitigated by the ECS-first framing in incident-response.md and the explicit "Railway path (dev only)" labeling introduced there.

Rollback​

There is no rollback to Railway for production. If ECS provisioning breaks, the mitigation is ECS-native (task rollback to a prior revision, scale-out, or — per ADR-028 — falling back to shared_railway for the agent-runtime path specifically, which is a dev/shared-infra fallback, not a production Railway reinstatement).

References​

Last updated on by Degen