humanwork-rag Deploys

Hosting (2026-06-04): Dev runs on Railway today (managed via rag/railway.toml). Staging and prod run on AWS ECS Fargate behind Cloud Map at humanwork-rag.humanwork.internal:1416, deployed via the manual Deploy Rag workflow (.github/workflows/rag-deploy.yml). The earlier "Railway-only, no AWS release path" framing (#1455) was reverted by #1527/#1541 — ECS is canonical for staging/prod.

humanwork-rag is the shared Hayhooks service for Haystack retrieval and ingestion. It stores chunks and embeddings in pgvector on the existing humanwork Postgres (per ADR-025) and routes embed/chat/rerank through OpenRouter (per ADR-026).

Local Development

Run Hayhooks from the rag/ package:

cd rag
pip install -e '.[dev]'
hayhooks run

Local Hayhooks defaults to http://localhost:1416. Set PG_CONN_STR, OPENROUTER_API_KEY, and HAYHOOKS_API_KEY before running ingestion or retrieval against a real database.

Railway (dev)

Railway uses the rag/railway.toml service definition. The service runs one replica, binds Hayhooks to 0.0.0.0:1416, loads pipelines from /app/pipelines, and exposes /status for health checks.

Required Railway variables:

Variable	Purpose
OPENROUTER_API_KEY	Single key for embeddings, chat, and reranking
HAYHOOKS_API_KEY	Shared key used by humanwork-api dev
PG_CONN_STR	Railway-managed Postgres connection string for Haystack storage

Access stays on Railway private networking at humanwork-rag.railway.internal:1416; there is no public ingress. Cost target is at or below $50/mo total for the Hayhooks service, Postgres, and OpenRouter usage in dev.

Deploy by merging the desired commit to the branch wired to the Railway environment or by triggering a Railway redeploy for that commit. Roll back from the Railway deploy history to the previous healthy deploy.

ECS (staging + prod)

Staging and prod run as the humanwork-rag ECS Fargate service in the shared humanwork-<env> cluster, registered in Cloud Map at humanwork-rag.humanwork.internal:1416. Terraform source of truth: infra/terraform/modules/stack/main.tf (#1527 added the service + cleaned up RAGFlow remnants).

Deploy

Manual via the Deploy Rag workflow (.github/workflows/rag-deploy.yml, workflow_dispatch-only):

1. GitHub → Actions → Deploy Rag → Run workflow.
2. Select environment: staging or environment: prod (gated by the #1559 dev→staging→main promotion flow — staging deploys from staging branch, prod from main).
3. Optionally pin an image tag; default is :latest-<env>.
4. The workflow renders the task definition, calls aws ecs update-service, and waits for stability.

Environment variables

Sourced from Secrets Manager + plain task-def environment[], wired by infra/terraform/modules/stack/:

Variable	Source	Purpose
OPENROUTER_API_KEY	Secrets Manager	Embeddings, chat, rerank
HAYSTACK_API_KEY	Secrets Manager	Shared key used by humanwork-api (sent as X-API-Key, #1588)
PG_CONN_STR	Secrets Manager (RDS Proxy URL)	pgvector storage on the shared humanwork RDS

Health check

Container-level (per #1586): curl -fsS http://localhost:1416/status. Do not use wget — the task image does not ship it.

Health Check (from inside the VPC)

For staging/prod, exec into an ECS task or use the bastion:

curl -fsS http://humanwork-rag.humanwork.internal:1416/status

For dev (Railway private network):

curl -fsS http://humanwork-rag.railway.internal:1416/status

Smoke Test

Run after each deploy. X-API-Key is required (#1588 — Authorization: Bearer is rejected):

# Staging / prod (run from inside the VPC)
curl -fsS \
  -H "X-API-Key: $HAYSTACK_API_KEY" \
  -H "Content-Type: application/json" \
  http://humanwork-rag.humanwork.internal:1416/retrieval/run \
  -d '{"query":"health check","org_id":"smoke","top_k":1}'

# Dev (Railway)
curl -fsS \
  -H "X-API-Key: $HAYHOOKS_API_KEY" \
  -H "Content-Type: application/json" \
  http://humanwork-rag.railway.internal:1416/retrieval/run \
  -d '{"query":"health check","org_id":"smoke","top_k":1}'

Rollback

ECS (staging + prod)

aws ecs update-service \
  --cluster humanwork-<env> \
  --service humanwork-rag \
  --task-definition humanwork-rag-<env>:<prev-rev>
aws ecs wait services-stable --cluster humanwork-<env> --services humanwork-rag

Railway (dev)

Roll back from the Railway deploy history to the previous healthy deploy.

Related

• haystack-deploy.md — overall Haystack deploy procedure
• ADR-024, ADR-025, ADR-026
• PRs: #1527 (ECS service + RAGFlow cleanup), #1541 (rag-deploy.yml), #1586 (curl health check), #1588 (X-API-Key).