Skip to content
Beef Docs

Fennec Runbook

Operational guide for shipping, rolling back, and unsticking the fennec admin SPA in Railway.

Deploy topology

  • Service: beef-fennec on Railway, region us-east4-eqdc4a, 1 replica (apps/fennec/railway.json).
  • Builder: DOCKERFILE against apps/fennec/Dockerfile. The Dockerfile builds the Vite bundle and runs serve -s dist -l 3000 in the runner stage (see apps/fennec/Dockerfile CMD). Railway runtime is V2 per apps/fennec/railway.json.
  • Healthcheck: GET / with a 120s timeout. Restart policy: ON_FAILURE, max 10 retries.
  • CDN / edge: Railway’s built-in routing. There is no dedicated CloudFront / Cloudflare distribution for fennec configured in this repo. TODO(@law): confirm no upstream CDN exists at the platform layer; runbook below assumes Railway-only.

CI gates (before any deploy)

.github/workflows/fennec.yml runs on push/PR to main and release when apps/fennec/** changes. Two required jobs:

JobCommandWorking dir
fennec-lintpnpm lintapps/fennec
fennec-typecheckpnpm tscapps/fennec

pnpm lint runs ESLint with --max-warnings 0 — any warning fails CI. The workflow does not include a build, deploy, or test step today — Railway performs the production build during deploy.

Standard deploy

Railway watches the release branch (TODO(@law): confirm the watched branch in the Railway service settings — .github/workflows/fennec.yml gates both main and release, but the deploying branch is set in the Railway service config, which is not in-repo).

  1. Merge to the deploy branch.
  2. Railway auto-builds against apps/fennec/Dockerfile.
  3. New revision becomes active when healthcheck GET / returns 200.
  4. Smoke-test:
    • Open the fennec URL.
    • Sign in with a known operator account.
    • Verify /dashboard loads (admin-gated) and /users/me returns 200.

If the new revision fails healthcheck, Railway leaves the previous revision serving traffic.

Rollback

Two options, in order of preference:

  1. Railway revision rollback. From the beef-fennec service → Deployments → select the previous good deploy → “Redeploy”. This is instant and does not require a code change.
  2. Git revert. Revert the offending commit on the deploy branch and let the workflow + Railway rebuild. Use this when the bad deploy already shipped a config change that revision rollback won’t undo.

When rolling back after an env-var change, redeploy explicitly — env values are baked into the SPA bundle (Vite), so rolling back code without rebuilding will keep the new env values in the old code.

Build cache

The fennec.yml workflow uses actions/setup-node@v4 with cache: "pnpm" and cache-dependency-path: apps/fennec/pnpm-lock.yaml. To force a clean install (when CI installs look corrupted):

  1. Bump apps/fennec/pnpm-lock.yaml (e.g. via pnpm install and commit).
  2. Or, in GitHub UI, delete the pnpm-lock.yaml cache entry under Actions → Caches.

Railway has its own Docker build cache. To bust it: in the Railway service, Settings → “Reset Build Cache” (TODO(@law): confirm the exact UI label) and trigger a new deploy.

CDN flush

There is no dedicated external CDN configured for fennec (per the topology section above). If a stale-asset incident occurs, the likely culprits are browser cache or Railway edge:

  1. Browser: instruct affected operators to hard-reload (Cmd-Shift-R / Ctrl-F5). The bundle’s hashed asset names should make this rare.
  2. Railway edge: trigger a fresh deploy. Railway routes by deploy ID, so a redeploy invalidates whatever was cached at the edge.
  3. If a third-party CDN is added later: document the purge command here. TODO(@law): revisit once a CDN exists.

Common operational tasks

Update an env var

  1. Set the new value in Railway → beef-fennec → Variables.
  2. Trigger a manual redeploy. The SPA must rebuild because Vite inlines REACT_APP_* values.
  3. Smoke test as above.

Change Clerk environment (dev → prod or vice versa)

  1. Update REACT_APP_CLERK_PUBLISHABLE_KEY on the matching Railway env.
  2. Coordinate with brain — its CLERK_SECRET_KEY and webhooks must match the same Clerk instance. See standards/auth-model.md.
  3. Redeploy fennec, then verify GET /users/me succeeds.

Add a new role-gated route

  1. Add the page under apps/fennec/src/pages/.
  2. Add a <Route> in apps/fennec/src/App.tsx.
  3. Wrap the element in <PrivateRoute minRole={UserRole.X}> from apps/fennec/src/components/PrivateRoute.tsx.
  4. Lint, typecheck, deploy via standard flow.

Database access (Strapi backend)

The embedded Strapi uses the Neon sirloin database with schema=fennec (see apps/fennec/backend/.env). Schema migrations are owned by Strapi’s own lifecycle — do not run ad-hoc DDL against the schema.

Incident triage one-liner

fennec down → check Railway deploy → check brain availability →
check Clerk status (status.clerk.com) → check Strapi backend deploy →
DNS / cert sanity

Pair this runbook with fennec-oncall.md for paging context.