Testing Strategy

This page is policy and pyramid. For commands and per-framework usage, see Testing.

Pyramid Per Service

Each service has its own pyramid because the tooling and risk profile differ.

sirloin (Go) — wide unit base, real-DB integration tier

Layer	Tool	What counts	Where
Unit	testing + testify	Pure handler / service / repo logic with mocked deps	*_test.go next to source
Integration	testify/suite + testcontainers (Postgres) + MinIO test server + bufconn gRPC server	Real DB, real S3, in-memory gRPC	apps/sirloin/internal/pkg/testingsuite/
Sandbox / external	Same suite, build-tagged sandbox / integration	Hits Chargebee sandbox	make run-tests-all
Coverage gate	verify-coverage target in apps/sirloin/Makefile	Threshold currently 3% (Check test coverage meets threshold (default 3%))	CI gate

The Sirloin pyramid is integration-heavy. BaseTestSuite and GRPCTestSuite use isolated databases on real Postgres, which catches BUN serialization bugs. Each test process migrates a template once, including River tables, then clones a unique database for each fixture. Cleanup drops only that fixture’s database. Testcontainers removes the shared server when the process exits. Raw migration and index tests continue to use dedicated containers. See Testing for fixtures.

brain (NestJS) — unit-heavy with NestJS testing modules

Layer	Tool	What counts	Where
Unit	Jest + @nestjs/testing	Service / processor logic with useValue mocked providers	*.spec.ts next to source
Integration	Jest with a child process and local HTTP/OTLP collector	Native HTTP trace parentage, correlated logs, shutdown and fatal-error export	apps/brain/src/common/otel/runtime-export.spec.ts
Coverage gate	None enforced — apps/brain/package.json:24 runs jest --coverage but no coverageThreshold is set	—	—

brisket (Next.js) — component + hook unit tests

Layer	Tool	What counts	Where
Unit	Vitest + @testing-library/react (not Jest — common gotcha)	Component / hook / utility tests	colocated with source
Integration	Vitest with a local HTTP/OTLP collector	Native HTTP trace parentage, correlated logs, browser-report labels	apps/brisket/src/lib/telemetry/export.integration.test.ts
E2E / API	Playwright	Cross-service flows against a deployed stack	tests/
Coverage gate	None enforced — apps/brisket/vitest.config.ts does not set coverage.thresholds	—	—

fennec (React) — typecheck + lint as the floor

Layer	Tool	What counts	Where
Type	tsc --noEmit	Compile-clean TS	pnpm tsc
Lint	ESLint with --max-warnings 0	Zero warnings tolerated	pnpm lint
Unit	Vitest + @testing-library/react + @testing-library/jest-dom	Component / hook tests	colocated
Coverage gate	None enforced — no Vitest config in apps/fennec/ and no coverageThreshold in apps/fennec/package.json	—	—

flank (TS workflow editor UI)

Layer	Tool	What counts	Where
Unit	Vitest	Editor / client-utility logic	colocated
Coverage gate	None enforced — apps/flank/vitest.config.ts does not set coverage.thresholds	—	—

round (Go ML) and strip (Go SSR)

Layer	Tool	Notes
Unit	Go testing	make run-tests with -race
Integration	None enforced — both services run only go test -race -count=1 ./... (apps/round/Makefile:5, apps/strip/Makefile:28); strip exposes test-coverage targets but no integration tier	—

Cross-service E2E

• Playwright under tests/ (config tests/playwright.config.ts).
• Run with cd tests && npm run test:api.
• The Playwright config (tests/playwright.config.ts) defaults ENV=dev, pointed at https://fennec-dev-api.sexty.dev/ (tests/testConfig.ts). No GitHub workflow under .github/workflows/ invokes the tests/ Playwright suite — it is run manually (cd tests && npm run test:api).

Coverage Targets

Service	Target	Source
sirloin	3% (verify-coverage)	apps/sirloin/Makefile — note this is a low floor, real coverage is much higher
brain	None enforced — jest --coverage runs but no coverageThreshold is configured in apps/brain/package.json
brisket	None enforced — apps/brisket/vitest.config.ts has no coverage.thresholds
fennec	None enforced — no Vitest config nor coverageThreshold in apps/fennec/package.json
flank	None enforced — apps/flank/vitest.config.ts has no coverage.thresholds
round	None enforced — apps/round/Makefile only runs go test -race -count=1 ./...
strip	None enforced — apps/strip/Makefile:31-44 exposes test-coverage/test-coverage-summary targets but does not gate on a threshold

The sirloin 3% threshold exists to prevent regressions, not as an aspirational target. Treat current coverage as the de facto floor when you touch a package.

Mocking Policy

Go (sirloin, round, strip) — mockery v3

• Mocks generated from interfaces by mockery v3, configured in apps/sirloin/.mockery.yml.
• Output naming: mock_*.go, same directory as the interface.
• Generate via make generate-mocks.

When to mock vs use real:

Dependency	Mock?	Why
Postgres	No — use TestDatabase (testcontainers)	BUN serialization bugs only surface against real PG
S3 / object storage	No — use EnableMinIOTestServer()	Catches HMAC + bucket-policy issues
Service peers (Brain HTTP, Round gRPC)	Yes — generated/interface mock	We don’t want test runs cross-spawning services
Chargebee	Yes for unit tests; real sandbox for integration-tagged tests	Sandbox is rate-limited (-p 1 enforced in run-tests-all)
Clock / random	Yes	Determinism

TS (brain, brisket, fennec, flank)

• Brain: Test.createTestingModule + useValue for provider injection. Mocks live inline in beforeEach per test file.
• Brisket / fennec: prefer Testing Library + vi.mock for module-level mocks. Avoid mocking React components — use real DOM.

Test Data

Service	Mechanism	Source
sirloin	FixtureBuilder (functional options) — CreateUser, CreateCredit, CreateCharacter, CreateMedia with WithImageCredits, WithCharacterStatus, etc.	apps/sirloin/internal/pkg/testingsuite/fixtures.go
brain	No centralised factories found in apps/brain/src/**/*.spec.ts; tests construct objects inline (e.g. media-processing.service.spec.ts).
brisket / fennec / flank	Inline data per test — no shared factory module under apps/{brisket,fennec,flank}/src/.

Seed scripts (production / dev environments rather than tests) live per service. The former flank seed runners (seed.ts / seed-sync.ts) were deleted with flank’s legacy engine. sirloin and brain have no committed dev-seed scripts; local data comes from make dev-up-d plus manual flows.

CI Test Execution

Workflow files in .github/workflows/:

Workflow	Service	Triggers tests	Notes
sirloin.yml	sirloin	make ci-all mirror — verify-mod-tidy, verify-migrations, modernize, lint, govulncheck, build, run-tests, verify-coverage	Unit only; run-tests-all (Chargebee sandbox) is gated
brain.yml	brain	pnpm install → prisma generate → pnpm lint / pnpm tsc / pnpm test (three parallel jobs in .github/workflows/brain.yml)
brisket.yml	brisket	pnpm install → pnpm lint / pnpm typecheck / pnpm test (three parallel jobs in .github/workflows/brisket.yml)
fennec.yml	fennec	lint + tsc with max-warnings 0
flank.yml	flank	typecheck + lint + test
strip.yml	strip	golangci-lint (pinned v2.14.0), a templ generate freshness check, go build ./..., and go test -race ./...; the strip job is the aggregate check	Railway’s Strip build regenerates Templ code, so stale generated files fail the deploy
neon-branching.yml	infra	Provisions branched Postgres per PR	See Data Model. Keeps one preview-ready PR comment, marked <!-- beef-preview-ready -->, and updates it on later pushes
playwright-smoke.yml	integration	Wait for Preview, then Run Smoke Tests against the PR’s preview	If provisioning failed, Wait for Preview fails and Run Smoke Tests is skipped instead of failing a second time. Re-running the failed jobs re-runs the smoke job
docs-quality.yml	docs	markdownlint-cli2 over docs/**/*.md; lychee link-check (offline on PR, network on cron); pnpm build of Astro Starlight; ADR template check; proto sync check; docs-required check (.github/workflows/docs-quality.yml)
ai-hygiene.yml, up-to-date-check.yml	meta	non-test gates

PRs merge through the GitHub merge queue instead of a “branch must be up to date” rule. Every workflow that produces a required check also runs on merge_group, so each PR is tested once against the latest main at merge time. Per-PR jobs (Neon/Railway previews, Playwright smoke, docs-required) skip there because they already ran on the PR head. Merge main into a PR branch only to resolve conflicts.

There is no central monorepo test workflow — each service runs its own pipeline.

Flaky Test Policy

TODO(@law): no documented policy. Suggested: any test that fails non-deterministically must be either fixed within one business day or quarantined with a Linear issue referenced in the skip annotation. Update this section once the policy is ratified.

When to Write a Test

A test is required for any change that touches:

• Billing or payments — see ADRs saga-pattern-for-distributed-payments, distributed-locks-for-payments, deferred-subscription-activation. These flows have non-trivial state machines and partial-failure paths.
• Auth, RBAC, or session — anything involving Clerk, JWT verification, or StripAdminRole.
• Generation pipeline — credit debit, brain generation insert, S3 upload, media row creation. The flow crosses three services.
• Schema migrations — sirloin SQL or Prisma. Add a regression test that exercises the new column / constraint.
• Public REST or gRPC surface — proto changes need handler tests.

A test is encouraged for everything else. A test is not required for: docs-only changes, dev-only scripts, generated code, or formatting-only PRs.

Cross-References

• Per-framework commands and fixtures: Testing
• Data flow context: Data Model Overview
• Mockery config: apps/sirloin/.mockery.yml

Open follow-ups

1. Flaky test policy ratification (TODO(@law)).