Testing Strategy

This page is policy and pyramid. For commands and per-framework usage, see Testing.

Pyramid Per Service

Each service has its own pyramid because the tooling and risk profile differ.

sirloin (Go) — wide unit base, real-DB integration tier

Layer	Tool	What counts	Where
Unit	testing + testify	Pure handler / service / repo logic with mocked deps	*_test.go next to source
Integration	testify/suite + testcontainers (Postgres) + MinIO test server + bufconn gRPC server	Real DB, real S3, in-memory gRPC	apps/sirloin/internal/pkg/testingsuite/
Sandbox / external	Same suite, build-tagged sandbox / integration	Hits Chargebee sandbox	make run-tests-all
Coverage gate	verify-coverage target in apps/sirloin/Makefile	Threshold currently 3% (Check test coverage meets threshold (default 3%))	CI gate

The sirloin pyramid is genuinely integration-heavy — BaseTestSuite / GRPCTestSuite give you a real Postgres per suite, so most “unit” tests in sirloin are technically integration tests. This is by design (real DB catches BUN serialization bugs). See Testing for fixtures.

brain (NestJS) — unit-heavy with NestJS testing modules

Layer	Tool	What counts	Where
Unit	Jest + @nestjs/testing	Service / processor logic with useValue mocked providers	*.spec.ts next to source
Integration	None — no integration suite found in apps/brain/src/**; brain uses Jest unit specs only	—	—
Coverage gate	None enforced — apps/brain/package.json:24 runs jest --coverage but no coverageThreshold is set	—	—

brisket (Next.js) — component + hook unit tests

Layer	Tool	What counts	Where
Unit	Vitest + @testing-library/react (not Jest — common gotcha)	Component / hook / utility tests	colocated with source
E2E / API	Playwright	Cross-service flows against a deployed stack	tests/
Coverage gate	None enforced — apps/brisket/vitest.config.ts does not set coverage.thresholds	—	—

fennec (React) — typecheck + lint as the floor

Layer	Tool	What counts	Where
Type	tsc --noEmit	Compile-clean TS	pnpm tsc
Lint	ESLint with --max-warnings 0	Zero warnings tolerated	pnpm lint
Unit	Vitest + @testing-library/react + @testing-library/jest-dom	Component / hook tests	colocated
Coverage gate	None enforced — no Vitest config in apps/fennec/ and no coverageThreshold in apps/fennec/package.json	—	—

flank (TS workflow engine)

Layer	Tool	What counts	Where
Unit	Vitest	Workflow + adapter logic	colocated
Schema validation	pnpm validate:seeds	Workflow seed JSON conforms	apps/flank
Coverage gate	None enforced — apps/flank/vitest.config.ts does not set coverage.thresholds	—	—

round (Go ML) and strip (Go SSR)

Layer	Tool	Notes
Unit	Go testing	make run-tests with -race
Integration	None enforced — both services run only go test -race -count=1 ./... (apps/round/Makefile:5, apps/strip/Makefile:28); strip exposes test-coverage targets but no integration tier	—

Cross-service E2E

• Playwright under tests/ (config tests/playwright.config.ts).
• Run with cd tests && npm run test:api.
• The Playwright config (tests/playwright.config.ts) defaults ENV=dev, pointed at https://fennec-dev-api.sexty.dev/ (tests/testConfig.ts). No GitHub workflow under .github/workflows/ invokes the tests/ Playwright suite — it is run manually (cd tests && npm run test:api).

Coverage Targets

Service	Target	Source
sirloin	3% (verify-coverage)	apps/sirloin/Makefile — note this is a low floor, real coverage is much higher
brain	None enforced — jest --coverage runs but no coverageThreshold is configured in apps/brain/package.json
brisket	None enforced — apps/brisket/vitest.config.ts has no coverage.thresholds
fennec	None enforced — no Vitest config nor coverageThreshold in apps/fennec/package.json
flank	None enforced — apps/flank/vitest.config.ts has no coverage.thresholds
round	None enforced — apps/round/Makefile only runs go test -race -count=1 ./...
strip	None enforced — apps/strip/Makefile:31-44 exposes test-coverage/test-coverage-summary targets but does not gate on a threshold

The sirloin 3% threshold exists to prevent regressions, not as an aspirational target. Treat current coverage as the de facto floor when you touch a package.

Mocking Policy

Go (sirloin, round, strip) — mockery v3

• Mocks generated from interfaces by mockery v3, configured in apps/sirloin/.mockery.yml.
• Output naming: mock_*.go, same directory as the interface.
• Generate via make generate-mocks.

When to mock vs use real:

Dependency	Mock?	Why
Postgres	No — use TestDatabase (testcontainers)	BUN serialization bugs only surface against real PG
S3 / object storage	No — use EnableMinIOTestServer()	Catches HMAC + bucket-policy issues
gRPC peers (brain, round)	Yes — generated mock	We don’t want test runs cross-spawning services
Chargebee	Yes for unit tests; real sandbox for integration-tagged tests	Sandbox is rate-limited (-p 1 enforced in run-tests-all)
Clock / random	Yes	Determinism

TS (brain, brisket, fennec, flank)

• Brain: Test.createTestingModule + useValue for provider injection. Mocks live inline in beforeEach per test file.
• Brisket / fennec: prefer Testing Library + vi.mock for module-level mocks. Avoid mocking React components — use real DOM.

Test Data

Service	Mechanism	Source
sirloin	FixtureBuilder (functional options) — CreateUser, CreateCredit, CreateCharacter, CreateMedia with WithImageCredits, WithCharacterStatus, etc.	apps/sirloin/internal/pkg/testingsuite/fixtures.go
brain	No centralised factories found in apps/brain/src/**/*.spec.ts; tests construct objects inline (e.g. media-processing.service.spec.ts).
brisket / fennec / flank	Inline data per test — no shared factory module under apps/{brisket,fennec,flank}/src/.

Seed scripts (production / dev environments rather than tests) live per service. The only first-party seed runners found are apps/flank/server/seed.ts and apps/flank/server/seed-sync.ts (workflow/adapter seeds). sirloin and brain have no committed dev-seed scripts; local data comes from make dev-up-d plus manual flows.

CI Test Execution

Workflow files in .github/workflows/:

Workflow	Service	Triggers tests	Notes
sirloin.yml	sirloin	make ci-all mirror — verify-mod-tidy, verify-migrations, modernize, lint, govulncheck, build, run-tests, verify-coverage	Unit only; run-tests-all (Chargebee sandbox) is gated
brain.yml	brain	pnpm install → prisma generate → pnpm lint / pnpm tsc / pnpm test (three parallel jobs in .github/workflows/brain.yml)
brisket.yml	brisket	pnpm install → pnpm lint / pnpm typecheck / pnpm test (three parallel jobs in .github/workflows/brisket.yml)
fennec.yml	fennec	lint + tsc with max-warnings 0
flank.yml	flank	typecheck + lint + test + validate:seeds
neon-branching.yml	infra	Provisions branched Postgres per PR	See Data Model
docs-quality.yml	docs	markdownlint-cli2 over docs/**/*.md; lychee link-check (offline on PR, network on cron); pnpm build of Astro Starlight; ADR template check; proto sync check; docs-required check (.github/workflows/docs-quality.yml)
ai-hygiene.yml, pr-description-check.yml, up-to-date-check.yml	meta	non-test gates

There is no central monorepo test workflow — each service runs its own pipeline.

Flaky Test Policy

TODO(@law): no documented policy. Suggested: any test that fails non-deterministically must be either fixed within one business day or quarantined with a Linear issue referenced in the skip annotation. Update this section once the policy is ratified.

When to Write a Test

A test is required for any change that touches:

• Billing or payments — see ADRs saga-pattern-for-distributed-payments, distributed-locks-for-payments, deferred-subscription-activation. These flows have non-trivial state machines and partial-failure paths.
• Auth, RBAC, or session — anything involving Clerk, JWT verification, or StripAdminRole.
• Generation pipeline — credit debit, brain generation insert, S3 upload, media row creation. The flow crosses three services.
• Schema migrations — sirloin SQL or Prisma. Add a regression test that exercises the new column / constraint.
• Public REST or gRPC surface — proto changes need handler tests.

A test is encouraged for everything else. A test is not required for: docs-only changes, dev-only scripts, generated code, or formatting-only PRs.

Cross-References

• Per-framework commands and fixtures: Testing
• Data flow context: Data Model Overview
• Mockery config: apps/sirloin/.mockery.yml

Open follow-ups

1. Flaky test policy ratification (TODO(@law)).