AI Development

AI Development Guide

Goals

• Keep secrets and local operator state out of Git.
• Make AI behavior reproducible across contributors.
• Standardize MCP usage, skills, and agent instructions.

Local-Only Files (Must Stay Untracked)

• .mcp.json
• .agents/mcp_config.json
• .codex/config.toml
• .claude/settings.local.json
• apps/sirloin/.claude/settings.local.json
• .serena/
• apps/sirloin/.serena/
• .playwright-mcp/

Use these tracked templates when setting up a machine:

• .mcp.json.example (Claude Code MCP servers)
• .claude/settings.local.example.json (Claude Code MCP allowlist and permissions)
• .agents/mcp_config.json.example (Antigravity workspace MCP servers)
• .codex/config.example.toml (repository Codex MCP defaults)
• .codex/environments/config.example.toml (optional Codex MCP servers)
• .codex/environments/README.md

For Antigravity, copy .agents/mcp_config.json.example to .agents/mcp_config.json (go run scripts/copy-env-example.go -example .agents/mcp_config.json.example -dest .agents/mcp_config.json), fill in local credentials there, and use "disabled": true on individual server entries to keep unneeded servers off. Keep ~/.gemini/config/mcp_config.json free of Beef servers ({"mcpServers": {}}) so repository MCPs stay workspace-scoped rather than global (agy mcp CLI subcommands edit only the global file).

Greptile Repository Default

beef-greptile is enabled in the Claude, Codex, and Antigravity templates. Its endpoint is https://api.greptile.com/mcp. Each developer authenticates in each client; the templates contain no credentials. See the Greptile setup guide.

On a new checkout, create the ignored runtime files from the tracked templates:

go run scripts/copy-env-example.go -example .mcp.json.example -dest .mcp.json
go run scripts/copy-env-example.go -example .claude/settings.local.example.json -dest .claude/settings.local.json
go run scripts/copy-env-example.go -example .codex/config.example.toml -dest .codex/config.toml
go run scripts/copy-env-example.go -example .agents/mcp_config.json.example -dest .agents/mcp_config.json

The helper refuses to overwrite existing files. For an existing checkout, merge only the beef-greptile entry from each template into the corresponding runtime file. Also add beef-greptile to Claude’s enabledMcpjsonServers.

Authenticate from the repository root:

• Claude: run claude mcp login beef-greptile, or select beef-greptile in /mcp and choose Authenticate. For SSH, use claude mcp login beef-greptile --no-browser.
• Codex: trust the repository, then run codex mcp login beef-greptile. For SSH, use codex mcp login beef-greptile --no-browser.
• Antigravity CLI: launch agy in the repository, open /mcp, select beef-greptile, and complete its OAuth prompt. The installed CLI’s agy mcp subcommands manage global configuration, so do not use them to register this server.

For a headless login, open the printed authorization URL on your machine, approve access, and paste the final redirect URL or requested code into the client that started login. Keep OAuth responses out of chat and tracked files. Restart each client after setup. Ask it to call get_me, select the owning organization, and call list_knowledge_bases. A successful read verifies access; server discovery alone does not. If the list is empty, check repository access and ask Greptile support about Knowledge Base enrollment.

MCP Hardening Rules

• Start from minimal server enablement (enableAllProjectMcpServers: false in .claude/settings.local.json, "disabled": true for unneeded Antigravity server entries).
• Greptile is the default context server. Enable other servers only when needed.
• Never commit bearer tokens, DSNs with passwords, or API keys.
• Prefer read-only credentials for analytics/database MCP servers.

Agent Instruction Files

• AGENTS.md is the shared cross-agent execution contract for this repo.
• apps/*/AGENTS.md carries cross-harness service notes for each app.
• CLAUDE.md imports the nearest AGENTS.md; do not duplicate guidance there.

Project Skills

Portable project skills live in .agents/skills; .claude/skills contains compatibility symlinks to the same files. Codex and Antigravity CLI read .agents/skills directly. Use knowledge-base whenever codebase context is needed, then verify relevant local sources. The set also covers task planning and delivery, plan review, change verification, docs, shared contracts, Brain migrations, billing triage, unit testing, Brisket code review, Foxy design/process work, and Neon Postgres. /prototyping is explicit-only: it skips repository gates to reach a Railway preview on a do-not-merge/ draft PR.

These are designed to reduce repeated prompt setup for common high-cost tasks.

Bundled skill helpers use standalone Go programs with no third-party packages. Run each helper with the command in its owning skill. make check-skill-helpers tests them and cross-builds for macOS as part of make check-agent-quality. The Foxy Tabs Chrome extension retains JavaScript to call Chrome’s browser APIs.

Agentic Task Quality

Before editing, dev-task requires a contract proportional to the task. A small, bounded change needs an outcome, acceptance statement, and check. Complex, risky, or ambiguous work also needs stable acceptance-criterion IDs, evidence oracles, scope and consumers, effect boundaries, and stop conditions. Compact contracts retain published process gates. Use the impact router to ground them:

go -C tools/agent run ./cmd/agent-impact -path apps/brain/src/example.ts -strict
go -C tools/agent run ./cmd/agent-impact -changed -strict

The first command predicts guides, consumers, risk, and checks for intended paths. The second fails when a changed path under an enforced repository root is unmapped. The router defines a minimum verification floor; task-specific risk can require more evidence. Unmapped enforced paths intentionally fail: new packages and tools need explicit verification rules rather than a catch-all that could select insufficient checks.

After merging the base branch into a pull request branch, route the merge commit:

go -C tools/agent run ./cmd/agent-impact -merge HEAD -strict

It reports Validation: focused only when the resolution is exactly the union of both sides’ changes, every resolved path belongs to a rule with merge_checks, and no resolved, pull-request, or same-area incoming path falls under a high or critical rule. Focused validation lists those merge checks and names the full checks it leaves to exact-head CI. Anything else reports Validation: full with its reasons and the normal checks.

The router and result grader share tools/agent/internal/agentpolicy in the dedicated tools/agent/go.mod module. Both reject malformed policy before resolving paths. There is no root go.mod or go.work. Application Go modules remain independent, and agent tooling adds no third-party dependency.

Run the module’s tests with go -C tools/agent test ./.... Each executable has its own directory under tools/agent/cmd/. The commands discover the repository root from their working directory. Relative input paths resolve from that root; trial output paths must remain absolute. Saved-trial inspection also works outside a checkout when using a compiled executable.

The vendor-neutral suite in .agents/evals/single-shot.json covers contracts, migrations, billing, UI, containers, docs, integrations, ambiguity, and unsafe requests. CI validates its schema and routing references. It also validates every existing skill evals/evals.json and the harness suite, including unique case IDs/names, required fields, and confined file and working-directory paths. These checks do not call a model:

make check-agent-quality

After a calibrated single-shot trial, grade its JSON result with:

go -C tools/agent run ./cmd/check-agent-evals -result /path/to/result.json

Trial execution is intentionally outside required PR CI. The grader requires the expected decision, in-scope paths, acceptance evidence, passing checks, no unauthorized effects, and a completed independent review with no unresolved Critical or Important findings.

The grader recomputes impact rules and required checks from changed paths. Each check record must match the manifest command and include its working directory, zero exit code, and captured-output evidence. This is structural validation, not cryptographic provenance: a trusted trial runner or reviewer must collect the result; the model under evaluation must not grade its own claims.

A read-only completeness review uses UNVERIFIED when required execution evidence is missing or does not match the reviewed head and diff. Source code, test files, and an author’s unsupported success claim cannot establish PASS.

Captured instruction trials

make agent-evals previews the harness suite without calling a provider or creating output. Opt in to execution with an exact model ID and a new absolute output directory outside the checkout:

make agent-evals
make agent-evals ARGS='-execute -harness codex -model gpt-6-astra -account main2 -ref HEAD -out /var/tmp/agent-scratch/codex-trial-01'
make agent-evals ARGS='-execute -harness claude -model claude-fable-5-1 -ref HEAD -out /var/tmp/agent-scratch/fable-trial-01'

Use an account available on your machine. Subswapper and the selected harness must already be installed and authenticated. -suite selects another skill eval file, -case selects one case, and -repeat repeats it. -timeout limits each process, including its subprocess group, and defaults to three minutes.

The runner archives the committed -ref, records its commit and tree IDs, and creates a separate Git inventory for nested instruction discovery. Uncommitted changes and local files are absent. Tracked environment files other than examples are omitted and listed in source.json. Unsafe archive paths and escaping symlinks fail setup. The suite is supplied from the invoking checkout and captured separately, so the same cases can compare two commits.

Each case records its prompt, rubric, working directory, exact argv, stdout, stderr, duration, exit status, observed model identifiers, and transcript hash. Output directories and files are private and never reused. Transport failures, timeouts, and reported model substitutions fail execution. A successful process still requires independent review of the response and tool records against the captured rubric. An empty observed-model list does not attest to model identity. Codex model fields remain observations and never set the verified-identity flag. Claude answer identity comes from assistant messages. Configured models and usage-accounting models are recorded separately; a background model in modelUsage does not by itself prove an answer-model substitution. Explicit fallback remains an error.

Use go -C tools/agent run ./cmd/run-agent-evals -inspect /absolute/trial-directory to check an existing transcript hash and reparse its evidence with the current parser. This prints a new assessment without calling a provider or overwriting the original artifacts. It retains recorded process failures and timeouts. Hashes detect accidental transcript changes; they do not authenticate editable metadata or establish independent provenance. New captures frame each stream separately. Inspection still accepts the legacy concatenated hash format for older evidence, which cannot detect bytes moved between stdout and stderr.

These are read-only instruction trials, separate from implementation-result grading above. They check decisions and observed instruction use, not executable acceptance criteria or a real browser journey. Codex uses its read-only sandbox; Claude receives only Read, Glob and Grep tools, with access to the snapshot root when a case starts in a nested directory. The runner limits configuration and MCP exposure. Child processes receive an explicit runtime environment allowlist, excluding inherited API keys, tokens, provider overrides, and shell startup hooks. Subswapper supplies account authentication from its configured storage. Managed policy, account routing, and readable host files remain external inputs; this is not a credential-isolating or hermetic container. See the current Codex non-interactive documentation and Claude programmatic usage when updating either adapter.

CI Guardrail

The instruction validator rejects canonical instruction/skill symlinks and references outside the repository. Claude compatibility skill symlinks must resolve to the canonical skill tree. It checks single-line Markdown links and reference definitions, plus exact inline-code paths to bundled resources. It skips fenced examples, commands, templates, runtime state files, and complex or multiline Markdown syntax. This is a bounded resource check, not a complete Markdown parser or proof of harness instruction loading.

scripts/check_ai_hygiene.go runs in CI and fails when:

• local-only AI files become tracked,
• known secret-bearing patterns are committed.

Run locally:

make check-ai-hygiene
make check-agent-quality