Media tagging operations and deployment

Media tagging operations and deployment

Media tagging uses one persisted policy and immutable work runs. It no longer requires rotating finite campaigns. The URL is retained for existing links. See architecture for ownership, relationships and privacy rules.

All commands below run from apps/sirloin. Use the intended database and existing Brain configuration. Keep secrets outside checked-in files. Commands default to dry run. Mutations require --dry-run=false --confirm and the current --expected-epoch. Configuration and activation are separate actions. No command here grants new budget or cohort approval.

Before deployment

1. Keep paid media dispatch disabled during deployment. Check Brain source-revision support. On an existing Brain database, run apps/brain/prisma/migrations/20260927120000_add_tagging_source_revision/prebuild_concurrent_index.sql through psql on a direct owner connection with target_database=<confirmed database> and confirmation=PREBUILD_BRAIN_TAGGING_ENROLLMENT_INDEX before the Brain migration. The script pre-adds the two columns and builds the exact enrolled-generation inventory index concurrently; confirm that it reports ready and valid. Do not run it against main or production without a deployment approval. Sirloin discovers enrolled Generations through a bounded pull; Brain no longer sends tagging nudges. The old queued consumer drains earlier jobs.
2. This iteration installs one media migration, 178, after main migration 177; it creates no feature SQL functions or triggers. Historical previews built from the retired feature schema needed an approved rebuild from main. For a current preview with identical schema, verify the DDL before aligning migration metadata; do not reset it or rewrite its ledger to mimic a fresh database.
3. For a large existing catalogue, run the concurrent index prebuild through a direct owner connection before migration 178. It needs target_database and confirmation=PREBUILD_MEDIA_TAG_EXAMPLE_INDEXES. Confirm both indexes are valid and ready.
4. Apply the forward migrations through the repository migrator. They create only the six active media tables. Prompt lifecycle tables and media birth columns are deferred.
5. Deploy the current workers. Use bounded recovery and report commands to inspect state. The stored policy controls paid dispatch. Source discovery may import eligible sources while policy is disabled; review that cohort before activating spending.

Rebuilding preview does not erase historical provider charges or grant spending authority. Retain external spend and approval records. Activation remains a separate approved operation. No legacy import, retirement migration or historical schema fixture ships with this iteration.

Policy changes, reruns and financial reconciliation must use tagging-ops with operator authorization. The worker uses the existing application database role. Accepted-evidence, cost and transition guards run in storage transactions; direct SQL can bypass them. Do not repair accepted tags, send timestamps or settled costs with ad hoc SQL.

Policy and runtime controls

Policy storage owns recipe, vocabulary, mode, optional expiry, daily cap, optional lifetime cap and attempt limits. Operator writes advance the epoch and clear any acknowledged cost_breach_at marker. Automatic cost-breach halt records the marker without changing the epoch. The effective policy reports disabled while this marker is present; the configured enabled column remains operator-owned. The disabled migration seed retains the historical FP8 recipe and authorizes no cohort. Before enrolling or activating work with the current binary, explicitly authorize the FP4 recipe under the approved policy; deployment alone does not change stored policy.

Control	Meaning
Stored tagging_policy.enabled	Sole operator switch for paid sends; the seeded policy is disabled. Changes take effect without a process restart.
Stored tagging_policy.request_ceiling_microusd	Reserved request amount, bounded to 2,000,000 microUSD by the runtime.
OPENROUTER_API_KEY	Existing provider credential shared with Sirloin. An enabled policy needs a configured key before sending.
SIRLOIN_TAG_SCRATCH_ROOT	Private temporary preparation directory.

Each recovery sweep uses code limits: 10 newly imported sources, 10 Brain discovery items, 10 known-source repairs, and 5 failed-observation retries. Due-work repair has its separate bound. The former tag dispatch, provider-key, ceiling, sweep-limit, automatic-campaign, and runtime recipe/vocabulary variables no longer control this worker. Do not retain old operational recipes.

Enrollment and policy

1. Inspect bounded --action=inventory pages for catalogue examples or explicitly enrolled Brain Generations. Brain enrollment uses its existing authenticated enrollment API.
2. Authorize a paused policy with --action=authorize --expected-epoch=<current> --mode=pilot --recipe=media-aggregate-v2-parasail-fp4 --vocabulary=v1.4 --daily-cap-microusd=<approved> --request-ceiling-microusd=<approved> --max-attempts-per-run=<approved> --reservation-seconds=<approved> --send-seconds=<approved>. Lifetime, source attempt caps and expiry are optional additional limits. Apply only within existing budget and media-transfer authority.
3. For a catalogue pilot, select each example with --action=enroll --example-id=<UUID> --expected-epoch=<current>. Preview first, then apply with --dry-run=false --confirm. Pilot ignores NULL and false enrollment. A changed enrolled example remains selected. --action=opt-out persists false and withdraws its evidence.
4. After route, privacy, sample quality and operational gates pass, preview and apply --action=activate --expected-epoch=<current>. Source discovery uses bounded code defaults; the stored policy activates paid dispatch. Activation itself sends no provider request.
5. Use --action=backfill and --action=recover with the current epoch and a bounded page limit. Neither calls the provider directly. Follow with --action=report --page-limit=50, continuing --after-work-id=<UUID>.

Continuous mode uses --mode=continuous and the same required daily cap, request ceiling, per-run limit and finite deadlines. It automatically enrolls NULL catalogue rows and honors explicit false opt-out. Brain discovery remains restricted to Brain-enrolled sources in both modes. Switching modes requires approval. Publication never waits for tagging.

Paid dispatch requires at least 150 seconds remaining on both the job and the stored attempt deadline after the final authority checks. Allow additional time for those checks when choosing --send-seconds; the maximum is 240 seconds. The HTTP call is capped by the earlier of its 120-second timeout, the job deadline and the stored attempt deadline minus a 30-second finalization margin. If too little time remains before dispatch, the provider is not called and the reservation is closed with zero cost. A response that exceeds the stored deadline still cannot publish tags; any known cost remains recorded.

There is no manifest file, digest, frozen revision selection or pilot admission window. Re-tagging changed enrolled input can consume more of the approved daily budget. Fresh source and complete-byte checks still govern result reuse.

Reservations include qualified retries and operator reruns. UTC rollover permits new daily exposure only under the existing policy; unresolved older charges still count. A source-wide unknown charge blocks further work across recipe, revision, digest and incarnation changes.

To close a sent or unknown liability with verified provider evidence, preview --action=reconcile --attempt-id=<UUID> --final-microusd=<verified amount> --evidence-reference=<stable reference>. Inspect the returned attempt, provider response ID and proposed amount, then apply the same arguments with --dry-run=false --confirm. Keep the evidence reference in the operational audit record. This action settles cost only; it does not restore erased labels or bypass quarantine without a trustworthy final amount. Unknown liabilities do not age out automatically.

Rerun, pause and rollback

For a reviewed rerun, use --action=rerun --work-id=<UUID> --action-id=<stable UUID> --expected-epoch=<current>. Preview, then apply. Reuse the action ID on retry. It creates a new immutable run, not a reset of accepted evidence or consumed attempts. At most two outstanding reruns are allowed; reservation follows authorization order.

--action=pause --expected-epoch=<current> stops new paid work and retains accepted tags. The final pre-send check observes pause and expiry even if preparation began earlier. Repair, cleanup and rescue remain active. Sent-marked uncertainty retains its ceiling until trustworthy settlement.

For one immutable run, use --action=rollback --work-id=<UUID> --recipe=<recipe> --expected-epoch=<current>. The selected run loses its payload and current pointers. Other runs of the same recipe keep their claims and results. This action does not pause or advance the policy. It retains attempt accounting and permits late cost settlement. Repeating the same rollback has no further effect.

Without --work-id, --action=rollback --recipe=<recipe> --expected-epoch=<current> remains the broad recipe operation. It atomically pauses the policy and erases evidence and pointers for that recipe. It retains financial history and authorization. Preview the intended scope before applying either command. Reprocessing requires separate approval.

To stop tagging one catalogue example without deleting it, use --action=opt-out --example-id=<UUID> --expected-epoch=<current>. The operation persists disabled enrollment, withdraws its pointer and erases catalogue-owned tag payload. Catalogue sync cannot enroll it again. Shared Brain-generation evidence and financial accounting remain. Both commands default to dry run; applying them requires --dry-run=false --confirm. Verify the report after each operation.

After activation, use pause, per-recipe suppression and forward repair with a compatible binary. A database restore requires reconciliation of external spend; it is not an automatic rollback procedure.

For provider rate limits, dispatch selects the requested retry delay with a 60-second first floor and a 120-second later floor. Attempt storage adds that delay to the database clock inside the existing finish transaction. Dispatch checks use the database clock observed when work is read. Host clock differences cannot shorten these paid-retry floors.

Current media recipe

The current recipe is media-aggregate-v2-parasail-fp4: GLM-5.3-Flash through Parasail FP4 without fallback, vocabulary v1.4, 845 canonical names and 97 added definitions. It uses GLM only, not WD. Historical qualification artifacts identify the former media-aggregate-v2-parasail-fp8 recipe and its evaluated sample. Those artifacts do not qualify the FP4 route, new carousels, a larger population, private prompt routing or another provider configuration. Complete the route and sample-quality gates before activation.

Preparation preserves originals and uses private scratch. Each video source is limited to 31 seconds, 64 MiB, 25 megapixels and an 8192-pixel side. Distinct original and preview objects become sequential segments in one temporary video and one provider request. The composite can reach 40 seconds. Same-object aliases use one segment. A filename alone does not prove equal bytes or equal coverage, so _480p is not discarded on its name.

Aggregate bounds remain 96 MiB downloaded, 32 MiB prepared and 48 MiB per request. Required-role failure stops dispatch. Segment roles and ranges accompany the request without private object identities. Audio-stream presence alone does not prove audible evidence. Cuts alone do not prove motion, interaction or combined person counts. See provider, video and composition for exact parsing and preparation rules.

Short QA procedure

Use the PR1 QA checklist for a small prepared sample, four acceptance checks and a short result report. Engineering supplies automated-test evidence and handles setup and cleanup. Keep provider work synthetic unless a separate budget and media-transfer approval exists. Record the exact tested revision and environment. Prompt provenance and analysis are outside PR1.