Media tagging architecture

Media tagging architecture

Sirloin tags explicitly enrolled Brain Generations before publication and tags catalogue examples asynchronously. Brain owns source eligibility and revisions. Sirloin pulls its descriptors, prepares media, calls the provider, and stores results. Brain does not produce semantic tags. Manual catalogue categories remain separate.

Customer-authored prompt provenance has a separate capture and privacy lifecycle. Paid prompt analysis remains deferred. This media release adds no tag editor, recommendations, long-video support or automatic tagging of every user generation.

Media flow

flowchart LR
  B[Brain: enrolled Generation] -->|bounded inventory pull| S[Sirloin admission]
  S[Sirloin admission] -->|pull descriptor| B
  C[Catalogue sync: new or changed example] -->|publish without waiting| E[media.examples]
  E --> S
  S -->|fresh checks and full preparation| W[Durable work identity]
  W -->|WorkID wakeup| R[River worker]
  R -->|reserve and final source check| P[Provider]
  P -->|labels and cost| A[Guarded acceptance and accounting]
  A -->|current proven result| E

Admission uses bounded discovery and separate quotas for failed imports, known-source repair and due work. A durable UUID cursor prevents low IDs from monopolizing repair. A failed Brain observation is recorded before its discovery cursor advances. A missed River insert is repaired from persisted work after restart. Bun storage and River inserts are deliberately separate commits. Brain completion no longer posts a Sirloin tagging nudge; the bounded pull discovers enrolled Generations. The existing queued nudge consumer remains temporarily to drain earlier jobs.

A queued WorkID is a wakeup, not spending authority. Dispatch checks the source before preparation, reservation, sending and acceptance. Reservation binds the locked policy epoch, recipe, source incarnation, revision and eligibility epoch. Pilot admits explicitly enrolled sources; continuous mode auto-enrolls catalogue rows unless explicitly opted out. Changed enrolled sources remain selected, subject to fresh authority and the daily cap. The final check also rejects expiry, pause and elapsed request deadlines. Remote I/O occurs outside database transactions.

Source updates invalidate current pointers and pending authority. Storage operations preserve accepted historical evidence. Reuse after a revision-only change requires fresh preparation proving identical complete input. Published copies may have different object keys, so cross-source reuse compares ordered roles, source bytes and prepared bytes. An explicit catalogue result takes precedence over a borrowed Brain result; delayed older runs cannot replace newer results.

These checks detect observed changes. They cannot make Brain reads, Sirloin writes and provider requests atomic. A change after the final check can still occur. Reconciliation withholds detected stale results; it cannot recall transmitted media.

Active media schema

Migration 178 installs six media tables after main migration 177. After main migrations 179–183 (funnels, Primer payment type, asset-root hardening, and billing obligation support), provenance migration 184 adds four prompt tables and media birth protection; paid analysis remains deferred. There is no legacy-table creation, import or retirement path. apps/sirloin/internal/app/migrate/schema/178_media_tag_work_expansion.sql defines the media relationships below. Existing previews require verified schema and migration-ledger reconciliation before deploying these renumbered migrations; do not reset data or force a migration replay.

erDiagram
  EXAMPLES ||--o| TAG_SOURCES : "example_id + instance, CASCADE"
  TAG_SOURCES ||--o{ SEMANTIC_TAG_RESULTS : "source_id, CASCADE"
  TAGGING_POLICY ||--o{ TAGGING_ATTEMPTS : "policy_id, RESTRICT"
  TAGGING_ATTEMPTS o|--o| SEMANTIC_TAG_RESULTS : "accepted_attempt_id + work id, RESTRICT"
  SEMANTIC_TAG_RESULTS o|--o{ EXAMPLES : "result pointer, SET NULL"
  TAGGING_POLICY { boolean id PK bigint epoch boolean enabled timestamptz cost_breach_at text mode bigint daily_cap_microusd }
  TAG_SOURCES { uuid id PK text namespace text source_key bigint incarnation bigint revision bigint eligibility_epoch bytea quarantine_key jsonb required_roles }
  SEMANTIC_TAG_RESULTS { uuid id PK uuid source_id FK text recipe bytea media_digest integer run_no uuid authorization_id text status text_array tags uuid accepted_attempt_id FK }
  TAGGING_ATTEMPTS { uuid id PK uuid work_id boolean policy_id FK bytea quarantine_key bigint reserved_microusd bigint final_microusd text state date budget_day }
  EXAMPLES { uuid id PK uuid tagging_instance bigint tagging_revision boolean tagging_enrolled uuid semantic_tag_result_id FK }
  TAGGING_CHECKPOINTS { text name PK bigint cursor uuid source_cursor }
  TAGGING_RECOVERY_FAILURES { text source_key PK bigint generation_db_id timestamptz failed_at }

tag_sources has one current authority row per namespace and source key. semantic_tag_results has one work/result row per source incarnation, recipe, full prepared-input digest and authorized run. Explicit reruns create immutable new runs, with at most two outstanding authorizations. Replaying an action ID does not add work or attempts.

tagging_attempts retains money and request fences after source deletion. Its work_id intentionally has no ownership FK. The accepted-result FK proves that acceptance references an attempt for that work. A source-independent quarantine hash prevents unknown charges from being bypassed by new revisions, recipes, digests or incarnations.

The singleton policy serializes budget reservation and stores approved caps. Checkpoints and recovery failures provide bounded restart and discovery repair. A single example/result table cannot represent unpublished Brain sources, shared accepted evidence, retained financial liabilities and durable discovery failures.

Tags use TEXT[]. Required and observed roles use JSONB because each entry carries role, index, identity and byte digests. Sirloin incarnation and revision use positive BIGINT. Brain’s tagging_source_revision is NOT NULL with default 1; normal writers increment it, but the database does not enforce positivity. Brain’s existing inventory DB ID remains limited by its Prisma Int and API int32 representation; cursor storage does not expand that wire limit.

Accounting, recovery and writes

The ledger charges known final cost or conservatively retains the request ceiling. Daily exposure includes today’s attempts and unresolved older liabilities once. UTC rollover never clears quarantine. Lifetime caps include retained history. First settlement of a trustworthy cost above its ceiling also records tagging_policy.cost_breach_at in the same transaction. A non-null marker makes the effective policy disabled. It blocks new reservations, sending, final dispatch checks and new result acceptance. A late settlement from an older policy halts the current policy. Replaying an already settled attempt does not halt it again after operator acknowledgment.

Never-sent reservations expire to zero-cost cancellation. Sent-marked crashes and timeouts become unknown, including a failure between marking and HTTP. Late trustworthy settlement closes only that attempt. It cannot revive labels or extend its deadline. Rescue continues while paid dispatch is disabled. Exhausted runs leave the due queue rather than starving healthy work.

The fleet dispatch lock remains. A database active-attempt constraint is an additional guard. Ordinary catalogue writers lock sources and examples in UUID order without locking the spending policy. Spending and settlement transactions lock policy before their source, work and attempt rows. They update revision and invalidate pointers in their existing transaction. Privacy erasure scrubs derived content while retaining accounting.

The media migration creates no SQL functions or triggers. Storage transactions enforce accepted-result state, valid attempt transitions, one-shot settlement, provider-response identity and operator-only rerun authorization. The worker uses the existing application database role; policy enablement, budgets and rerun authorization remain controlled by application paths.

Guarantee	Enforcement after simplification	Boundary
References, uniqueness, scalar types and valid row shape	SQL FK, indexes and CHECK	Retained in migration 178
Accepted evidence, settlement and attempt transitions	Locked storage operations and idempotent guards	Trusted application writes
Rerun authorization	Storage checks operator privilege before creating a run	Operator authorization is enforced through storage operations
Stop after a cost breach	Durable marker committed with the first cost settlement	Automatic halt keeps the epoch; explicit operator reconfiguration clears the marker and advances the epoch

Changed trust boundary: PostgreSQL no longer compares old and new accepted evidence, settled costs, send timestamps or halt state through triggers. Direct SQL using the application role can bypass those application guards. Maintenance and operator tools must use the storage operations; raw SQL is trusted and must preserve these invariants. This design does not claim protection against a compromised writer credential.

Activation and evidence

Paid media dispatch defaults off. Deployment does not authorize a larger cohort or budget. Read the deployment runbook before enabling.

Local tests use synthetic media, controlled HTTP and disposable PostgreSQL. They establish implementation behavior, not hosted readiness or live provider quality. Historical Wei evidence covers the pinned GLM recipe and evaluated sample only. Carousels, population-wide quality, privacy routing and current operational provider behavior remain separate activation gates.