Media Generation

Media Generation

Purpose

Document the user-facing generation flow from Brisket through Sirloin, Brain, and Round.

Participants

• Brisket initiates generation and displays credit pricing.
• Sirloin validates usage, deducts credits at request time, and stores user-facing media state.
• Brain handles async generation records, queues, provider integrations, and calls to round.
• Round serves internal model inference.
• Workflow-backed generations (media.workflow_name set, isBrainWorkflowMedia routing in sirloin) execute on Brain’s workflow engine.

Sequence

sequenceDiagram
  participant Brisket
  participant Sirloin
  participant Brain
  participant Round
  Brisket->>Sirloin: Create media/generation request
  Sirloin->>Sirloin: Validate usage and deduct credits
  alt workflow_name set
    Sirloin->>Brain: Execute published workflow
  else default path
    Sirloin->>Brain: Send media to brain
  end
  Brain->>Round: Inference when needed
  Brain-->>Sirloin: Generation status/result
  Sirloin-->>Brisket: User-facing media state

Source Links

• apps/brisket/src/hooks/use-credits.tsx
• apps/brisket/src/lib/constants.ts
• apps/sirloin/internal/app/services/media/
• apps/brain/src/modules/

Image Moderation

Brain can moderate input or source images before generation when an image is present, and it moderates generated outputs before they become available. Pre-generation moderation uses visual checks only today. Post-generation moderation also runs demographics age scoring and can run celebrity checks for NSFW generations.

A flagged single output fails the generation. For carousel output, Brain moderates every panel and drops the flagged ones; the run fails only if no panel passes. Sirloin then applies the product minimum of max(2, ceil(priced / 2)) delivered panels (3 of 5). Below it, the carousel fails with a full refund. At or above it, the carousel is delivered and the undelivered panels are refunded. There is no partial status: a carousel is either AVAILABLE or FAILED. See Media Credit Costs. See Image Moderation And Age Scoring for checkpoint rules, age-scoring thresholds, and provider fallback behavior.

State Transitions

Sirloin deducts credits when the request is accepted. Brain records and processes async generation, including workflow-backed media when media.workflow_name is set. Re-edits with source_id can be free.

Reference To Video

REFERENCE_TO_VIDEO is a Kitsune media generation path for creating a video from a required video prompt and character reference imagery, optionally guided by a reference video and pose image. Fennec exposes it in the media generator with image_path, video_path, video_prompt, video_provider, and a text-to-video model selection. image_path is a pose image and video_path is an optional source video.

Brain supports this generation type with WaveSpeed and Atlas providers. Before scheduling provider work, Brain resolves the character’s latest body and frontal-face onboarding images, preferring NSFW variants when the flow is NSFW and both NSFW body and face references exist. It upscales the body and face references through Fal-backed UpscaleImage commands, storing each result under a global deterministic media/reference-upscales/{hash}.{output_format} cache key. This reference-to-video flow currently requests jpg output. Cache hits are marked as Cached in generation events. Brain then sends those upscaled images, the selected pose image when present, and the selected reference video when present to the provider text-to-video endpoint. If the request has a shorter video duration than the selected reference video, Brain first stores or reuses a deterministic media/reference-video-cuts/{hash}.mp4 trim and sends that shorter reference.

When a media example was originally generated by REFERENCE_TO_VIDEO, replaying that example uses the same REFERENCE_TO_VIDEO flow. The original pose image and source video are reused only when the source generation metadata has sourceImagePath or sourceVideoPath; examples created from character references alone replay without sending a reference video.

Successful output is stored as the canonical video path and a compressed 480p variant. Brain also tries to extract the first video frame with ffmpeg for the preview image; if extraction fails, it falls back to the selected pose image or the upscaled body reference.

Welcome Pack Dispatch

Automatic welcome images apply only to legacy characters. The character-status worker queues up to ten images for eligible legacy characters; recipe-driven CV2 characters skip this step. Recipe creation previews still come from the recipe’s own workflow.

A character’s welcome pack is the set of media rows Sirloin creates at character creation from stored examples, rather than from a user’s generation request. Sirloin builds them in apps/sirloin/internal/app/services/media/welcomepack.go, the single place every welcome-pack producer goes through, so a row’s shape does not depend on which caller assembled the pack.

Welcome pack rows follow the same routing rule as any other Sirloin-created generation, so an example that is workflow-backed runs its own workflow:

• A row created from a workflow-backed example carries both workflow_name and the example’s authored contract inputs in generation_inputs_json. The worker branches on workflow_name and dispatches the row to Brain’s workflow engine rather than the legacy media path.
• The contract inputs are resolved from the stored example row through the same merge the user-initiated path uses — including the _url/_path rename fallback — so replaying an example as a welcome pack item and generating it from explore trigger the workflow identically.
• A row created from a legacy example (no workflow name and no family key) stays on the legacy media path, which is where it belongs.
• Only an uncaptioned item replays its example. A captioned item generates from its caption, is not linked back to the example row, and keeps the legacy path.

Both columns are written together or neither is: a name with no inputs fails Brain’s contract validation as a permanent 4xx, which fails and refunds the generation instead of running it. A workflow-backed example that cannot be resolved to a complete dispatch — it stores no contract inputs, or its stored inputs are unreadable — keeps the legacy path it runs on today, and the reason is logged.

The moderator-only shop-VI catalogue pack is a separate producer that builds its own rows inside Brain and does not go through this path.

Provider Routing

Brain routes image and video generation to external inference providers (Atlas, WaveSpeed, RunPod, FAL). Each generation type has an executor that manages an ordered fallback chain of providers — if the first provider fails to schedule, the executor tries the next.

For image and image-sequence generation, the fallback chain order is configurable at runtime via the inference-traffic-split application setting. This enables progressive rollout of new providers by controlling what percentage of requests try each ordering. Video executors use the provider specified in the job data without fallback chains.

See Brain Inference Providers for the full provider inventory, adapter architecture, and traffic split configuration.

Invariants

• Credit deductions happen at request time, not completion time.
• NSFW pricing and full-access credit behavior must follow the media credit-cost standard.
• Brain owns generation records; Sirloin owns user-facing media state.
• REFERENCE_TO_VIDEO requires video_prompt, video_model, and video_provider set to WAVESPEED or ATLAS; image_path and video_path are optional input references.
• REFERENCE_TO_VIDEO requires a character body reference and frontal-face reference image before provider scheduling starts.
• A carousel that delivers at least MinCarouselPanels panels, full or short, completes in one guarded transaction (CompleteCarouselMedia). The parent turns AVAILABLE, one child row is created per delivered panel, and any undelivered panels are refunded together.
• A carousel below MinCarouselPanels does not reach CompleteCarouselMedia. completeCarousel calls FailMediaAndRefund instead (apps/sirloin/internal/app/worker/checkmediageneration.go), so the parent turns FAILED, no child rows are created, and the full cost is refunded as a failed_generation_refund.
• Brain drops a carousel panel only for a content verdict. A moderation outage on any panel (no result, or every provider failed) fails the whole run, so the user is refunded in full and can retry (apps/brain/src/modules/domain/workflow/nodes/core/media-output.node.ts).
• Both transactions skip a carousel that is already AVAILABLE or FAILED. So a webhook/poller race cannot duplicate children or refunds, and a late success after a timeout-fail cannot deliver a carousel that was already refunded.

Error Paths

Insufficient credits stop the request before async work starts. Provider or queue failures surface through Brain status and logs. Workflow-backed failures must preserve media state and execution traces in Brain for inspection.

When Brain blocks or fails a generation for moderation, it records a normalized media failure reason on the generation metadata and Sirloin persists it on media.media.failure_reason. ListMedia exposes that value as Media.media_failure_reason so Brisket can show stable user-facing failed-media messages for underage, celebrity, nudity, illegal, offensive, or generic moderation failures.

A Fal rejection is classified as EXTERNAL_MODERATION_FAILED or EXTERNAL_VALIDATION_FAILED, and Brisket shows its generic failure message for both; see Brain Errors.

Carousel Partial Delivery Rollout

FOXY-925 ships Brain, Sirloin and the proto change in one release, because the behaviour can only be tested end to end. Railway builds each service independently, so deploy order is a manual release step:

1. Deploy Sirloin first and confirm it is healthy. On its own it only refunds short carousels from provider shortfalls; the old Brain still fails the whole run when any panel is flagged.
2. Then deploy Brain, so it delivers the panels that passed moderation.

If Brain goes live before Sirloin, users pay the full price for 1-4 panels with no refund. For the same reason, once the new Brain is live, do not roll back Sirloin alone: roll back Brain first, or both together.

Tests And Verification

• cd apps/sirloin && make run-tests
• cd apps/brain && pnpm test
• cd apps/flank && pnpm test (workflow-authoring editor regressions)

Related Decisions And Standards

• Media Credit Costs
• Auth Boundaries