Image And Video Moderation

Image And Video Moderation

Purpose

Document how image and video moderation is applied during character onboarding and media generation, including checkpoint behavior, celebrity recognition and approved celebrities, demographics age scoring, stored evidence, and provider fallbacks.

Participants

• Brisket uploads reference images and displays onboarding moderation outcomes.
• Sirloin requests character reference verification and maps Brain errors into user-facing character dataset image failure reasons. It also stores each user’s approved celebrity and answers Brain’s lookup for it.
• Brain owns moderation orchestration, onboarding image results, generation checks, and persisted moderation evidence.
• Hive provides visual moderation, celebrity recognition, and demographics age scoring.
• Gemini can be used as a fallback for nudity and age estimates.
• Fennec reads stored moderation results for review and inspection, filterable by workflow.

Checkpoints

Checkpoint	Used for	Demographics age check	Celebrity recognition
ONBOARDING	Character reference images scored by Brain for Sirloin	Yes	Always when celebrity recognition is enabled
MATERIALIZE_ASSET	Generated character assets before they enter a durable dataset	Yes	Always when celebrity recognition is enabled
PRE_GENERATION	User or source media before generation	Video matching a declared upload storage identity or resolving under media-inputs/, with the postGeneration thresholds	Every NSFW generation
POST_GENERATION	Generated image or video before it becomes available	Yes	Every NSFW generation

Sequence

sequenceDiagram
  participant Brisket
  participant Sirloin
  participant Brain
  participant Hive
  participant Gemini

  Brisket->>Sirloin: Upload character reference image
  Sirloin->>Brain: Verify character reference image
  Brain->>Brain: Create onboarding result in PROCESSING state
  Brain->>Hive: Visual moderation
  Brain->>Hive: Celebrity recognition
  Brain->>Hive: Demographics age scoring
  alt Hive fallback needed
    Brain->>Gemini: Nudity or age estimate fallback
  end
  Brain->>Brain: Combine moderation and validation errors
  Brain-->>Sirloin: Score and rules
  Sirloin-->>Brisket: Pass or fail

sequenceDiagram
  participant Brain
  participant Provider as Generation Provider
  participant Hive
  participant Gemini

  opt Source image exists
    Brain->>Hive: PRE_GENERATION visual moderation
    opt NSFW generation
      Brain->>Hive: Celebrity recognition
    end
    alt Provider fallback needed
      Brain->>Gemini: Nudity or age fallback
    end
  end
  Brain->>Provider: Generate media
  Provider-->>Brain: Generated image or video output
  Brain->>Hive: POST_GENERATION visual moderation across the media
  Brain->>Hive: POST_GENERATION demographics age scoring
  opt NSFW generation
    Brain->>Hive: Celebrity recognition across image or sampled video frames
  end
  opt Hive visual moderation unavailable
    alt Generated output is video
      Brain->>Brain: Fail closed with a provider-unavailable verdict
    else Generated output is image
      Brain->>Gemini: Nudity or age fallback
    end
  end
  Brain->>Brain: Flag, drop, fail, or release output

Source Links

• apps/brisket/src/features/character-page-ccv4/hooks/use-nsfw-onboarding.tsx
• apps/sirloin/internal/app/services/characters/createcharacter.go
• apps/brain/src/modules/domain/moderation/services/image-moderation.service.ts
• apps/brain/src/modules/domain/character/services/kitsune-onboarding.service.ts
• apps/brain/src/modules/domain/moderation/celebrity-approval.ts
• apps/brain/src/modules/application/sirloin/adapters/sirloin-character-read.adapter.ts
• apps/sirloin/internal/app/services/characterruntimecontrol.go
• apps/sirloin/internal/pkg/storage/approvedcelebrities.go
• apps/brain/src/modules/domain/moderation/services/moderation-age-policy.service.ts
• apps/brain/src/modules/domain/moderation/repositories/image-moderation-result.repository.ts
• apps/brain/src/modules/application/hive/
• apps/fennec/src/pages/ModerationReview/ModerationDetailModal.tsx
• apps/fennec/src/pages/ModerationReviewPage.tsx

State Transitions

Brain persists moderation evidence in image_moderation_result and related tag rows. Onboarding also persists an onboarding image result, starting in PROCESSING and ending with pass or fail details that Sirloin maps to character image failure reasons.

Generation checks run inside Brain media processing and the workflow moderation boundary. Both paths enforce the persisted verdict for image and video outputs. Unsafe outputs are removed, safe outputs continue, and a generation fails when no safe output remains. Workflow execution raises a non-recoverable moderation error for an unsafe final output.

Invariants

• Demographics age scoring runs for ONBOARDING, MATERIALIZE_ASSET and POST_GENERATION, and at PRE_GENERATION for user-uploaded input video. Input images at PRE_GENERATION are not age-checked. Input video uses the postGeneration thresholds and the same per-person tracking rule as below, so an uploaded video showing an underage person blocks the generation before it runs.
• A call_model video target receives a user-upload age check when its recognized storage key matches one of the execution’s frozen canonical upload identities, including a versioned suffix, or its resolved storage key is under media-inputs/. Sirloin supplies both user_upload_input_keys (input names and _path/_url siblings) and user_upload_storage_keys (bare canonical storage identities), overriding same-named form values. It collects authoritative uploaded source-media paths (display_scope internal_input) before signing them, then adds normalized raw upload inputs under media-inputs/, recursively through arrays and objects. A signed URL’s host, bucket prefix or later expiry therefore cannot remove the source row’s canonical identity.
• Brain freezes Sirloin’s bare canonical identities directly, filtering invalid values, URLs and staff input-files/ keys. It also resolves declared input values, recursively through arrays and objects, retaining only canonical keys under media-inputs/. That snapshot passes separately from child input maps into both new and resumed child executions. Renaming inputs, omitting or altering child declarations, or providing a form value shaped like runtime provenance cannot replace it. The independent media-inputs/ check still applies when declarations are absent or cleared, including executions started outside Sirloin (Fennec, Flank or Foxy360).
• Brain recognizes bare keys, URLs under its configured public bucket, and presigned URLs under its configured storage API endpoint and bucket. Standard AWS regional and dualstack URLs are recognized within the configured AWS partition and exact bucket; custom, R2, FIPS and PrivateLink endpoints retain exact configured-host checks. Recognized targets match canonical upload identities by exact key or a /-delimited versioned suffix, after URL decoding. The hostname, bucket or arbitrary path prefix does not become part of the frozen identity.
• An HTTP URL on an unrecognized host remains external: SafeUrlService still checks it, and matching a declared upload never grants storage-key resolution or other storage privileges. Its decoded URL path can conservatively trigger an age check when it ends with a frozen canonical key at a path-segment boundary, whatever segments precede it. An input-files segment in that prefix does not exclude the match, because a path-style bucket can itself be named input-files; a staff URL that nests a frozen upload key therefore gets an extra age check rather than none. Malformed percent encoding and empty, . or .. path segments do not match; unrecognized-host paths cannot append a version suffix after the frozen key. This fallback adds an age check only; it does not establish ownership.
• Staff references in Brain’s input-files/ library are never user uploads, even if a declaration names them, so their video age check is skipped at PRE_GENERATION. Gallery media under characters/, reference cuts under media/ and provider URLs are also skipped unless they match a frozen canonical upload identity. A derived upload is checked at PRE_GENERATION when its new key remains under media-inputs/ or matches a frozen identity; other derived media is age-checked at POST_GENERATION. Final generation outputs remain age-checked at POST_GENERATION.
• Hive demographics returns one output per sampled frame (typically one for an image), each with one record per detected face. Brain combines faces from every output; a missing or malformed output, or a face it cannot parse, makes the verdict incomplete while keeping the faces it did parse. Brain only uses faces that have both an age_regression class and a face detection score.
• A face is ignored for underage decisions when its face score is below the checkpoint face score threshold.
• A non-ignored face is underage when either age regression is below the checkpoint regression threshold or combined baby, toddler, pre-teen, and teenager confidence is greater than or equal to the checkpoint underage confidence threshold.
• An image or video is underage when any tracked person in it is underage. Hive returns no identity per face, so the adapter links boxes across sampled frames, closest pairs first, with one face per track per frame. Links must satisfy maxCentreJump, maxSizeRatio and maxMissedFrames; a confident person face must also satisfy maxAgeJump relative to the track’s last confident person face. Faces below the detection threshold and non-person faces neither trigger nor reset the age-jump check. A face with no match or no usable box starts its own track. See Demographics Configuration for defaults and bounds. Each track is judged on its confident person faces:
  • At least minFacesForMedian such faces: underage when the median age is below ageRegressionThreshold, the median underage confidence is at or above underageConfidenceThreshold, or at least minUnderageRun of those faces in a row are underage.
  • Fewer than minFacesForMedian such faces: underage when any of them is underage. A person seen only briefly or split across tracks is judged strictly.
  • None: not underage.
• A face with other_gender ≥ nonPerson.minOtherGender and combined baby + toddler < nonPerson.maxBabyToddler is recorded with otherGender and isNonPerson: true. It is still linked but excluded from track verdicts, like a face below the detection threshold. This applies to images too; cached image checks retain the flag only under matching configuration. Known gap: a real child with an ambiguous gender and low baby/toddler scores can be excluded by this heuristic.
• An image is a single frame, so every face is its own short track: any non-ignored underage face blocks.
• Onboarding demographics underage and older age-regression validation both produce the same UNDERAGE onboarding error.
• Bound onboarding and materialized character assets run celebrity recognition whenever the global celebrity check is enabled. Unbound rejection evidence and targets explicitly listed in ctx.skipModeration bypass this call.
• A celebrity match clears only when it is the character owner’s approved celebrity. See Approved Celebrity for the match rule.
• A KYC selfie that face-matches an onboarding image clears celebrity_detected only when all of these hold: exactly one distinct celebrity (name and category) at or above celebrityThreshold is left after removing the owner’s approved celebrity; Hive demographics returned verdictStatus: 'complete' with polygonCount === 1; and the demographics verdict is not the Gemini fallback (source !== 'gemini'). While demographics is disabled the verdict is never complete, so the selfie never clears a celebrity. The clear applies to that one response only; the stored row keeps hasCelebrity: true, so it does not carry to later requests.
• Onboarding’s own celebrity gate blocks the image as CELEBRITY_DETECTED when the celebrity verdict is incomplete or the moderation result is missing entirely, not only when a celebrity is definitively detected — a persisted row can fail closed with hasFlaggedContent: true while onboarding’s separate pass/fail decision still needs this same treatment, since it reads its own validation errors, not the row’s hasFlaggedContent. KYC suppression above only clears a definitively detected celebrity; it cannot clear an unresolved verdict, since there is no known identity yet to compare against.
• NSFW onboarding does not fail just because nudity is present, but can fail if expected nudity is missing.
• Non-onboarding NSFW checks ignore nudity tags so expected NSFW output is not blocked by its own content type.
• Generation celebrity checks run for every NSFW request at both PRE_GENERATION and POST_GENERATION, not POST_GENERATION alone. The approved celebrity applies the same way as at onboarding. SFW generation does not run celebrity recognition.
• NSFW-generated video uses the same celebrity policy for text-to-video and reference/image-to-video. Hive returns timestamped outputs; Brain scans every output, keeps the highest-scoring match for each name and category pair, and stores every threshold-crossing frame timestamp and score.
• A failed approved-celebrity lookup fails closed: Brain applies no approval, so every celebrity at or above threshold stays flagged.
• A checkpoint that requires demographics but never obtains a usable Hive or Gemini verdict fails closed: the result is flagged the same as a positive underage finding, for both image and video outputs.
• A checkpoint that requires celebrity recognition but never obtains a usable Hive verdict fails closed the same way, for both image and video outputs. There is no Gemini celebrity fallback.

Demographics Configuration

Demographics settings live under hive-moderation-config.demographic:

demographic.enabled
demographic.checkpoints.onboarding.ageRegressionThreshold
demographic.checkpoints.onboarding.faceScoreThreshold
demographic.checkpoints.onboarding.underageConfidenceThreshold
demographic.checkpoints.postGeneration.ageRegressionThreshold
demographic.checkpoints.postGeneration.faceScoreThreshold
demographic.checkpoints.postGeneration.underageConfidenceThreshold

The optional tracking and nonPerson objects accept partial overrides; omitted keys use these defaults. Their schema is in apps/brain/src/modules/application/applicationSettings/application-settings.schemas.ts, shared bounds are in apps/brain/src/modules/application/hive/hive-demographic-config.ts, and the effective defaults and decision rules are in apps/brain/src/modules/application/hive/adapters/hive-demographics.adapter.ts. No migration seeds these keys: the defaults live in code, and an operator who wants to tune a value adds demographic.tracking or demographic.nonPerson with only the keys to override in Fennec’s application settings editor (hive-moderation-config), in each environment.

Key under demographic.tracking	Default	Allowed range (inclusive)	Meaning
maxCentreJump	1.5	0.5–3	Maximum box centre distance for linking, divided by the larger face size. Face size is the larger of box width and height.
maxSizeRatio	2	1.2–4	Maximum ratio of the larger face size to the smaller face size for linking.
maxMissedFrames	2	Integer 0–5	Maximum sampled frames a track may miss before a returning face starts a new track.
maxAgeJump	20	5–40	Maximum age difference in years from the track’s last confident person face for linking.
minFacesForMedian	3	Integer 2–10	Minimum confident person faces needed to use the median and underage-run rule; shorter tracks use any underage face.
minUnderageRun	3	Integer 1–10	Minimum consecutive underage faces among confident person faces that blocks a track judged by the median rule.

Key under demographic.nonPerson	Default	Allowed range (inclusive)	Meaning
minOtherGender	0.99	0.9–1	Minimum other_gender confidence for a non-person face.
maxBabyToddler	0.5	0–0.5	Exclusive upper bound on combined baby and toddler confidence for a non-person face. Both non-person conditions must hold.

The defaults come from replaying 25 videos through the adapter (TRUCOM-103): 23 videos falsely blocked at PRE_GENERATION on 16 October and two TRUCOM-72 test videos. Before tracking, all 23 false flags blocked; with these defaults, 3 still block.

Default	Why
Median rule, minFacesForMedian 3	Each false flag had one person whose per-frame age swung several years (for example 24, 17, 18, 15, 22) around an adult median. One misread frame no longer blocks them.
maxCentreJump 1.5	One person’s face moved 1.10–1.64 face sizes between one-second samples; 1.0 split real people. The 1.64 jump still splits, and both resulting tracks block on their own.
maxAgeJump 20	Box linking alone joined an 84-year-old man and the 23-year-old woman who replaced him in the same spot, hiding her in his adult median. One person’s age moved at most 17.5 years between samples, apart from a single 58-year misread, which now splits into its own adult track.
minUnderageRun 3	A run of underage faces cannot be outvoted by a longer adult stretch. In the replay, only videos the median already blocked had such a run.
nonPerson 0.99 / 0.5	A pumpkin was read as an 8-year-old face (detection 0.80, other_gender 1.0). Hive defines other_gender as a non-human face or one whose gender cannot be told; the baby/toddler guard keeps gender-ambiguous infants in the verdict. How other_gender behaves on children is not confirmed with Hive.

The bounds keep an override from switching a protection off: for example, minOtherGender 0 would exclude ordinary children, and an unbounded maxCentreJump would merge an adult and a child in different parts of the frame into one adult median.

The application-setting schema rejects unknown keys in demographic, tracking and nonPerson. The Hive adapter also validates directly edited rows, rejecting unknown keys, non-object groups, and overrides that are not finite numbers within the shared bounds (or integers where required). Only omitted keys use defaults.

Invalid configuration fails closed

Runtime validation logs hive_demographics_invalid_config with the offending key and throws HiveDemographicConfigurationError. The moderation check returns no usable result and writes no verdict; it cannot reuse a cache entry or bypass the error through Gemini, even if Hive visual moderation also fails.

There is no separate preGeneration demographics config: uploaded video checked at PRE_GENERATION uses postGeneration thresholds. MATERIALIZE_ASSET uses onboarding thresholds.

Approved Celebrity

Sirloin stores at most one approved celebrity per user in users.approved_celebrities: a name and an optional category. It applies to every character that user owns, including characters created later. Strip admins set or remove it with StripSetUserApprovedCelebrity (proto/sirloin/v5/strip.proto). Brain keeps no copy.

Brain reads the value through CharacterRuntimeControlService.GetCharacterApprovedCelebrity (proto/brain/v1/character_runtime.proto), called by SirloinCharacterReadAdapter.getApprovedCelebrity with a 3 second deadline. Brain calls it only when a complete Hive celebrity verdict has only well-formed matches and at least one at or above celebrityThreshold, so moderation without a celebrity costs no Sirloin call. A verdict with any malformed match gets no lookup and no approval. Sirloin resolves the character’s owner and returns that user’s approval, or found: false when the character or the approval does not exist.

The match rule lives in apps/brain/src/modules/domain/moderation/celebrity-approval.ts:

• Names are compared after Unicode NFC normalisation, trimming, collapsing whitespace and lowercasing. The name is the key.
• Category is optional free text, stored as whatever string Hive sends. It only separates two celebrities with the same name. A match is refused only when both the approval and Hive carry a non-empty category and the normalised categories differ. An empty approved category matches any category.
• Only matches at or above celebrityThreshold count. Approved matches are recorded in metadata.allowlistedCelebrities, and hasCelebrity stays true only when an unapproved match remains.

Failure and rollout behavior:

• If the lookup fails (Sirloin unreachable, timeout, UNIMPLEMENTED before Sirloin deploys), Brain logs approved_celebrity_read_failed and applies no approval. A Sirloin outage blocks approved users; it never clears unapproved ones.
• Deploy Sirloin before Brain. If Brain deploys first, approved users are blocked until Sirloin serves the RPC.
• Sirloin resolves the owner with ResolveCharacterAccountID (apps/sirloin/internal/pkg/storage/characterprojection.go), which does not filter on deleted_at. A soft-deleted character still resolves to its owner’s approval.
• The per-character skip_celebrity_check flag is dormant: no moderation or onboarding path reads it. The Sirloin and Brain columns, the Sirloin RPC and the proto field stay for rollback and are removed in a follow-up.
• Existing approvals were seeded from Brain evidence with the approved celebrities backfill (apps/sirloin/ops/approved_celebrities_backfill/README.md, linked from the Sirloin runbook), which also covers the catch-up run after Brain deploys.

Stored Evidence

Important persisted fields include:

• has_flagged_content: final moderation pass or fail value for the checkpoint.
• tags: visual moderation tags and their thresholds.
• metadata.demographics.faces: face score, age regression, underage confidence, age class scores, and per-face underage booleans.
• metadata.demographics.faces[].classScores: every finite Hive class score returned on the face polygon, keyed by class name. Absent on Gemini fallback faces and older rows.
• metadata.demographics.*Threshold: thresholds used for the decision.
• metadata.demographics.tracking and metadata.demographics.nonPerson: effective defaults merged with any configured overrides for the Hive verdict. Absent on Gemini fallback verdicts and older rows.
• metadata.demographics.isUnderage: media-level demographics underage decision: true when any entry in tracks is underage.
• metadata.demographics.tracks: one entry per track with trackId, frameIndexes, consideredFaceCount (person faces at or above the detection threshold), medianAge and medianUnderageConfidence over those faces, longestUnderageRun (the longest run of consecutive underage faces among them), the rule applied (median, any_frame, no_confident_face) and isUnderage. Each face carries the trackId it was linked to.
• metadata.demographics.frameCount: number of Hive demographics outputs (sampled frames) combined into the verdict; typically 1 for a single-output image. Absent when demographics is disabled, when Hive returns no output, and on rows written before this field existed. Distinct from the celebrity metadata.frameCount below.
• metadata.demographics.frameTimes: Hive sample timestamps aligned with frameIndex, including frames with no detected faces. Each entry is a finite numeric time or null when unavailable. Absent on older rows and non-Hive verdicts. Fennec uses these times for playback and snapshots, falling back to face or broken-frame timestamps, then the frame index, when unavailable.
• metadata.demographics.faces[].frameIndex, frameTime and box: which Hive output (sampled frame) the face came from, that output’s time, and the face box (top, right, bottom, left, Hive’s pixel dimensions). Frame indexes and boxes drive tracking; frame times align samples with playback in Fennec. frameTime is absent when Hive sent no finite numeric time; box when the dimensions were not finite. Absent on the Gemini fallback face and on rows written before these fields existed.
• metadata.demographics.brokenFrames: one entry per sampled frame that could not be fully evaluated, with its frameIndex, frameTime when known, and reason: missing_bounding_poly (the output had no bounding_poly array) or unparseable_polygon with droppedPolygonCount (faces dropped because Hive sent no usable age or score). Any entry makes the verdict incomplete. An empty array on a Hive verdict means every frame was evaluated. Absent on unavailable verdicts and older rows.
• metadata.demographics.source: gemini when the verdict was built from the Gemini age fallback instead of Hive; absent on Hive verdicts and on rows written before this field existed.
• metadata.hasCelebrity and metadata.celebrities: the celebrity verdict after the approved celebrity is applied, and the highest-scoring match per recognized name and category. metadata.celebrities keeps approved matches.
• metadata.celebrityThreshold and metadata.celebrityKeying: the threshold used, and name_category on rows that group matches by name and category.
• metadata.allowlistedCelebrities: matches at or above threshold that the owner’s approved celebrity cleared when the row was written.
• metadata.frameCount and metadata.flaggedFrames: sampled frame count and threshold-crossing celebrity timestamps and scores for video. flaggedFrames excludes frames cleared by an approval that applied when the row was written. Removing the approval later flips hasCelebrity and the flag back on the next check, but does not restore those frames.
• metadata.hiveTaskIds: Hive task IDs when available, including celebrity for eligible video calls.
• workflow_name: the name of the workflow that triggered the check, set from ExecutionContext.workflowName for checks run by the workflow engine. Nullable — direct generation-pipeline moderation and onboarding’s face/celebrity checks run outside the workflow engine and leave it unset, as do historical rows from before this field existed. Onboarding checks run through the QA onboarding_image_score node do set it, to the running QA workflow’s name. Lets Fennec filter the moderation tab by workflow.

For video evidence, open Fennec’s Video Demographics Analysis dialog.

Fennec’s moderation review shows an effective provider rather than the stored provider: a hive row whose demographics came from the Gemini fallback shows, and filters, as gemini (the API’s effectiveProvider). The stored provider stays hive because media failure-reason classification (MediaFailureReasonClassifierService) only classifies hive rows. Fennec also shows each error on its own line, labelled with the provider that returned it (Hive: or Gemini:) in place of Brain’s stored check prefix (moderation:, celebrity:, demographics:, demographics_gemini_fallback:, gemini:); the stored error string is unchanged.

A Fal content-policy rejection (EXTERNAL_MODERATION_FAILED) writes its own row at the PRE_GENERATION checkpoint when call_model receives it. The row has provider: 'fal', hasFlaggedContent: true, no tags, Fal’s reason in error, and metadata.externalRejection (provider, requestId, endpoint). Fennec shows the row with the external_moderation badge (“Provider moderation”) and a “Provider request” link to the request in Fal’s dashboard history (the link needs both requestId and endpoint; see Brain errors), and labels the error “Provider rejection” in the popup.

Existing moderation rows are not rewritten when config changes. When demographics is required, Hive image cache reuse requires only the persisted metadata.demographics.nonPerson snapshot to deep-equal the current effective configuration; tracking is irrelevant to image reuse. Gemini-sourced image verdicts (source: gemini) need neither Hive snapshot. Video cache reuse still requires both metadata.demographics.tracking and metadata.demographics.nonPerson snapshots to deep-equal the current effective configuration. Missing required snapshots or any difference cause a cache miss and fresh moderation. Video additionally requires the persisted ageRegressionThreshold, faceScoreThreshold and underageConfidenceThreshold to equal the current checkpoint thresholds; a missing or changed threshold triggers fresh moderation. For example, an age-20 video cached as passing at an age threshold of 18 must be checked again after that threshold increases to 21. This prevents an earlier video isUnderage verdict or image face isNonPerson flag from surviving changed settings. Invalid current overrides fail closed before a cached verdict can be reused. An eligible cached NSFW video without an explicit hasCelebrity verdict is incomplete and is moderated again when encountered. SFW generation rows do not require that celebrity field.

Cache hits re-derive hasCelebrity and allowlistedCelebrities from the stored celebrities and celebrityThreshold against the current approved celebrity, so setting or removing an approval takes effect on the next check without a backfill. flaggedFrames is only filtered: a cache hit drops frames the current approval clears, but cannot restore frames an earlier approval removed when the row was written. A complete cached celebrity verdict that may hold a celebrity (hasCelebrity: true, a stored match at or above threshold, or no well-formed celebrities array) and lacks celebrityKeying: 'name_category', celebrities or celebrityThreshold predates grouping by name and category, so it is moderated again instead. Video demographics caching follows the same completeness guard as image caching, described below.

A cached demographics verdict is reused only when it carries polygonCount — the number of face polygons Hive (or the Gemini age fallback, where it is always 1) actually evaluated. verdictStatus: 'complete' alone is not enough: a row from before polygonCount existed, or one written by a future adapter regression, cannot prove its cached fields are trustworthy, and is moderated again rather than trusted. Demographics needs no separate disabled check: while demographics is disabled, no checkpoint requires it, so the cache guard asks for no demographics verdict and a cached row, including one with verdictStatus: 'disabled', can be reused. Once demographics is enabled again, a checkpoint that requires it rejects that disabled row because it is not complete, and the media is checked again. Image and video validate different fields on top of that shared polygonCount gate:

• Image additionally requires a faces array whose length equals polygonCount (proving no polygon was silently dropped during parsing) and, on every entry, a finite age, score, and underageConfidence. Once the configuration snapshots match, cached checks re-evaluate the stored face scores against the current checkpoint thresholds, retaining the stored isNonPerson flags under the matching configuration.
• Video additionally requires isUnderage to be present as a boolean and frameCount to be a positive integer. A video row without frameCount was written before demographics checked every sampled frame, so its isUnderage may cover only the first frame, and a row without a tracks array was written before per-person tracking, so its isUnderage used the any-frame rule; both are moderated again instead of reused. Once the tracking, non-person and checkpoint threshold snapshots match, video uses the persisted isUnderage instead of re-deriving it from faces. Cached video underage verdicts do not affect checkpoints where demographics is not required, including staff references at PRE_GENERATION.

A cached celebrity verdict is reused only when metadata.verdictStatus is complete with hasCelebrity present as a boolean, or disabled while celebrity recognition remains currently disabled — unlike demographics, celebrity’s required-check condition does not imply the check is enabled, so a stale disabled verdict left over from before recognition was re-enabled genuinely needs this separate check to be rejected.

When a required demographics or celebrity check fails closed with no usable result at all, Brain still persists verdictStatus: 'incomplete' for that check (metadata.demographics.verdictStatus or the top-level metadata.verdictStatus) instead of omitting the field. Fennec’s reason filter and badges expose this as demographics_incomplete and celebrity_incomplete, independent reasons that a row can carry alongside, or instead of, moderation_unavailable (which reflects only the core visual-content classification, not celebrity or demographics).

Error Paths

Except for invalid demographics configuration, if Hive visual moderation fails for an image, Brain tries Gemini as a fallback for nudity and age. Gemini returns an age estimate only and does not return Hive age class scores. If Hive visual moderation fails for a video, Brain stores a blocked provider none verdict instead of using Gemini.

If Hive visual moderation succeeds but Hive demographics fails, Brain can use Gemini age as a demographics fallback for the age-regression path. The age-class confidence path is unavailable in that fallback. If Gemini also fails to return a usable age, or demographics is required and never returns a usable verdict at all (disabled counts as a verdict, not as missing), the checkpoint fails closed rather than passing silently. A non-numeric or non-finite Gemini age (for example a malformed provider response) counts as no usable age, the same as a missing one. When Hive demographics returns a malformed frame or some polygons it cannot parse alongside faces it can, the verdict is incomplete but keeps the parsed faces; if any parsed face is underage, the Gemini fallback is not used, because one Gemini age estimate for the whole image must not clear a face Hive already found underage, and the content stays flagged. Video never attempts the Gemini demographics fallback, so a required video demographics verdict that Hive does not deliver fails closed directly.

Hive celebrity recognition has no Gemini fallback for either image or video. If it is required and Hive does not return a usable verdict, the checkpoint fails closed directly.

If all moderation providers fail, Brain stores provider none, marks the result as flagged, and blocks the content. If celebrity or demographics was independently also unavailable on that same row, Brain still records verdictStatus: 'incomplete' for it so the row remains findable by reason, even though hasFlaggedContent was already forced by the provider failure.

QA: Reproducing Provider Failures In Flank

A mock_moderation_response workflow node (apps/brain/src/modules/domain/workflow/nodes/qa/mock-moderation-response.node.ts) lets QA reproduce any of the error paths above inside a Flank workflow, without rotating a real Hive or Gemini API key. It is a QA node (Flank palette group QA) and is available in every environment, including production. QA nodes run only in a sandbox execution: any workflow containing one always executes as a sandbox run, even when it is published, and each QA node refuses to run in any other execution (apps/brain/src/modules/domain/workflow/nodes/qa/assert-sandbox-execution.ts). A published workflow containing a QA node runs only for a signed-in ADMIN: CREATOR callers and Sirloin generations are refused, character compute never runs one, and a published non-QA workflow cannot reach QA nodes through a subworkflow (apps/brain/src/modules/domain/workflow/services/workflow-execution.service.ts).

Ready-made workflows: apps/flank/qa/moderation-provider-failures.qa-bundle.json holds the 12 QA Image|Video|Onboarding Hive … workflows (Timeout and Auth Fail for each case) plus 12 QA NSFW … copies. Import it in Flank under Settings → Import, then Test run a workflow with image_path (or video_path) and character_id. Re-importing updates workflows with the same name.

Video age checks without inference: import apps/flank/qa/video-age-check-passthrough.qa-bundle.json and use Flank Test run (sandbox), never Run or Publish. Publishing is rejected while qa_skip_generation: true is present. Non-sandbox executions fail before inference or downstream media delivery, even with capture_error. The flag does not change checkpoint resume or completed-child replay behavior. The workflow runs the same video through call_model at PRE_GENERATION, then media_output at POST_GENERATION. Its call_model sets qa_skip_generation: true: input moderation keeps the existing upload-provenance, skip and Hive-availability rules, then succeeds without inference, polling or provider retries. A moderation block still fails the run; capture_error is unnecessary. provider remains required but unused, and model is omitted. Success returns labelled moderation-only markers and empty media fields; see Call model configuration. The bundle imports as NSFW and keeps user_upload_input_keys. Use an existing media-inputs/... video key to exercise both age checkpoints; staff input-files/... video keys exercise only the POST_GENERATION age check, even when declared as uploads.

Carousel partial delivery (FOXY-925): apps/flank/qa/carousel-partial-delivery.qa-bundle.json holds six copies of SD5 SFW KackMe and V4 SFW Text to Carousel: QA Carousel KackMe|Text Panel 2 Blocked, … Panels 2-3 Blocked and … Panels 2-4 Blocked. Chained mock_moderation_response nodes flag those panels ({{nodes.store_from_urls_1.keys[N]}}, Hive “Nudity flagged”) before media_output. Every run completes, and media_output returns the unflagged panels: 4, 3 and 2. Brain does not enforce the carousel minimum. Sirloin does (MinCarouselPanels), so it would deliver the 3-panel result and fail the 2-panel one with a full refund. These runs are sandboxes, so no media is saved, nothing appears in Brisket, and Sirloin’s refunds do not run. The Sirloin storage tests cover the refunds. The KackMe copies feed FAL the public URL of image_path in place of trigger_1.image_url, which Sirloin sends only for gallery sources, so a Test run needs only image_path, character_id and aspect_ratio.

Moderation judges NSFW from the workflow’s explicitness level, not from the character (apps/brain/src/modules/domain/workflow/engine/executor.service.ts), and new workflows default to SFW. An SFW run flags nudity, so testing NSFW media needs an NSFW workflow. Flank bundles carry each workflow’s explicitness level as explicitnessLevel (apps/flank/app/lib/server-fns/config.ts), so the media QA NSFW … workflows import as NSFW. An update import keeps a workflow’s existing level when the bundle omits the field. The QA NSFW Onboarding … copies use the FACE_FRONTAL_NSFW slot with character_is_nsfw: true, since onboarding takes NSFW from the image type. NSFW onboarding also requires a KYC face match, so without a matching KYC photo those runs also report FACE_NOT_MATCHING_REFERENCE.

Asserting the outcome: an intended block would otherwise fail the workflow run. The media workflows therefore end in an expect_moderation_outcome QA node (apps/brain/src/modules/domain/workflow/nodes/qa/expect-moderation-outcome.node.ts) instead of media_output. It runs the same POST_GENERATION check with the same armed mocks, but never throws on a block: the run succeeds when the verdict matches the configured expected outcome (Blocked or Passed), and optionally the expected provider (Hive, Gemini, or None) and the check expected to be incomplete. It fails with every mismatch listed otherwise. Engine capture_error deliberately cannot catch a moderation block, so a published workflow can never swallow one.

How to connect the nodes:

1. Add one mock_moderation_response node per check you want to replace, and connect it with an edge into the node whose moderation it should affect (expect_moderation_outcome, media_output, onboarding_image_score, or call_model), directly or through other mock nodes. Nodes at the same graph depth run in parallel, so a mock with no edge path to the moderating node may arm too late and the real provider is called. This is the same ordering contract as skip_moderation.
2. Set each node’s target to the same media path/URL the triggering node uses (the same string value, since the override is keyed by that target).
3. Pick a Provider (Hive or Gemini), then a Scenario. The scenario dropdown lists only the selected provider’s scenarios, and changing the provider clears a scenario it no longer offers. A Hive scenario names the check it replaces (for example “Demographics: Timeout”); every check offers a Timeout and an Auth Issue (HTTP 500) error; each Gemini scenario replaces the Gemini fallback. The catalog in apps/brain/src/modules/domain/workflow/nodes/qa/mock-moderation-scenarios.ts is the source of truth and is not duplicated here.
4. To reproduce a specific row shape (for example, the Hive-demographics-403-masked-by-Gemini-fallback case in “Error Paths” above), arm Hive “Demographics: Timeout” or “Demographics: Auth Issue” and Gemini “Adult age” on the same target — moderation and celebrity checks run for real.
5. To reproduce “all moderation providers fail” (the row shape one paragraph up), stack four nodes on the same target with error scenarios: Hive moderation, celebrity and demographics, plus the Gemini failure. Each node arms one check, and nodes on the same target merge into one override set.
6. To reproduce an ONBOARDING row, connect the mock nodes to an onboarding_image_score node (apps/brain/src/modules/domain/workflow/nodes/qa/onboarding-image-score.node.ts) whose key equals the mock target. The real onboarding endpoint (POST kitsune/image-score) runs outside any workflow and cannot see a mock, so this sandbox-only QA node runs the same KitsuneOnboardingService.analyzeOnboardingPhoto check in-process with the armed overrides. Its output is the endpoint’s response plus accepted. Like the endpoint, it creates an onboarding image record.
7. Run the workflow and inspect the resulting fennec.image_moderation_result row (via Fennec’s moderation review page, or a direct query) to confirm the reproduction.

Mocks apply to video as well as images: for a media_output with video_path, the Hive moderation, celebrity, and demographics overrides replace the video checks the same way. Video never attempts the Gemini fallback, so a Gemini scenario has no effect on video, and a mocked Hive demographics error fails a video closed directly.

An armed override makes its check required, whatever the checkpoint or NSFW gating would normally decide: a mocked Timeout or Auth Fail fails the content closed and triggers the Gemini age fallback exactly like a real outage, and a mocked success is judged like a real verdict. This also lets a mock reproduce demographics paths for input images at call_model (PRE_GENERATION), where demographics does not normally run.

A mocked run persists an ordinary fennec.image_moderation_result row, so it appears in Fennec’s moderation review page like any other result. The row carries metadata.qaMockProviders (the checks that were mocked), and checkOrUseCached never reuses a row with that marker: a later real request for the same character and image runs the providers again instead of inheriting a mocked verdict.

Proactive Hive Availability Blocking

Separately from the per-request fail-closed behavior above, Brain tracks Hive availability across real calls and proactively blocks generation before attempting a Hive-backed moderation check, rather than letting every affected request fail individually.

HiveIncidentTrackerService (apps/brain/src/modules/application/hive/services/hive-incident-tracker.service.ts) tracks failure state independently per adapter (Moderation, CelebrityRecognition, and Demographics each have their own API key and failure mode), but applies a single global blocking decision: when ANY adapter is down, isBlocked() returns true and blocks all generation flows. This intentional v1 simplification defers per-flow-relevance blocking to a future version, while keeping per-adapter state so a rotated key or recovered adapter is tracked independently.

The tracker observes every real Hive adapter call outcome and classifies failures:

• Outage (5xx, timeout, ECONNRESET/ECONNABORTED, connection-refused, or a DNS failure resolving Hive’s host) — 5 consecutive outage-classified failures on an adapter trip that adapter’s block and start an internal 5-second recovery probe against it. Until then each failing request takes the Gemini fallback like any other per-request failure. A success or a non-outage failure on that adapter resets the streak. 2 consecutive successful probes clear it automatically. A failed probe — whatever its own classification — resets that streak to zero, and only a probe-flagged success counts toward it: a coincidental success from real traffic already in flight when the block tripped does not.
• Auth (401/403) — 5 consecutive auth-classified failures on an adapter trip its block. Any other outcome in between (an outage-classified failure, a rate limit, a Hive task that returns HTTP 200 with a non-success status, a probe success) resets the streak, so “5 consecutive” cannot span unrelated failure types. This does not auto-recover: fixing a bad key requires updating Brain’s environment variable and redeploying, and the tracker’s state is re-initialized (to healthy) on process start, so the redeploy itself clears the block. Tripping auth also stops that adapter’s outage-recovery probing if one was in progress — auth now owns the block, and a stale probe cannot fire a misleading “outage recovered” alert while the adapter is still blocked on auth.
• Everything else (rate limits, validation errors, actual moderation verdicts) is untouched by this mechanism.

Four entry points check HiveIncidentTrackerService.isBlocked() and enforce the global block. The three moderation gates (2–4) act only when the request has Hive-relevant work, so an already-ctx.skipModeration-skipped target does not trip them. The provider gate (1) does not look at ctx.skipModeration: in a graph with media_output, it fires whenever a provider node executes, and only a provider node on an untaken branch avoids it:

1. Provider node (executor.service.ts) — in a graph that has a media_output node (graphHasMediaOutput in engine/graph.ts), the executor cancels the run just before a paid node — call_model, call_openrouter or subworkflow (isProviderNode) — would execute, with the block reason, so no provider is paid for an output that media_output would then cancel. Without this, a run whose inputs need no moderation (for example a text-to-image run, or one whose inputs are skipped) would pay the provider and only be cancelled at media_output. A provider node on an untaken branch never executes, so it never triggers the gate, and a graph with media_output but no provider node, such as one that only re-publishes a skipped input, still runs.
2. Workflow engine (call-model.node.ts, PRE_GENERATION checkpoint) — throws HiveUnavailableError, which the executor maps to the execution’s cancelled status (not failed). A media-probe failure while Hive is blocked surfaces as the same error.
3. Legacy media path (InputModerationCommand, pre-generation) — throws HiveUnavailableError; it falls into the generic legacy failure handling and Sirloin’s timeout backstop.
4. Media output node (media-output.node.ts, POST_GENERATION checkpoint) — throws HiveUnavailableError before moderating a final output or any carousel panel that is not skipped. It still catches a block that trips while a run is already past execution start.

Onboarding, character asset materialization, and legacy output moderation are not gated: they keep the per-request fail-closed behavior and Gemini fallback described above.

HiveUnavailableError is caught by type in the workflow executor (maps to cancelled), the legacy image/image-sequence executors, and MediaFlowsProcessor. Each of those logs a warning instead of an error. The executor records the cancel with markHiveUnavailable and closes the cancelling node’s trace with the block reason, and only a cancelled run carrying that flag fails its media and notifies Sirloin; superseded or aborted cancels keep no media side effects. A gate inside a subworkflow cancels the child the same way, and the subworkflow node rethrows HiveUnavailableError, so the parent is cancelled as a Hive cancel too. One legacy path does not: an image-to-video request that MediaProcessingService runs through ImageToVideoCommand lands in storeFailedGeneration, which logs Media generation failed for <id> at error level once per blocked request. Alerting should therefore key on the tracker’s own hive_outage_suspected / hive_auth_blocked error logs, which fire once per incident. Brain has no Sentry SDK and this repository defines no Axiom monitor for these events: an Axiom monitor that alerts on them must be configured by the Brain owner.

This is reactive only — there is no baseline scheduled polling, so there is zero added cost or Hive traffic while everything is healthy. Recovery probes are scheduled only after an outage is suspected on a given adapter (every 5s, until it clears).

QA: Testing Hive Outages In Flank

mock_moderation_response replaces a Hive call before it is made, so it never feeds the tracker; only a real Hive request that fails trips the block. apps/flank/qa/hive-outage.qa-bundle.json holds four workflows to run while Hive is made to fail for real. Import it in Flank under Settings → Import, then Test run each with image_path and image_path_2 set to storage keys of existing SFW images.

Workflow	Hive healthy	Before the block trips (5 consecutive failures per adapter)	While blocked
QA Hive Outage Image Output	Completed	Completed via the Gemini fallback, or Failed when Gemini is unreachable too; the fifth consecutive failure on an adapter trips the block	Cancelled with the block reason
QA Hive Outage Carousel Output	Completed	Same as Image Output	Cancelled; no panel reaches Hive
QA Hive Outage Skip Moderation	Completed	Completed	Completed; skipped targets bypass the gate
QA Hive Outage Call Model Gate	Failed with Unsupported inference provider (moderation passed; its provider is deliberately unknown so nothing is generated)	Same as Image Output	Cancelled before any provider call

The block reason is Hive is currently unavailable (<adapter>); retrying automatically. for an outage and Hive authentication is failing repeatedly for <adapter>; its API key needs to be reset. for an auth block. Brain logs hive_outage_suspected once per adapter when an outage trips, hive_outage_recovered when it clears, and hive_auth_blocked when auth trips.

To make Hive fail for real, run the stack locally and follow Testing Hive Outages. It uses the apps/flank/qa/hive-fault-proxy/ container to switch outages, timeouts and recovery on and off inside a running Brain.

Tests And Verification

• cd apps/brain && pnpm test
• Target apps/brain moderation service and Hive adapter specs when changing checkpoint behavior.
• cd apps/sirloin && make run-tests
• Use Fennec moderation review pages to inspect persisted moderation evidence after integration changes.
• image-moderation.service.spec.ts’s 'reproduces FOXY-963 #13230/#13257...' and 'reproduces "Hive and every fallback down"...' tests are the automated equivalent of the manual Flank reproduction above.

Related Decisions And Standards

• Character Creation
• Media Generation
• Service Architecture