Image Moderation And Age Scoring

Image Moderation And Age Scoring

Purpose

Document how image moderation is applied during character onboarding and image generation, including checkpoint behavior, demographics age scoring, stored evidence, and provider fallbacks.

Participants

• Brisket uploads reference images and displays onboarding moderation outcomes.
• Sirloin requests character reference verification and maps Brain errors into user-facing character dataset image failure reasons.
• Brain owns moderation orchestration, onboarding image results, generation checks, and persisted moderation evidence.
• Hive provides visual moderation, celebrity recognition, and demographics age scoring.
• Gemini can be used as a fallback for nudity and age estimates.
• Fennec reads stored moderation results for review and inspection.

Checkpoints

Checkpoint	Used for	Demographics age check
ONBOARDING	Character reference images scored by Brain for Sirloin	Yes
PRE_GENERATION	User or source image before edit generation	No
POST_GENERATION	Generated image output before it becomes available	Yes

Sequence

sequenceDiagram
  participant Brisket
  participant Sirloin
  participant Brain
  participant Hive
  participant Gemini

  Brisket->>Sirloin: Upload character reference image
  Sirloin->>Brain: Verify character reference image
  Brain->>Brain: Create onboarding result in PROCESSING state
  Brain->>Hive: Visual moderation
  Brain->>Hive: Celebrity recognition
  Brain->>Hive: Demographics age scoring
  alt Hive fallback needed
    Brain->>Gemini: Nudity or age estimate fallback
  end
  Brain->>Brain: Combine moderation and validation errors
  Brain-->>Sirloin: Score and rules
  Sirloin-->>Brisket: Pass or fail

sequenceDiagram
  participant Brain
  participant Provider as Generation Provider
  participant Hive
  participant Gemini

  opt Source image exists
    Brain->>Hive: PRE_GENERATION visual moderation
    alt Provider fallback needed
      Brain->>Gemini: Nudity or age fallback
    end
  end
  Brain->>Provider: Generate media
  Provider-->>Brain: Generated output
  Brain->>Hive: POST_GENERATION visual moderation
  Brain->>Hive: POST_GENERATION demographics age scoring
  opt NSFW generation
    Brain->>Hive: Celebrity recognition
  end
  alt Provider fallback needed
    Brain->>Gemini: Nudity or age fallback
  end
  Brain->>Brain: Flag, drop, fail, or release output

Source Links

• apps/brisket/src/features/character-page-ccv4/hooks/use-nsfw-onboarding.tsx
• apps/sirloin/internal/app/services/characters/createcharacter.go
• apps/brain/src/modules/domain/moderation/services/image-moderation.service.ts
• apps/brain/src/modules/domain/moderation/services/moderation-age-policy.service.ts
• apps/brain/src/modules/application/hive/
• apps/fennec/src/pages/ModerationReview/ModerationDetailModal.tsx

State Transitions

Brain persists moderation evidence in image_moderation_result and related tag rows. Onboarding also persists an onboarding image result, starting in PROCESSING and ending with pass or fail details that Sirloin maps to character image failure reasons.

Generation checks run inside Brain media processing. A flagged single output fails generation. For carousel output, Brain can drop flagged panels and fail only when no safe panels remain.

Invariants

• Demographics age scoring runs for ONBOARDING and POST_GENERATION, not PRE_GENERATION.
• Hive demographics returns one record per detected face. Brain only uses faces that have both an age_regression class and a face detection score.
• A face is ignored for underage decisions when its face score is below the checkpoint face score threshold.
• A non-ignored face is underage when either age regression is below the checkpoint regression threshold or combined baby, toddler, pre-teen, and teenager confidence is greater than or equal to the checkpoint underage confidence threshold.
• An image is underage when any non-ignored face is underage.
• Onboarding demographics underage and older age-regression validation both produce the same UNDERAGE onboarding error.
• Onboarding always runs celebrity recognition.
• NSFW onboarding does not fail just because nudity is present, but can fail if expected nudity is missing.
• Non-onboarding NSFW checks ignore nudity tags so expected NSFW output is not blocked by its own content type.
• Generation celebrity checks run for NSFW requests.

Demographics Configuration

Demographics settings live under hive-moderation-config.demographic:

demographic.enabled
demographic.checkpoints.onboarding.ageRegressionThreshold
demographic.checkpoints.onboarding.faceScoreThreshold
demographic.checkpoints.onboarding.underageConfidenceThreshold
demographic.checkpoints.postGeneration.ageRegressionThreshold
demographic.checkpoints.postGeneration.faceScoreThreshold
demographic.checkpoints.postGeneration.underageConfidenceThreshold

There is no preGeneration demographics config because pre-generation does not run demographics age scoring.

Stored Evidence

Important persisted fields include:

• has_flagged_content: final moderation pass or fail value for the checkpoint.
• tags: visual moderation tags and their thresholds.
• metadata.demographics.faces: face score, age regression, underage confidence, age class scores, and per-face underage booleans.
• metadata.demographics.*Threshold: thresholds used for the decision.
• metadata.demographics.isUnderage: image-level demographics underage decision.
• metadata.hiveTaskIds: Hive task IDs when available.

Existing moderation rows are not rewritten when config changes, but cached checks re-evaluate stored scores when the cached metadata has the needed demographics fields.

Error Paths

If Hive visual moderation fails, Brain tries Gemini as a fallback for nudity and age. Gemini returns an age estimate only and does not return Hive age class scores.

If Hive visual moderation succeeds but Hive demographics fails, Brain can use Gemini age as a demographics fallback for the age-regression path. The age-class confidence path is unavailable in that fallback.

If all moderation providers fail, Brain stores provider none, marks the result as flagged, and blocks the content.

Tests And Verification

• cd apps/brain && pnpm test
• Target apps/brain moderation service and Hive adapter specs when changing checkpoint behavior.
• cd apps/sirloin && make run-tests
• Use Fennec moderation review pages to inspect persisted moderation evidence after integration changes.

Related Decisions And Standards

• Character Creation
• Media Generation
• Service Architecture