Character recipe analytics

This contract implements the instrumentation for FOXY-615. Sirloin owns committed actions and outcomes; Brisket owns actual screen exposure navigation clicks, and failures occurring before the server receives a file. Both use the existing PostHog integration and the same authenticated user identity.

Events

Event	Trigger	Measurement
Character Flow Started	A new Sirloin recipe session commits; never a read or replay	Creation and management starts
Character Stage Viewed	Brisket presents a stage Sirloin has confirmed; a predicted stage shown before its save returns counts only once Sirloin confirms it, and never when it is reverted; repeated renders and polling do not count	Exposure, navigation, and stage dropoff
Character Navigation Prediction Reverted	Brisket showed a predicted next stage (a plain navigation save, or the success screen of a free completion run) and Sirloin then settled the session elsewhere	How often the client-side transition mirror disagrees with Sirloin
Character Question Viewed	A paginated question is presented after prefill loading and question transitions	Question exposure and inferred dropoff
Character Question Continue Attempted	A user selects an auto-advance choice, presses primary Continue, or retries submission from a question	Intent to progress, not accepted answers or committed saves
Character Questionnaire Retry Attempted	A user presses enabled Try again on the generation-failure summary	Deliberate generation retry using saved answers
Character Questionnaire Prefill Applied	A nonempty saved-answer patch applies successfully	Automatic progress through questions
Character Navigation Clicked	An enabled shared stage Back or Close control is clicked	Deliberate navigation, including local form Back clicks
Character Action Result	A meaningful action settles, or asynchronous work is accepted; includes direct management mutations	Saves, selections, blockers, asynchronous outcomes
Character Upload Attempted	Brisket handles a deliberately submitted file, before local validation	All file attempts, including local rejection
Character Upload Result	A local rejection, preparation/transfer failure, or server verification/validation outcome	Per-image acceptance, rejection, failure reasons
Character Flow Finished	Sirloin commits completion or explicit discard	Successful runs and deliberate cancellation

Context includes schema_version=1, experience=cv2, producer, character_id, session_id where applicable, recipe_id, recipe_version, flow_key, flow_purpose, character_type, and relevant stage/action/attempt identifiers. environment comes from the existing SDK wrappers. A management mutation without a recipe session must not manufacture a session or flow start/finish.

Character Navigation Prediction Reverted carries predicted_stage_id and actual_stage_id plus the displayed stage’s context. A failed save that refreshes the session to a newer revision (for example a conflict from another tab) also counts, because the prediction was not confirmed. For a completion run the event waits until the run settles: a failed run that returns to the previous stage counts, a completed one does not. It is diagnostic: do not combine it with navigation or stage-view counts.

Character Navigation Clicked includes navigation_action=back|close and the displayed stage’s context. It is emitted before the navigation handler runs, so it measures intent, not successful navigation, draft persistence, or discard. Local form-page Back clicks count even when no server action occurs. Hidden or disabled controls, renders, and polling do not emit it. Recipe action results remain separate; do not combine their counts with navigation clicks. Button labels and form values are not included.

Character Question Viewed and Character Question Continue Attempted carry field_key, one-based question_index, question_count for the current visible branch, and question_visit_id. Rerenders, polling and busy changes do not create visits; Back/revisits do. Paginated Back/Close events also include the current field key, position and count. The recipe runner remounts tracking for a different session or stage.

Continuation attempts add method=selection|continue|retry. They occur before validation and before the selection animation’s 500 ms delay: an invalid answer or a selection canceled by leaving is still an attempt. Typing, disabled clicks, secondary actions and automatic prefill submission do not count. Deliberate retries count separately, even within one question visit. Use existing blocked action events and server results to distinguish intent from successful submission; do not label these events “questions answered successfully.”

The generation-failure summary emits Character Questionnaire Retry Attempted with action_id before resubmitting saved answers. It has stage/session context, but no question view or visit: the summary displays all answers together. Busy or disabled retries do not count; server action results describe the retry outcome.

Prefill events contain applied field_keys, never their values. Automatically skipped questions receive no fabricated view or human attempt. Failed or empty patches emit no applied event. If a question is subsequently displayed, including after a failed automatic submission, it receives a normal view. These browser events collect no raw answers, labels or selected option identifiers. Use view timestamps and subsequent progress within the declared observation window for inferred dropoff; no browser-close abandonment event or active-attention timer is introduced.

Character Flow Started includes character_gender=male|female|unknown, derived from the session’s pinned recipe gender constraint. This describes the character’s configured gender; an unspecified constraint reports unknown. Events emitted before this property was introduced do not contain it.

Character Flow Started deliberately omits stage_id, stage_category and stage_variant: session creation commits before eligibility normalization can skip the configured entry stage. Use Character Stage Viewed to measure the first screen actually displayed. Earlier sandbox start events may contain the pre-normalization entry stage; do not interpret those properties as screen views.

Use recipe stage IDs for detailed debugging and stage categories for comparisons. A recipe’s screen ordering is not universal: optional stages, revisits and conditional skips must not be required steps in every funnel. Compare equivalent journeys within recipe/version and flow purpose before aggregating them.

For asynchronous work, phase=accepted is a queue admission milestone; phase=settled is the product outcome. Immediate actions emit only a settled result. A successfully executed workflow can still reject an image. A retryable execution failure does not finish its enclosing flow.

Sirloin normalizes workflow rejection codes through CharacterAnalyticsReason in apps/sirloin/internal/pkg/storage/characteranalytics.go. Known RI CV2 codes such as FACE_NOT_VISIBLE, BODY_NOT_IN_FRAME, NOT_STANDING and AGE_BACKSTOP retain their distinct lowercase values in reason_code. This applies to upload validation results and settled provisioning action results. Unknown or missing rejection codes remain unknown; raw failure messages never enter analytics. Adding a workflow rejection code requires checking this allowlist, including codes authored in stored workflows rather than repository fixtures. The FOXY-1547 expansion affects newly emitted events after deployment; historical unknown reasons are unchanged.

At request boundaries, known input/eligibility/conflict errors are rejected; technical errors are failed with a bounded reason such as internal_error or service_unavailable. Server-side cancellation and ambiguous timeouts do not claim a settled action outcome. Upload-verification failures belong to the dedicated upload result, so they do not also create a generic action failure. Earlier failures, such as an invalid asset ID or failed asset lookup, remain action results.

Upload prepare/finalize plumbing does not produce duplicate generic action successes. Deliberate file attempts have distinct upload_attempt_id values; automatic request replay retains the original identifier. If preparation or the storage PUT rejects, Brisket emits one correlated outcome=failed result with phase=preparation or phase=storage_transfer. The bounded reason is request_interrupted for an abort/timeout, otherwise preparation_failed or storage_network_error; an HTTP error retains storage_http_error and its status. These paths end the browser attempt before finalization, even if storage received bytes. They do not claim server validation rejected the image. An ambiguous finalization timeout or browser disappearance leaves the attempt unresolved.

Change metadata reports field keys and image roles rather than their contents. Ordinary events exclude names, prompts, descriptions, filenames, object URLs, uploaded content, raw provider errors, and full recipe value snapshots.

The explicit exception is a settled Character Action Result with outcome failed or rejected for a VI creation workflow execution, regardless of its declared purpose. Its questionnaire_answers object contains the submitted field-bound inputs, including custom text when the recipe binds it. Keys are original questionnaire field keys; values come from that operation’s saved input snapshot, never current or recovered session values. This also covers retries, reveal-stage rerolls, and other failed creation actions with field-bound inputs. Fixed workflow inputs, unbound fields, assets, identity data, and workflow output are excluded.

Success, acceptance, RI, adult-enablement, management, client validation, and pre-execution rejection events do not receive answers. Missing or malformed inputs leave the existing failure event intact. If the JSON-encoded answer object exceeds 32 KiB, omit it entirely and set questionnaire_answers_omitted="too_large"; individual answers are never silently truncated. Use operation_id to inspect the saved execution inputs when answers are omitted. Collection remains best effort and applies only to new events after deployment.

Successful mutations include changed_field_keys and changed_asset_roles when applicable. Direct management saves use the same action event without inventing a flow.

Known limits for future recipes

These fixes are deferred while recipes avoid the following patterns. Revisit tracking before introducing either pattern:

• Transitions between flows within one session: flow_purpose currently follows the current flow key, so creation activity can become labelled management. Derive purpose from the persisted CreatedCharacter session origin in both Sirloin and Brisket; keep flow_key as the current location.
• Local photo capture followed by a later upload: the selected file survives in the browser draft, but its attempt ID does not. Uploading it later creates a second attempt and leaves the first unresolved, inflating apparent abandonment. Persist the attempt ID with the draft file and reuse it through upload and draft restoration; a deliberate replacement selection should get a new ID.

Dashboard definitions

Build one dashboard with the following tiles, filtered to production and the intended rollout cohort. Exclude staff/test traffic consistently in both cohorts. Always show raw numerator and denominator alongside a percentage.

Tile	Numerator / denominator or grouping
Creation conversion, RI and VI separately	Distinct creation sessions completed with a usable character / distinct creation sessions started
Stage dropoff	Distinct sessions viewing a stage without subsequent progress/completion within the chosen window / distinct sessions viewing that stage
Validation image-failure rate	Rejected validation attempts / resolved validation attempts (accepted or rejected only)
Users hitting a validation failure	Distinct users with at least one rejected validation attempt / distinct users with at least one resolved validation attempt
Upload health	Attempts and results by phase/outcome/reason; unresolved attempts reported separately
Initial RI body composition	Latest creation-completion body source, one record per RI character; uploaded/generated/stock/mixed/none/unknown shown separately
Upfront body-photo adoption	RI creation users attempting a body upload / RI creation users; also show the accepted-upload rate separately
Later real-body replacement	Distinct initially synthetic RI characters with a later committed management change to uploaded body / eligible initially synthetic RI characters
Management usage	Settled successful actions grouped by action kind, changed field keys and image roles; separate flow-based and direct saves
Waiting and retries	Async duration percentiles, and distinct deliberate attempts per session/action or image role

Use a fixed, displayed conversion window (start with 24 hours for creation) and only include starts old enough to have the full window when reporting dropoff. Keep a separate eventual-completion view for people who return later. Duration from flow start to completion is wall-clock time, including time away; it is not active attention. Waiting time is execution duration, not model inference time.

For replacement adoption, use a cohort with equal follow-up time (for example seven days after initial completion) and exclude characters too recent to have that window. A failed upload or an unsaved selection is not a replacement. Count users rather than characters in a separately labelled user-adoption tile; do not mix those denominators.

Body composition is based on asset provenance, not the legacy-named RootedInVerifiedUpload flag, which also admits stock assets. Derived assets must retain their actual origin. Unknown lineage stays unknown. Do not merge stock or unknown into customer-uploaded photos.

Committed dataset events carry state_version, body_source_before and body_source_after, plus body_<source>_count_before/after for uploaded, generated, stock, mixed and unknown. Counts describe bound image slots, not derivation edges. The current recipe convention identifies body bindings by generation-scope roles containing body; new recipes must preserve that role meaning. A body generated from a face-only reference counts as generated, while a derived body retaining a body-upload origin counts as uploaded. Optional enrichment reads immutable versions after commit. Each enrichment path has a two-second timeout independent of request cancellation; missing properties mean enrichment was unavailable and must not be treated as zero.

Events describe the instrumented cohort. To claim a percentage over all current datasets, take a separate initial inventory and apply subsequent changes, or query current Sirloin state. Untouched pre-existing characters cannot be inferred from newly introduced events. A latest-snapshot chart must be labelled with its cohort and observation window.

Legacy continuity

Retain existing events. Normalize at query time; do not double-emit every legacy event from CV2. Historical records do not necessarily carry per-attempt or per-session identifiers, so avoid claiming historical retry-level deduplication.

Legacy event / property	Comparable milestone	Caveat
Character Media Prescreened, character_media_prescreen_status	Client prescreen result	Success passes only the local gate, not server validation
Character Media Uploaded	File transfer completed	Not an accepted-image or usable-character success
Character Media Accepted, character_media_outcome=validation_success	Accepted server validation	Exclude later manual-review submission events
Same event, validation_failed	Rejected validation or technical validation failure	character_media_outcome_reason=validation_api_error is technical failure, not content rejection
Same event, review_eligible	Not accepted by automatic validation	Keep a separate review-eligible breakdown; include with rejection only when the declared metric measures automatic-validation failure
Same event, review_requested	Manual review requested	Not another image attempt or validation result
Character Generation Dashboard Viewed, screen_name	Actual stage exposure	Map equivalent stages explicitly; legacy screen names are not recipe stage IDs
Character Generation Submitted	Generation requested	Not equivalent to a usable CV2 character
Character Dataset Edit Saved	Committed dataset edit	Legacy changed-image count/types do not provide before/after provenance

For stage comparisons, legacy Reference Photo and Photos Upload are upload milestones; Generating is a waiting milestone; Settings Update and Settings Confirmation are distinct settings steps. Select the corresponding CV2 stage/variant for each recipe. Do not combine every form or upload stage into a single ordered funnel step and call it identical to the old flow.

The issue’s approximate 30% image-failure and 70% affected-user figures are historical reference values, not assertions verified by this change. Before comparing rollout results, record the original insight, date range, population, manual-review treatment and technical-error treatment. Publish a strict validation-rejection view and an all-failures view separately if the old insight combined them. If historical definitions cannot be recovered, label the new baseline rather than claiming numerical continuity.

Validation comparison query

The following PostHog SQL template compares one matched dataset-photo cohort: automatic-validation rejection, including review eligibility but excluding technical failures. It deliberately returns no rows until mapping is completed and its WHERE 0 guard removed. Set the exact deployed recipe/version, flow, source upload stage and asset role, then the equivalent legacy media category and is_dataset_edit value (false for creation, true for management). For example, legacy SFW body photos use kitsune_full_body; this does not establish which CV2 role has equivalent validation. Run creation and management separately.

Verify that the pinned recipe action runs comparable content validation. Exclude initial reference-photo validation and workflow-free uploads: both can otherwise enter the CV2 population, and phase=validation alone does not distinguish them. Repeat the mapping review for each new recipe version. Replace the example dates with the agreed comparison window and add the same rollout/staff exclusions to both branches. It uses supported PostHog aggregations. This is a rollout query template, not evidence of a query run against production.

WITH mapping AS (
    SELECT '' AS recipe_id, 0 AS recipe_version,
           '' AS flow_key, '' AS flow_purpose,
           '' AS stage_id, '' AS asset_role,
           '' AS legacy_media_category, false AS legacy_is_dataset_edit
    WHERE 0 -- Remove only after filling and verifying the cohort above.
), outcomes AS (
    SELECT
        'cv2' AS experience,
        distinct_id,
        properties.upload_attempt_id AS attempt_id,
        properties.outcome AS outcome,
        timestamp
    FROM events CROSS JOIN mapping AS m
    WHERE event = 'Character Upload Result'
      AND properties.recipe_id = m.recipe_id
      AND properties.recipe_version = m.recipe_version
      AND properties.flow_key = m.flow_key
      AND properties.flow_purpose = m.flow_purpose
      AND properties.stage_id = m.stage_id
      AND properties.asset_role = m.asset_role
      AND properties.environment = 'production'
      AND properties.schema_version = 1
      AND properties.phase = 'validation'
      AND properties.outcome IN ('accepted', 'rejected')
      AND properties.upload_attempt_id IS NOT NULL
      AND properties.upload_attempt_id != ''
      AND timestamp >= toDateTime('2026-09-17 00:00:00')
      AND timestamp < toDateTime('2026-09-24 00:00:00')
    UNION ALL
    SELECT
        'legacy' AS experience,
        distinct_id,
        toString(uuid) AS attempt_id,
        if(properties.character_media_outcome = 'validation_success',
           'accepted', 'rejected') AS outcome,
        timestamp
    FROM events CROSS JOIN mapping AS m
    WHERE event = 'Character Media Accepted'
      AND properties.screen_style = 'onboarding 2.0'
      AND properties.screen_name = 'Photos Upload'
      AND properties.character_media_outcome_category = m.legacy_media_category
      AND properties.is_dataset_edit IS NOT NULL
      AND properties.is_dataset_edit = m.legacy_is_dataset_edit
      AND properties.environment = 'production'
      AND properties.character_media_outcome IN
          ('validation_success', 'validation_failed', 'review_eligible')
      AND coalesce(properties.character_media_outcome_reason, '')
          != 'validation_api_error'
      AND timestamp >= toDateTime('2026-09-17 00:00:00')
      AND timestamp < toDateTime('2026-09-24 00:00:00')
), resolved AS (
    SELECT experience, distinct_id, attempt_id,
           argMax(outcome, timestamp) AS outcome
    FROM outcomes
    GROUP BY experience, distinct_id, attempt_id
)
SELECT experience,
       count() AS resolved_images,
       countIf(outcome = 'rejected') AS rejected_images,
       rejected_images / nullIf(resolved_images, 0) AS image_failure_rate,
       uniqExact(distinct_id) AS uploading_users,
       uniqExactIf(distinct_id, outcome = 'rejected') AS affected_users,
       affected_users / nullIf(uploading_users, 0) AS affected_user_rate
FROM resolved
GROUP BY experience

Keep mapping to one cohort per query; pooling roles or recipes can change the denominator even when their individual failure rates stay constant. Legacy rows missing the selected screen/category/edit properties are excluded; quantify that coverage gap before claiming historical continuity.

The legacy branch counts validation events because the historical emitter lacks attempt IDs. It cannot repair historical duplicate emissions. Add a separate reason breakdown over the same filtered rows. Measure upload-stage dropoff by session and stage; browser selection IDs and server preparation IDs differ, so joining their events on attempt ID cannot establish whether an upload is unresolved.

Collab generations

A collab is a Brain fennec."Generation" row whose input_variables.character_ids holds more than one character: the primary character first, then each one resolved from an @mention. Fennec’s Generations Gallery and Dashboard expose it as the Collab only checkbox, which sends is_collab=true and combines with every other filter.

SELECT count(*) AS collabs
FROM fennec."Generation" g
WHERE g."input_variables" #> '{character_ids,1}' IS NOT NULL
  AND g.success = true
  AND g.in_progress = false;

Break down by number of characters with jsonb_array_length(g."input_variables"->'character_ids'). Only use jsonb_array_length together with the predicate above: it throws on a non-array value.

Scope:

• Multi-mention creates on normal pages count, not only the Collab page.
• Edits, videos and carousels of a collab carry one character and do not count.
• Not counted: rows from workflow versions published before 2026-09-22, rows whose workflow version can’t be resolved, and sandbox runs.
• Fennec’s character filter matches only the primary character_id.

Index: idx_generation_multi_character_dbid (partial, "dbId" DESC) serves this predicate. If a CONCURRENTLY build is cancelled, it leaves an INVALID index that IF NOT EXISTS will not replace. Repair it with DROP INDEX CONCURRENTLY IF EXISTS "fennec"."idx_generation_multi_character_dbid";, then pnpm exec prisma migrate resolve --rolled-back 20261006120000_add_generation_multi_character_index, then redeploy.

Acceptance and rollout witness

Local tests validate the emitter contract. Hosted dashboards and production ingestion require a separate rollout witness; a local test passing does not establish that PostHog received an event.

1. Complete one RI and one VI creation. Verify one flow start and finish, actual visible stages, and usable-character completion. Revisit a stage and refresh its data to distinguish navigation from render/poll duplicates.
2. Submit a locally invalid file, a file with a definite transfer failure, a server-rejected file, and a valid file. Verify per-file correlation, phase and reason; ensure finalization timeout followed by backend success is not counted as rejection.
3. Retry a mutation automatically, then retry deliberately. Verify replay does not add another committed action/result; a deliberate upload has another attempt. Verify a rolled-back transaction publishes no committed success.
4. Complete a synchronous management save and an asynchronous recipe edit. Verify changed keys/roles, settled results and completion. Discard another management flow and verify it is not counted as a successful edit.
5. Start with a synthetic body, return later and save a real body upload. Verify before/after provenance and replacement-cohort membership. Verify stock and unknown sources remain distinguishable from customer uploads.
6. Compare equivalent legacy/CV2 milestones using the definitions above. Confirm the four RI ingestion tiles are queryable and denominators match their labels.
7. Interrupt PostHog delivery. Character actions must continue to work. Existing SDK batching is best-effort: post-commit emission/replay suppression does not provide crash-proof exactly-once delivery or historical backfill.

Rollback the instrumentation code if event volume or semantics are wrong; keep product mutations intact. Filter invalid taxonomy-version/time ranges out of insights, document the gap, and deploy corrected definitions. Do not rewrite historical data or silently relabel old successes/failures.