Sirloin Errors
The errors operators and clients see most often, with code references and
remediation. Domain billing errors are defined in
apps/sirloin/internal/app/services/billing/domain/errors.go. Cross-cuts:
security-model for auth-related errors,
observability for log/trace lookup.
Most billing errors are wrapped by BillingError (Op, Entity, Err,
Details) — search Axiom by Op="checkout" etc. when triaging.
Domain Errors
| Error | When raised | Meaning | Remediation | Source |
|---|---|---|---|---|
subscription not found | RPCs that resolve a subscription by id | Caller passed a stale/foreign id, or DB ↔ Chargebee drift | Verify id in admin (strip); reconcile via cmd/scripts/wallet-recovery-* | billing/domain/errors.go |
invoice not found | Payment recording, dunning retry | Chargebee invoice id missing locally | Re-run event poller or call SubmitPaidInvoice | billing/domain/errors.go |
customer not found | Any subscription op | Clerk user lacks Chargebee customer | Recreate via checkout flow | billing/domain/errors.go |
coupon not found | ValidateCoupon, checkout | Coupon code invalid or removed in Chargebee | Caller should clear input | billing/domain/errors.go |
item price not found | ListProducts, checkout | Plan/addon price missing in Chargebee | Sync plans (cmd/scripts/backfill-purchase-prices) | billing/domain/errors.go |
payment not found | GetPayment | Primer/Chargebee payment missing | Wait for poller; verify transactionID | billing/domain/errors.go |
invalid subscription state | Cancel / reactivate / downgrade | State machine refuses transition | Read billing-pitfalls; fix in admin | billing/domain/errors.go |
invalid invoice state | Payment recording | Invoice already paid/voided | No action; idempotent retry safe | billing/domain/errors.go |
subscription is not active | Usage-gated RPCs (GenerateMedia) | Sub is future, cancelled, or paused | Direct user to checkout / reactivation | billing/domain/errors.go |
payment currency does not match invoice currency | Payment recording | Primer settled in non-matching currency | Manual reconcile in Chargebee | billing/domain/errors.go |
service temporarily unavailable | Any Chargebee/Primer call | Circuit breaker open | Wait, retry; check provider status | billing/domain/errors.go |
rate limited by downstream service | Chargebee/Primer 429 | Upstream throttled | Backoff; reduce poller cadence if sustained | billing/domain/errors.go |
Webhook Errors
| Status | Path | Cause | Remediation |
|---|---|---|---|
401 Unauthorized | /webhooks/chargebee/events | Basic auth header missing or wrong | Set SIRLOIN_CHARGEBEE_WEBHOOK_USERNAME / _PASSWORD; reconfigure Chargebee endpoint creds |
401 Unauthorized | /webhooks/primer/disputes, /webhooks/primer/payments | HMAC mismatch or signedAt outside 3-min skew | Re-verify SIRLOIN_PRIMER_WEBHOOK_SECRET; check NTP drift on host |
401 Unauthorized | /webhooks/ondato/verification-status | Shared-secret mismatch | Verify SIRLOIN_ONDATO_WEBHOOK_SECRET |
405 Method Not Allowed | any webhook | Caller used GET | Use POST |
413 Payload Too Large | /webhooks/primer/disputes | Body > 1 MiB | Trim payload upstream; otherwise no path |
Source: internal/app/services/billing/webhooks/chargebeewebhook.go,
internal/app/services/billing/disputes/primerwebhook.go.
gRPC Errors
Sirloin maps domain errors to gRPC status codes at the handler edge. Common mappings (TODO(@zen): exhaustive table — derive from each handler):
| gRPC code | Typical cause | Examples |
|---|---|---|
NOT_FOUND | Domain *NotFound errors | subscription/invoice/customer/coupon |
FAILED_PRECONDITION | invalid * state, subscription is not active | usage-gated RPCs |
UNAVAILABLE | Circuit breaker open (ErrServiceUnavailable) | Chargebee/Primer outage |
RESOURCE_EXHAUSTED | ErrRateLimited | upstream 429 |
UNAUTHENTICATED | Missing/invalid Clerk token | brisket / strip handlers |
PERMISSION_DENIED | Strip RPC without admin role | StripService |
Startup Errors
| Error / log | Meaning | Remediation |
|---|---|---|
MissingError("SIRLOIN_DATABASE_URL") | DB DSN unset | Set env var; sirloin refuses to start |
using default port :50051 (warn) | SIRLOIN_PORT unset | Acceptable in dev |
Chargebee webhook credentials not set (warn) | Username or password empty | Webhooks will return 401 until set |
Flank encryption key not set (warn) | (legacy) FlankStorageService disabled — superseded by brain’s workflow API | Safe to ignore on the brain-backed path; set key only if still using the retired flank storage surface |
Flank service token not set (warn) | (legacy) FlankResolveSecrets rejects all — superseded by brain | Safe to ignore on the brain-backed path; set token only if still using the retired surface |
Primer API key not set (warn) | Primer checkout disabled | Set key for prod |
Primer webhook secret not set (warn) | Primer webhooks rejected | Set secret |
Source: apps/sirloin/internal/app/config/config.go.
Upstream-Propagated Errors
Chargebee and Primer errors surface inside BillingError.Err with the
provider’s status / code. Recurring patterns:
| Provider error | What it means | Action |
|---|---|---|
Chargebee 404 | Resource not in Chargebee (returned as nil, not an error — see billing-pitfalls) | Treat as *NotFound; don’t retry |
Chargebee 429 | Rate limited | Backoff; align poller |
Chargebee 5xx | Provider outage | Circuit breaker opens; alerts fire |
Primer Authentication failed | Bad API key or wrong env (sandbox vs prod) | Verify SIRLOIN_PRIMER_API_KEY prefix |
Primer Idempotency key conflict | Reusing the wrong key | Use transactionID per billing-pitfalls |
Resolved
SubmitPaidInvoicedoes not distinguish “already credited” from “invoice not found” at the gRPC code level. Already-processed payments are treated as success (idempotent:PurchaseExistsshort-circuits and returns an emptySubmitPaidInvoiceResponse). Missing payment/invoice returnsNotFound. Other status mappings:UnavailableforerrPrimerPaymentStillProcessing,FailedPreconditionforerrPrimerPaymentFailed,Internalfor everything else (apps/sirloin/internal/app/services/billing/submitpaidinvoice.go:182-:194,:351-:368,:238-:240).
TODO(@zen)
- TODO(@zen): exhaustive gRPC code mapping per RPC.
- TODO(@zen): full enumeration of
BillingErrorOpvalues used in production.