Round API

Round API

Round exposes a single gRPC service, round.v1.RoundService, plus a small HTTP surface for liveness on the same listener. There is no public REST surface — only sirloin and brain talk to round, and they both use gRPC.

The full proto definition lives at proto/round/v1/round.proto, with generated Go bindings at apps/round/internal/pkg/pb/round/v1/. The generated reference is also rendered under Generated References → Round v1.

Listener

Round serves gRPC and the HTTP health endpoint on the same port using h2c multiplexing (see apps/round/internal/app/server/server.go).

Concern	Value
Bind	HOST:GRPC_PORT (defaults 0.0.0.0:8080)
Protocol	gRPC over HTTP/2 (h2c, plain text) — TLS terminated at the platform edge
HTTP path	/health (HTTP/1.1 OK / NOT OK string body, see services.HealthService.HTTPHandler)
gRPC reflection	enabled (reflection.Register) — used by grpcui / grpcurl
gRPC health	grpc.health.v1.Health/Check and Watch registered

Resource limits applied at the gRPC server (see server.go):

• MaxConcurrentStreams = 100
• MaxRecvMsgSize = 15 MiB, MaxSendMsgSize = 15 MiB (accounts for base64 overhead on top of MAX_BINARY_SIZE)
• Connection timeout 10 s, keepalive every 2 min, keepalive timeout 20 s, min keepalive 1 min

RPCs

Infer(InferRequest) returns (InferResponse)

Runs inference for a model selected by model_id. Input is mutually exclusive: callers set either text or image_base64, never both. options is a free-form JSON string forwarded to the model.

Model selection. model_id is matched against the in-memory registry populated at startup (apps/round/cmd/app/main.go::loadModels). Today the registered IDs are:

• embeddings — text → 384-dim L2-normalized vector (BAAI/bge-small-en-v1.5)
• face-detection — image → bounding-box JSON (RetinaFace)
• face-embedding — image → 512-dim float vector (LVFace-B_Glint360K), optional, only registered if the LVFace ONNX file is present at startup

Use ListModels to discover the live set rather than hard-coding IDs.

Request shape.

message InferRequest {
  string model_id = 1;       // e.g. "embeddings", "face-detection"
  oneof input {
    string text         = 2;
    string image_base64 = 3; // JPEG or PNG, base64 standard encoding
  }
  string options = 4;        // optional, JSON, model-specific
}

For embeddings, options={"pooling":"cls"} selects the semantic-search contract 2.0.0. Requests without this option retain legacy mean pooling and report contract 1.0.0. This compatibility mode prevents mixed document and query vectors while Sirloin rolls out.

Response shape. output is a model-specific JSON string. metadata is a JSON string containing a model-specific object. Callers use it for observability and contract checks (see services/inference.go).

message InferResponse {
  string output   = 1; // embeddings: JSON array of floats; face: JSON with boxes
  string metadata = 2; // JSON, e.g. {"model_version":"2.0.0","dimensions":384,"pooling":"cls","normalization":"l2"}
}

Validation order (all INVALID_ARGUMENT — see round-errors):

1. model_id non-empty.
2. Either text or image_base64 non-empty.
3. len(text) <= MAX_TEXT_LENGTH.
4. image_base64 decodes as standard base64.
5. Decoded bytes <= MAX_BINARY_SIZE.

Then the request is dispatched through registry.Infer(ctx, model_id, input) and any failure from the model layer is wrapped as INTERNAL (inference failed: %v).

ListModels(ListModelsRequest) returns (ListModelsResponse)

Returns one ModelInfo per registered model (model_id, name, description, input_type, output_type, version). Used by callers that want to enumerate capabilities at boot. The empty request message is reserved for future pagination.

grpc.health.v1.Health

Both overall ("") and per-service ("round.v1.RoundService") statuses are managed by services.HealthService. They flip to SERVING only after loadModels completes, and to NOT_SERVING immediately on shutdown so load balancers stop routing during the drain window.

The /health HTTP endpoint mirrors the overall serving status — Railway uses it as the platform healthcheck (apps/round/railway.json → healthcheckPath: /health, timeout 120 s).

Latency expectations

Treat these as soft targets, validated locally with make bench-embeddings and via the round_* p95 traces in axiom. TODO(@law): codify formal SLOs — none are checked into the repo today (no SLO file under docs/src/content/docs/services/ for round, no alert thresholds in apps/round/railway.json beyond healthcheckTimeout: 120).

RPC / model	p50	p95	Notes
Infer / embeddings (≤512 tokens)	~15 ms	~40 ms	CPU, single ONNX session, batch=1
Infer / face-detection (≤1 MiB JPEG)	~50 ms	~150 ms	CPU, RetinaFace mv1_0.25
Infer / face-embedding (after detection crop)	~30 ms	~80 ms	CPU, optional model
ListModels	<1 ms	<5 ms	in-memory registry
Health.Check	<1 ms	<5 ms	atomic status read

Round runs CPU-only. There is no GPU code path today (no CUDA execution provider in internal/pkg/onnxrt).

Sequence

sequenceDiagram
  participant C as Caller (sirloin or brain)
  participant R as Round gRPC
  participant V as Validator
  participant Reg as Registry
  participant M as ONNX model
  C->>R: Infer(model_id, text|image_base64, options)
  R->>V: validate shape and limits
  V-->>R: ok or InvalidArgument
  R->>Reg: lookup model by id
  Reg-->>R: ok or NotFound (Internal today)
  R->>M: model.Infer(ctx, input)
  M-->>R: output, metadata
  R-->>C: InferResponse

Caller wiring

• sirloin consumes round via apps/sirloin/internal/pkg/pb/round/. Hosts come from SIRLOIN_ROUND_HOST (default round:8080).
• brain consumes round via its generated client. Host comes from BRAIN_ROUND_HOST (default round:8080). See services/brain for the brain side.
• Both clients use plaintext gRPC inside the Railway private network.

See services/round-env for full host and port details, and services/round-errors for the gRPC status codes callers need to handle.