Router overview
The router is stateless. It reads a static router.toml and a dynamic endpoint snapshot, then routes each request by priority, weight or latency with per-request failover.
Request path
Section titled “Request path”- Request target check: a target that does not start with
/(absolute-form without a path, such ashttp://host?x, or*) or that has a.or..segment (also with a;parameter, such as..;), raw or percent-encoded, gets400with error typebad_request_targetand never reaches an upstream, since the router appends the target to the endpoint URL. - Route match on host, path tree and headers. With
protocol: openai, the body is buffered (up to 1 MiB) to read themodelkey. - Score endpoints with one function:
priority_spillover(default),weighted,ewma_latency,locality, optional sticky affinity. - Attempt engine: phase timeouts (connect 2 s, first byte 30 s, idle 60 s, total 10 m), retry before the first committed byte, retry budget, hedging in v0.2. Outcomes:
Success,Capacity,Transient(including a provider edge error such as Modal’s404 modal-http:for a stopped app, matched by the signature the snapshot carries per endpoint),Fatal,ClientAbort; the table is in Health and circuits. - Auth: client
Authorizationstripped, provider header injected from the snapshot. Replicate endpoints receive onlyPOST /predictions; other requests skip them, and a route with no other endpoint answers404not_forwarded.
Snapshot sources
Section titled “Snapshot sources”| Source | When |
|---|---|
grpcs:// stream from hull controller |
Teams; carries Degraded signals back |
snapshot.json written by hull deploy |
Solo dev, CI |
| HTTPS or object-store URL polled | Multi-region routers without controller reachability |
The gRPC stream and HTTPS URLs take a CA bundle, a client certificate and a bearer token under [snapshot]; plaintext grpc:// and http:// need insecure = true. See Controller stream.
The router nacks a streamed snapshot older than the one it holds. From any source, a route that arrives with no endpoints keeps the endpoints the router holds; removing the route from the snapshot stops serving it. See Versions and empty routes.
The last snapshot is persisted to disk and served indefinitely on control-plane loss, with an alert after 1 h.
Configuration
Section titled “Configuration”listen = "0.0.0.0:8080"admin_listen = "0.0.0.0:9090"
[snapshot]source = "file:///var/lib/multihull/snapshot.json"
[probe]interval = 5timeout = 2[timeouts], [circuit], [admission], [pressure], [probe] and [retry] tune the data plane; router/router.example.toml lists every key with its default. Everything else is dynamic. SIGHUP reloads TLS; SIGTERM drains for up to 10 m.
Observability
Section titled “Observability”OTel spans router.request and upstream.attempt{provider, endpoint, n, outcome}. Prometheus: router_requests_total{route,endpoint,outcome}, router_responses_total{route,status}, router_failovers_total{from,reason}, router_circuit_state, router_probe_total{endpoint,outcome}, router_upstream_ttft_seconds, router_queue_wait_seconds, router_concurrency_limit, router_retry_budget_remaining, router_output_tokens_total. JSON access log with attempts[]. /debug/endpoints, /debug/sessions and /debug/config on the admin listener.
Live view
Section titled “Live view”hull top polls the admin listener and renders the router in the terminal:
hull top --admin http://127.0.0.1:9090 --interval 1.0Per endpoint: provider, state (serving, recovering, down), circuit, probe, requests per second, in-flight requests, p50 and p95 upstream time to first token. Per route: responses per second, errors per second (4xx and 5xx responses), the error total and failovers by reason. Rates come from counter deltas between polls, so the first frame shows dashes. When the router stops answering the view keeps the last frame, says router unreachable, retrying and keeps polling. Ctrl-C exits.
Set route.auth.apiKeys.from in the spec to env:NAME (comma-separated keys) or file:PATH (one key per line, lines starting with # are comments, a relative path resolves against the working directory of hull). Clients send Authorization: Bearer hull_<id>_<secret>.
<id>is ASCII letters and digits.<secret>is printable ASCII without spaces and may contain underscores. Generate one withecho "hull_$(openssl rand -hex 4)_$(openssl rand -hex 32)".- Hashing contract: lowercase hex
blake3("<id>_<secret>"), the key without itshull_prefix. The control plane publishes only these hashes in the snapshot’sauth.api_key_hashes; the router hashes the presented key the same way and compares it with every hash in constant time.proto/testdata/api-key-hash.jsonis a test vector both sides assert. hull deploy,destroy,snapshotandcontrollerreject a malformed key before acting. The error names the source and entry, never the key.- A missing, malformed or unknown key gets
401with error typeunauthorized. - A route with
auth.apiKeysnever runs open. If its source is unset, missing, empty or only comments,hull deploy,destroy,snapshotandcontrollerstop with an error naming the route and the source, and a running controller keeps its last snapshot and logs the error. - The snapshot marks such a route
required. On arequiredroute with no key hashes the router answers every request with401and logs a warning naming the route. Snapshots without the flag, from older control planes, keep an empty hash list open. - A route without
auth.apiKeysaccepts every request. - Per-key rate limits and provider or service allowlists are planned, not enforced yet.
Deploying
Section titled “Deploying”The Helm chart in charts/multihull runs the router as a Deployment with a Service, ConfigMap for router.toml, and an optional PodDisruptionBudget and controller. A grpc snapshot source needs TLS values or an explicit plaintext opt-in; see Controller stream.
Install from OCI
Section titled “Install from OCI”Each v* release pushes the chart to oci://ghcr.io/mishraprafful/charts/multihull. The chart version and appVersion are the release version without the v. The router image defaults to ghcr.io/mishraprafful/multihull-router:<appVersion> and the controller image to ghcr.io/mishraprafful/multihull-controller:<appVersion>; the same release publishes both. See Controller in the cluster for the controller’s values.
The chart and images are public, so no registry login or pull secret is needed:
helm install multihull oci://ghcr.io/mishraprafful/charts/multihull --version 0.2.0 \ -n multihull --create-namespacehelm show values oci://ghcr.io/mishraprafful/charts/multihull --version 0.2.0 lists every value. A private fork publishing its own packages needs helm registry login ghcr.io and a pull secret named in imagePullSecrets.
Workload namespace
Section titled “Workload namespace”hull init targets namespace: multihull. The chart renders that Namespace (workloads.namespace, workloads.createNamespace: true) with the chart labels plus multihull.dev/managed-by: multihull, so one helm install leaves the cluster ready for hull deploy; hull never creates namespaces. The Namespace carries helm.sh/resource-policy: keep, so helm uninstall leaves it and the workloads in it; remove it with kubectl delete namespace. Helm cannot create its own release namespace from a template: when workloads.namespace equals the release namespace, as in the install above, the chart skips the manifest and --create-namespace creates it. workloads.forceCreateNamespace: true renders it anyway.
Artifact Hub
Section titled “Artifact Hub”Chart.yaml carries the artifacthub.io/* annotations (license, links, images, changes) and charts/artifacthub-repo.yml holds the repository metadata. The listing appears once the owner registers oci://ghcr.io/mishraprafful/charts/multihull on Artifact Hub and the GHCR packages are public; the next release tag then pushes the metadata to the artifacthub.io tag next to the chart.