Multihull

make demo in five beats: one spec and hull deploy; streaming completions through one URL; hull top live; the primary stopped, traffic moves, errors stay at 0; the primary restored, traffic returns. Free and local: examples/demo.
One spec
Image, GPU class, scaling and targets in one multihull.yaml. Each provider gets its native payload.
Always warm
Two providers stay hot by default. Fallbacks never scale to zero. Health beats price.
Per-request failover
A stateless Rust router retries across providers before the first byte. Callers see one URL.
Multihull in 23 seconds. The numbers come from the 0.2.0 cloud demo.
A minimal spec
Section titled “A minimal spec”Two targets, one hostname. The control plane turns this into a Kubernetes Deployment and a Modal app. In 0.2.0 these two providers deploy and fail over; RunPod, Baseten and Replicate deploy on main, unreleased.
apiVersion: multihull/v1name: llama-8b
container: image: ghcr.io/acme/vllm-llama:1.4.0 port: 8000 health: {path: /health}
resources: gpu: [L4, A10G]
targets: - provider: gke-prod type: kubernetes priority: 1 - provider: modal-main type: modal priority: 2
route: hostname: llama.api.acme.com protocol: openaiWhat hull status shows
Section titled “What hull status shows”The router owns liveness. The CLI prints what it sees. Three targets shown for the three states; 0.2.0 deploys Kubernetes and Modal.
| Target | State | Replicas | GPU |
|---|---|---|---|
| gke-prod | Ready | 2/2 | L4 |
| modal-main | Degraded | 1/1 | A10G |
| runpod-eu | Serving failover | 1/1 | L4 |
Start with the Quickstart, read how targets and failover work, or see what 0.2.0 proves.