Quickstart
Seven steps take a container from your laptop to two providers behind one URL. A first run against a kind cluster on a laptop took six minutes of wall clock, cluster creation, a router build and one detour included; make demo from a fresh clone took 73 seconds.
-
Install
Terminal window uv tool install 'multihull[modal]==0.2.0'hull --versionThe
modalextra brings the Modal SDK; without ithull doctorcannot verify Modal credentials. For the unreleasedmainbranch instead:uv tool install 'multihull[modal] @ git+https://github.com/mishraprafful/multihull@main#subdirectory=python'. -
Initialise a spec
hull initwritesmultihull.yaml. With a Dockerfile in the directory it uses that build; without one it writes the placeholder imageghcr.io/ORG/IMAGE:TAG, a GPU request and two targets,k8s-primary(the current kubeconfig context, namespacemultihull) andmodal-warm.Terminal window hull initEdit the file before going on: the image, each target’s
kubernetes.contextand, for a CPU container,resources.gpu: []. To try Kubernetes alone, delete themodal-warmtarget and setreliability.minWarmProviders: 1;hull validaterefuses a spec whose warm targets are fewer than that number. A kind cluster reached from the host needsserviceType: NodePort, anodePortmapped in the kind config and an explicitendpoint; see Kubernetes. -
Check credentials
Terminal window $ hull doctorprovider type credentials detailk8s-primary kubernetes OK kubeconfig context gke_acme_europe-west4_prodmodal-warm modal OK Modal accepted ~/.modal.tomlCredentials come from each provider’s native location (kubeconfig,
~/.modal.tomlorMODAL_TOKEN_IDandMODAL_TOKEN_SECRET). Nothing goes in the file. -
Prepare the cluster
The spec targets the
multihullnamespace. Installing the chart creates it;hull deploystops with a one-line error when it is missing.Terminal window helm install multihull oci://ghcr.io/mishraprafful/charts/multihull --version 0.2.0 \-n multihull --create-namespaceThe chart also runs the router, but its default
filesnapshot source is only filled by the in-chart controller (controller.enabled=true, see Controller in the cluster). This page runs the router on your machine instead, in step 7. -
Plan
Renders every provider’s native payload to
.multihull/plan/and diffs it against recorded state.Terminal window hull plan -
Deploy and watch
The spec from
hull initprotects the route with keys from<NAME>_API_KEYS, where<NAME>is the service name in capitals with-as_.hull deploystops while that variable holds no keys. Without--applyit is a dry run.Terminal window $ export LLAMA_8B_API_KEYS="hull_$(openssl rand -hex 4)_$(openssl rand -hex 32)"$ hull deploy --apply$ hull statusprovider type status ref updatedk8s-primary kubernetes Ready context=..., deployment=llama-8b... 2026-10-10T07:25:31Zmodal-warm modal Ready app=multihull-llama-8b-modal-warm... 2026-10-10T07:26:02ZAll targets apply concurrently. One failing target never blocks the others. The deploy writes
.multihull/snapshot.json, the endpoint list the router serves from. -
Serve one URL
Get the router:
cargo install multihull --version 0.2.0, a binary from the GitHub release (linux amd64, linux arm64, macOS arm64) or the imageghcr.io/mishraprafful/multihull-router:0.2.0(linux/amd64andlinux/arm64). For an untaggedmain, clone the repository and runcargo build --release -p multihullinrouter/; the binary isrouter/target/release/multihull. Point it at the snapshot and send a request with the route’s hostname and one of the keys:Terminal window $ cat > router.toml <<'EOF'listen = "127.0.0.1:8080"admin_listen = "127.0.0.1:9090"[snapshot]source = "file://.multihull/snapshot.json"EOF$ multihull --config router.toml &$ curl http://127.0.0.1:8080/v1/chat/completions \-H "Host: llama-8b.example.com" \-H "Authorization: Bearer $LLAMA_8B_API_KEYS" \-H "Content-Type: application/json" \-d '{"model":"llama-8b","messages":[{"role":"user","content":"hello"}]}'hull top --admin http://127.0.0.1:9090shows which provider answered, its circuit and probe state and the request rate;curl http://127.0.0.1:9090/debug/endpointsis the raw view. Tear down withhull destroy --yes.
- Try the demo:
make demoruns three local docker targets behind one URL, streams completions through it and lets you stop the primary with zero client errors;make demo-clouddoes the same on kind and Modal. - Run the router for a team: see Router overview or the Helm chart in
charts/multihull. - Prove failover: scale the primary to zero under load and watch
router_failovers_total;hull failover testis planned, not built. - Example spec:
examples/llama-8b/multihull.yamlin the repository.