Skip to content

Quickstart

Seven steps take a container from your laptop to two providers behind one URL. A first run against a kind cluster on a laptop took six minutes of wall clock, cluster creation, a router build and one detour included; make demo from a fresh clone took 73 seconds.

  1. Install

    Terminal window
    uv tool install 'multihull[modal]==0.2.0'
    hull --version

    The modal extra brings the Modal SDK; without it hull doctor cannot verify Modal credentials. For the unreleased main branch instead: uv tool install 'multihull[modal] @ git+https://github.com/mishraprafful/multihull@main#subdirectory=python'.

  2. Initialise a spec

    hull init writes multihull.yaml. With a Dockerfile in the directory it uses that build; without one it writes the placeholder image ghcr.io/ORG/IMAGE:TAG, a GPU request and two targets, k8s-primary (the current kubeconfig context, namespace multihull) and modal-warm.

    Terminal window
    hull init

    Edit the file before going on: the image, each target’s kubernetes.context and, for a CPU container, resources.gpu: []. To try Kubernetes alone, delete the modal-warm target and set reliability.minWarmProviders: 1; hull validate refuses a spec whose warm targets are fewer than that number. A kind cluster reached from the host needs serviceType: NodePort, a nodePort mapped in the kind config and an explicit endpoint; see Kubernetes.

  3. Check credentials

    Terminal window
    $ hull doctor
    provider type credentials detail
    k8s-primary kubernetes OK kubeconfig context gke_acme_europe-west4_prod
    modal-warm modal OK Modal accepted ~/.modal.toml

    Credentials come from each provider’s native location (kubeconfig, ~/.modal.toml or MODAL_TOKEN_ID and MODAL_TOKEN_SECRET). Nothing goes in the file.

  4. Prepare the cluster

    The spec targets the multihull namespace. Installing the chart creates it; hull deploy stops with a one-line error when it is missing.

    Terminal window
    helm install multihull oci://ghcr.io/mishraprafful/charts/multihull --version 0.2.0 \
    -n multihull --create-namespace

    The chart also runs the router, but its default file snapshot source is only filled by the in-chart controller (controller.enabled=true, see Controller in the cluster). This page runs the router on your machine instead, in step 7.

  5. Plan

    Renders every provider’s native payload to .multihull/plan/ and diffs it against recorded state.

    Terminal window
    hull plan
  6. Deploy and watch

    The spec from hull init protects the route with keys from <NAME>_API_KEYS, where <NAME> is the service name in capitals with - as _. hull deploy stops while that variable holds no keys. Without --apply it is a dry run.

    Terminal window
    $ export LLAMA_8B_API_KEYS="hull_$(openssl rand -hex 4)_$(openssl rand -hex 32)"
    $ hull deploy --apply
    $ hull status
    provider type status ref updated
    k8s-primary kubernetes Ready context=..., deployment=llama-8b... 2026-10-10T07:25:31Z
    modal-warm modal Ready app=multihull-llama-8b-modal-warm... 2026-10-10T07:26:02Z

    All targets apply concurrently. One failing target never blocks the others. The deploy writes .multihull/snapshot.json, the endpoint list the router serves from.

  7. Serve one URL

    Get the router: cargo install multihull --version 0.2.0, a binary from the GitHub release (linux amd64, linux arm64, macOS arm64) or the image ghcr.io/mishraprafful/multihull-router:0.2.0 (linux/amd64 and linux/arm64). For an untagged main, clone the repository and run cargo build --release -p multihull in router/; the binary is router/target/release/multihull. Point it at the snapshot and send a request with the route’s hostname and one of the keys:

    Terminal window
    $ cat > router.toml <<'EOF'
    listen = "127.0.0.1:8080"
    admin_listen = "127.0.0.1:9090"
    [snapshot]
    source = "file://.multihull/snapshot.json"
    EOF
    $ multihull --config router.toml &
    $ curl http://127.0.0.1:8080/v1/chat/completions \
    -H "Host: llama-8b.example.com" \
    -H "Authorization: Bearer $LLAMA_8B_API_KEYS" \
    -H "Content-Type: application/json" \
    -d '{"model":"llama-8b","messages":[{"role":"user","content":"hello"}]}'

    hull top --admin http://127.0.0.1:9090 shows which provider answered, its circuit and probe state and the request rate; curl http://127.0.0.1:9090/debug/endpoints is the raw view. Tear down with hull destroy --yes.

  • Try the demo: make demo runs three local docker targets behind one URL, streams completions through it and lets you stop the primary with zero client errors; make demo-cloud does the same on kind and Modal.
  • Run the router for a team: see Router overview or the Helm chart in charts/multihull.
  • Prove failover: scale the primary to zero under load and watch router_failovers_total; hull failover test is planned, not built.
  • Example spec: examples/llama-8b/multihull.yaml in the repository.