Skip to content

The spec

One multihull.yaml per service. A pydantic v2 model validates it; hull schema exports JSON Schema for editor completion.

Section Purpose
container Image or build, command, port, health probe, env, secret names
resources GPU classes (any-of, in preference order), GPU count, memory
scaling Concurrency per replica, min and max replicas
reliability Redundancy floor: minWarmProviders, overprovision, fallbackScaleToZero, spot
targets One entry per provider deployment with priority and provider-specific block
route Hostname, protocol, failover policy, auth, optional sticky routing

A normalised enum: L4, A10G, A100-40, A100-80, H100, H200, B200. Each translator maps classes to provider SKUs and reports which it can supply through gpu_inventory, which feeds hull doctor.

Never in the file. Each target reads its provider’s native location by default. Set credentials on a target to read them elsewhere, for example a second account or a kubeconfig outside KUBECONFIG:

targets:
- provider: gke-prod
type: kubernetes
priority: 1
credentials: file:~/.kube/prod.yaml
- provider: runpod-eu
type: runpod
priority: 2
credentials: env:RUNPOD_EU_KEY
  • env:NAME reads environment variable NAME. file:PATH reads a file; ~ expands to the home directory and a relative path resolves against the working directory of hull (use an absolute path for hull controller in a container).

  • The value has the provider’s native format:

    Type Native location Override value
    runpod RUNPOD_API_KEY the API key
    baseten BASETEN_API_KEY the API key
    replicate REPLICATE_API_TOKEN the API token; cog login still reads REPLICATE_CLI_AUTH_TOKEN or an existing login
    kubernetes kubeconfig (KUBECONFIG, ~/.kube/config) or in-cluster kubeconfig YAML; kubernetes.context picks the context, else its current-context
    modal MODAL_TOKEN_ID and MODAL_TOKEN_SECRET, or ~/.modal.toml .modal.toml TOML, profile tables only: the active = true profile, else a lone profile named default, never MODAL_PROFILE (see below); it needs token_id and token_secret (OAuth profiles and server_url are not read)
    docker ~/.docker/config.json for image pulls Docker config.json (auths, credsStore or credHelpers); a registry it has no entry for is pulled anonymously, and with docker.pull: false a missing image is pulled with it too
  • An override replaces the native location; a missing or unreadable override fails, it never falls back. hull doctor names the variable or path that is missing.

  • State keeps the reference (never the value), so a target removed from the spec can still be torn down: hull destroy --target <name> uses it and shows it in the confirmation. A plain hull destroy uses no stored reference: it stops before destroying anything, names each such target and prints the --target command to run first. Add your own --state and --snapshot-out to it; the hint asks for them instead of printing them, since a state URL can carry a password.

  • The value is never logged, printed, stored in state or written to a plan; errors name only the variable or path. The one exception is RunPod, Baseten and Replicate traffic: the router signs such a target’s requests and probes with that target’s key, so it reaches the snapshot as Authorization. RUNPOD_ENDPOINT_API_KEY and BASETEN_INFERENCE_API_KEY are process-wide and apply only to targets without credentials, since another account’s endpoint would reject them; hull doctor says so (see Security).

  • Modal proxy auth tokens (MODAL_PROXY_TOKEN_ID, MODAL_PROXY_TOKEN_SECRET) are traffic credentials, not deploy credentials, and are not overridden.

  • A Modal override ignores MODAL_PROFILE, which Modal itself reads before the active profile. It is process-wide while each override belongs to one target, often another account, so applying it to every override would fail targets whose file lacks that profile or quietly switch accounts. Mark the profile to use with active = true in the override file. hull doctor names the profile each override uses, and the error for an override with no active profile says when MODAL_PROFILE was set but not used. Targets without credentials keep Modal’s own rules, MODAL_PROFILE first.

  • With a Modal override, the modal app stop that hull destroy runs sees only the override: the native ~/.modal.toml, MODAL_PROFILE, MODAL_TOKEN_* and MODAL_OAUTH_* are hidden from it. Deploy, status, scale, logs and hull doctor run the Modal SDK inside hull, and it still reads the native config (MODAL_CONFIG_PATH, default ~/.modal.toml) even with an override: a malformed native file, one with several active profiles, or one with several profiles and none active while MODAL_PROFILE is unset or default fails every Modal target, overrides included (Modal’s own config check, run inside Client.from_credentials). Setting MODAL_PROFILE to one of the native profiles, or marking one active = true, gets override targets past that last case. A native server_url (or MODAL_SERVER_URL) also applies to override targets, read from the native profile Modal picks, so MODAL_PROFILE still selects that one. Only the tokens come from the override.

Service secrets are listed by name only and mirrored into each provider’s native secret store. hull deploy reads each value from the environment variable named after the secret in upper case, with - and . as _. Kubernetes, Modal, RunPod and Baseten pass it to the container under exactly the listed name (Docker uses that upper-case form instead; Replicate refuses secrets), so list the variable the program reads, such as HF_TOKEN. A valid environment variable name is letters, digits and underscores, not starting with a digit. Modal rejects any other name at deploy, so for a Modal target hull plan notes it and apply refuses it before calling Modal. For Kubernetes, RunPod and Baseten targets, hull validate, plan and deploy warn about it instead. Kubernetes still refuses a name that is not a valid Secret key, such as one with a space, and a container.env name its API server rejects (see Kubernetes). Docker refuses a container.env or docker.env name that is empty before any = or contains =, which the daemon would reject or split.

route.auth.apiKeys.from names where keys live: env:NAME holds a comma-separated list, file:PATH one key per line (# starts a comment line; a relative path resolves against the working directory). Each key is hull_<id>_<secret>: the id is ASCII letters and digits, the secret printable ASCII without spaces. hull rejects any other shape, and a source that yields no keys, before it deploys or publishes a snapshot, and only blake3("<id>_<secret>") reaches the router. See Router overview.

See examples/llama-8b/multihull.yaml in the repository or the architecture plan.