RunPod
Calls RunPod’s REST API v2 (https://api.runpod.io/v2) directly. apply creates one Serverless endpoint of type LOAD_BALANCER, named multihull-<svc>-<provider>, from the image; RunPod binds a template to it and deletes both together. On main, unreleased: tested against a mocked API, not yet run against RunPod.
- provider: runpod-eu type: runpod priority: 3 runpod: {dataCenters: [EU-RO-1, EU-SE-1]}Credentials: RUNPOD_API_KEY, sent as Authorization: Bearer. hull doctor lists endpoints with it and fails on 401 or 403. RunPod also requires a key on every inference request, so the router injects the same header from the snapshot, on traffic and probes. Set RUNPOD_ENDPOINT_API_KEY to a Restricted key with access to the endpoint to keep the deploy key out of the snapshot. A target with credentials (The spec) always signs traffic with its own key and ignores RUNPOD_ENDPOINT_API_KEY.
Requirements:
- The server listens on
0.0.0.0:<container.port>. The translator setsPORTandPORT_HEALTHto that port andHEALTH_CHECK_PATHtocontainer.health.path. RunPod’s load balancer polls that path: 200 healthy, 204 initializing, anything else unhealthy. resources.gpumaps to Serverless GPU pools, which RunPod schedules as a whole: L4 toAMPERE_24(shared with A5000 and 3090), A100-80 toAMPERE_80, H100 toADA_80_PRO, H200 toHOPPER_141. A10G, A100-40 and B200 have no pool;hull plannotes them, andapplyfails when no listed class has one. An empty list runs on CPU workers (cpu5c, 4 vCPUs).- The image must be public; private registry credentials are not wired yet.
Lifecycle
Section titled “Lifecycle”applyagain with an unchanged spec sends no update unlessworkersorscalingdrifted; a changed spec is onePATCH, and container changes roll out as workers cycle. With no state,applyadopts the endpoint by name instead of creating a second one.scalepatchesworkers.minandworkers.max; a 400 or 422 is a permanent refusal.hull logsstreams the logs of each active worker from--sinceuntil now. Workers already scaled down are not listed.destroydeletes the endpoint and the secrets it mirrored. A 404 counts as gone.
Translation
Section titled “Translation”| Spec field | RunPod |
|---|---|
container.image |
endpoint image |
container.command, port |
entrypoint with cmd: [], ports: ["<port>/http"], PORT, PORT_HEALTH |
resources.gpu, gpuCount |
gpu.pools, gpu.count; cpu when empty |
scaling.replicas |
workers.min, workers.max, idleTimeout: 5 |
scaling.concurrency |
scaling: {type: REQUEST_COUNT, requestCount} |
container.health |
HEALTH_CHECK_PATH; hull deploy --wait and hull status report Ready only once a worker is running or idle and <url><path> answers 2xx, while 204 is warming; the router probes the same URL and counts 204 as neither up nor down |
container.secrets |
RunPod secrets named multihull_<svc>_<provider>_<secret>_<hash> (non-alphanumerics become _; <hash> is 12 hex digits of SHA-256 over the exact service, provider and secret names, so targets whose names sanitise alike never share a secret), referenced as {{ RUNPOD_SECRET_<name> }}; new values reach workers when they restart |
runpod.dataCenters |
dataCenterIds |
| endpoint | https://<id>.api.runpod.ai, plus Authorization: Bearer injected by the router |
No edge error signature yet: the load balancer answers {"error":"no workers available"} when no worker started in time, but RunPod documents no status code for it, so a live run has to record one first.
Ref stored in state: endpoint id, name and URL, health path, a digest of the applied body and the mirrored secret names. Fallbacks stay warm because fallbackScaleToZero defaults to false, so workers.min is never 0 unless you opt in.