12 November 2026 · VRAMfarm team

GPU API for batch jobs: start a server at night, run the job, stop when done

A nightly job that needs 40 minutes on an RTX 4090 costs €0.39 a night if a script starts the server, runs the job and stops it. Leave the same server on around the clock so the job can run at 2 a.m., and it costs €14.16 a day. The difference is four API calls and a cron line. The scripts here do it in curl and Python, and add the part most examples skip: a spending cap that stops the server whatever goes wrong.

Before the first run: token, key, balance

A token. In Settings → API tokens, create a token for this job only, named something like “nightly-batch”. It starts with gf_live_ and is shown once, so store it right away: in your CI secrets, or in a file only your user can read, loaded as GF_TOKEN. You can have up to 10 active tokens and revoke each one separately.

A token can launch, list and stop servers. It can’t change your password or SSH key, and it can’t create new tokens, so a leaked token can cost you GPU time but can’t take over the account. The Last used column shows when each token was last used. Call the API from your own server, laptop or CI, never from a web page, where anyone could read the token.

An SSH key in Settings → Server access. It is installed on every new server, and the script uses it to run the job and fetch results.

A balance that covers at least one hour of all your running servers, including the one you launch. It’s a check, not a charge: you pay for the minutes you use.

Check all three in one call:

curl -s https://vramfarm.com/api/account -H "Authorization: Bearer $GF_TOKEN"

You want "verified": true, "ssh_key_set": true and a balance_eur that covers tonight’s job.

The four calls

The launch body takes gpu (for example rtx-4090) and optionally count, template, image, cmd, disk and ws_restore. The response comes back at once with "status": "provisioning". Preparing the machine takes 2–5 minutes with our templates and 8–15 with your own image, and none of it is billed. Billing starts when the status becomes running, by the minute, with a 15-minute minimum per launch.

A few behaviours worth knowing before you write the loop:

• Status goes provisioning → running → stopped. If a machine fails to start, we take another one for free and the id stays the same. If the server ends up stopped without you, error says why.
• Stopping twice is safe. Stopping a server that has already stopped returns {"ok": true, "already": true}, so the stop call can sit in a trap or finally without checks.
• Errors come as {"error": "...", "code": "..."}. Branch on code. 402 means not enough balance, 409 means no machine of that type is free, and 429, 502 and 503 are worth retrying after a pause.
• Limits: 60 requests a minute per token, 10 launches a minute, and up to 4 servers at a time. Poll the status every 5–10 seconds, not faster.

Every field and error is listed in the API documentation.

Our template over SSH, or your own Docker image

There are two ways to get your code onto the server.

template: "pytorch" gives you PyTorch, CUDA and Jupyter, ready in 2–5 minutes. The script copies the inputs over scp, starts the job over SSH and fetches the outputs. If the job needs a saved folder, ws_restore (on by default) brings it back, and "save": true on stop keeps it.

template: "custom" with image runs your own public image from Docker Hub or GitHub Container Registry, with all dependencies baked in. Pin a tag such as ghcr.io/you/batch:2026-11-12, not latest, so last night’s job and tonight’s run the same code. cmd is a command (up to 4,000 characters) that runs with bash in the background as soon as the container is up, with its output in /var/log/gf-cmd.log, so the image needs bash. Three differences matter for automation:

• the container keeps running after cmd finishes, so your script still has to stop it;
• idle auto-stop never stops servers from your own image, so a forgotten one runs until you stop it or the balance runs out;
• the saved folder doesn’t work with custom images yet, so results must leave the server before the stop, by scp or by the job uploading them itself.

That last option is the “submit a script and get the output back” pattern without SSH: the image runs the job, pushes the results to your bucket or a Hugging Face repo, and writes a marker file when it’s done. Moving data in and out is covered in moving datasets to a GPU server.

The nightly script in bash

This version uses the PyTorch template. It needs curl, jq and OpenSSH on the machine that runs it.

#!/usr/bin/env bash
set -euo pipefail
API=https://vramfarm.com/api
H="Authorization: Bearer $GF_TOKEN"; J="Content-Type: application/json"
MAX_EUR=3.00      # stop if this run has cost this much
MAX_MIN=120       # or if the job has run this long

ID=$(curl -sf "$API/instances" -H "$H" -H "$J" \
  -d '{"gpu":"rtx-4090","template":"pytorch","ws_restore":false}' | jq -r .id)
trap 'curl -s "$API/instances/stop" -H "$H" -H "$J" -d "{\"id\":$ID}" >/dev/null' EXIT

for i in $(seq 1 120); do                       # wait up to 20 minutes
  S=$(curl -sf "$API/instances/$ID" -H "$H")
  ST=$(jq -r .status <<<"$S")
  [ "$ST" = running ] && break
  [ "$ST" = stopped ] && { echo "did not start: $(jq -r .error <<<"$S")"; exit 1; }
  sleep 10
done
[ "$ST" = running ] || { echo "still not running, giving up"; exit 1; }

HOST=$(jq -r .access.ssh.host <<<"$S"); PORT=$(jq -r .access.ssh.port <<<"$S")
KH=$(mktemp)    # fresh known_hosts: ports are reused by other servers over time
O=(-o StrictHostKeyChecking=accept-new -o UserKnownHostsFile="$KH" -o ServerAliveInterval=30)

scp -P "$PORT" "${O[@]}" ./input.tar root@"$HOST":/workspace/
ssh -p "$PORT" "${O[@]}" root@"$HOST" 'cd /workspace && tar xf input.tar &&
  (nohup sh -c "python run.py > run.log 2>&1; echo \$? > exit_code" </dev/null >/dev/null 2>&1 &)'

START=$(date +%s)
until ssh -p "$PORT" "${O[@]}" root@"$HOST" test -f /workspace/exit_code; do
  SPENT=$(curl -sf "$API/instances/$ID" -H "$H" | jq '.cost_eur + .traffic_eur')
  MIN=$(( ($(date +%s) - START) / 60 ))
  if awk -v s="$SPENT" -v m="$MAX_EUR" 'BEGIN{exit !(s >= m)}' || [ "$MIN" -ge "$MAX_MIN" ]; then
    echo "cap reached: EUR $SPENT after $MIN min"
    scp -P "$PORT" "${O[@]}" root@"$HOST":/workspace/run.log ./ || true
    exit 2                                        # the trap stops the server
  fi
  sleep 60
done

scp -P "$PORT" "${O[@]}" root@"$HOST":/workspace/run.log root@"$HOST":/workspace/output.tar ./results/
echo "job exit code: $(ssh -p "$PORT" "${O[@]}" root@"$HOST" cat /workspace/exit_code)"

The job starts with nohup and writes exit_code when it ends, so a dropped SSH connection doesn’t kill it, and the script only polls a small file. The trap ... EXIT stops the server on every way out: success, the cap, a failed scp, or Ctrl+C.

Schedule it with cron, loading the token from a file only you can read:

0 2 * * *  . $HOME/.gf_token && /opt/batch/nightly.sh >> /opt/batch/nightly.log 2>&1

In GitHub Actions, on: schedule: - cron: "0 2 * * *" does the same, with the token in secrets.GF_TOKEN and the private SSH key in another secret.

The same job in Python, with your own image

Here the job lives in your Docker image, and cmd starts it. The script never copies code: it waits for the marker file, fetches the result and stops.

import os, subprocess, tempfile, time, requests

API = "https://vramfarm.com/api"
s = requests.Session()
s.headers["Authorization"] = f"Bearer {os.environ['GF_TOKEN']}"
MAX_EUR, MAX_MIN = 3.0, 120

def call(method, path, **kw):
    for _ in range(5):
        r = s.request(method, API + path, timeout=30, **kw)
        if r.status_code not in (429, 502, 503):
            break
        time.sleep(int(r.headers.get("Retry-After", 30)))
    if not r.ok:
        raise SystemExit(f"{r.status_code} {r.json().get('code')}: {r.json().get('error')}")
    return r.json()

sid = call("POST", "/instances", json={
    "gpu": "rtx-4090", "template": "custom",
    "image": "ghcr.io/you/batch:2026-11-12",
    "cmd": "mkdir -p /workspace && cd /workspace && python /app/run.py > run.log 2>&1; echo $? > exit_code",
})["id"]
try:
    while (srv := call("GET", f"/instances/{sid}"))["status"] != "running":
        if srv["status"] == "stopped":
            raise SystemExit(f"did not start: {srv['error']}")
        time.sleep(10)
    ssh = srv["access"]["ssh"]
    opts = ["-o", "StrictHostKeyChecking=accept-new",
            "-o", f"UserKnownHostsFile={tempfile.mkstemp()[1]}"]
    target = f"root@{ssh['host']}"
    t0 = time.time()
    while subprocess.run(["ssh", "-p", str(ssh["port"]), *opts, target,
                          "test -f /workspace/exit_code"]).returncode != 0:
        srv = call("GET", f"/instances/{sid}")
        spent = srv["cost_eur"] + srv["traffic_eur"]
        if spent >= MAX_EUR or time.time() - t0 > MAX_MIN * 60:
            raise SystemExit(f"cap reached at EUR {spent:.2f}")
        time.sleep(60)
    subprocess.run(["scp", "-P", str(ssh["port"]), *opts,
                    f"{target}:/workspace/output.tar", "results/"], check=True)
finally:
    call("POST", "/instances/stop", json={"id": sid})

Two notes. The server gets SSH even with your own image, but scp needs the image to include the usual OpenSSH tools; if yours is minimal, let the job upload its results to a bucket instead. And if the job is image generation, ComfyUI has its own HTTP API for queueing prompts, shown in ComfyUI on a rented GPU.

Thinking about the spending cap

There is no budget setting per job, so the cap lives in your script. Build it in layers, so one failure doesn’t cost a night:

1. Expected cost first. Before launching, read the live price with GET /api/offers?gpu=rtx-4090&count=1 (the price field is euros per hour for the whole machine, all its cards). Expected cost is price × expected hours, plus traffic. Set MAX_EUR to about 1.5–2 times that. If tonight’s price is above what you’re willing to pay, skip the run or try another card.
2. Time and money caps in the loop. cost_eur grows with each minute of the run, and traffic_eur adds the traffic, which is counted with some delay. Check both, plus a wall-clock limit, as in the scripts above.
3. A stop on every exit. trap ... EXIT in bash, finally in Python. Stopping twice is harmless.
4. A morning sweep. A second cron job lists running servers and stops anything older than your longest job. It catches the case where the machine running your script died mid-run:

# same API, H and J variables as in the nightly script
curl -s "$API/instances?status=running" -H "$H" |
  jq -r --arg cut "$(date -u -d '-4 hours' +%FT%TZ)" '.instances[] | select(.started_at < $cut) | .id' |
  while read -r id; do curl -s "$API/instances/stop" -H "$H" -H "$J" -d "{\"id\":$id}"; done

(date -d is the GNU version; on a Mac use date -u -v-4H +%FT%TZ.)

5. The balance as the outer wall. The balance is prepaid, so it is the absolute limit on what can be spent. Top up for about a month of jobs, not a year. We email you when less than an hour of runtime is left. But don’t rely on running out: a stop at zero balance doesn’t save anything, and the disk is deleted with the results on it.

When no machine of your card is free at 2 a.m. (409), wait a few minutes and try again, or fall back to a second card with the same or more memory. Prices and per-minute billing are explained in the billing guide. For the model memory, the VRAM picker tells you which cards fit your job.