H100s at $0.74 an hour: bidding blind in GPU spot auctions

TL;DR

• Our fitted bidding policy held H100 capacity at $0.74/hr against a $1.74 on-demand price (57% cheaper) with 97.9% uptime, replayed through our real bidder.
• The naive margin-defense strategy costs $2.00/hr under the same rules: 15% more than on-demand. Its simulator results looked great only because the simulator leaked information no live bidder has.
• Preemptions cluster in bursts (7.2x more variable than random) and no tier gives early warning about another, we tested. So the winning policy doesn't try to react or anticipate; it prices the risk upfront.
• Everything is fitted on the first 60% of the window and scored only on the final 40% the model has never seen, and it re-runs on the tracker's newest data every time this page loads.

How the market works

Vast.ai's interruptible tier is a continuous pay-as-bid auction, and four rules define the whole game:

• You pay your bid. Not the market price, your standing bid, per hour, as long as you're running. Every cent of headroom you add for safety is a cent you actually spend.
• Anyone can outbid you. When they do, your processes are killed on the spot. Your disk survives, and if your bid ever clears the bar again, the machine resumes in about a minute.
• On-demand renters beat everyone. No bid can defend against one, and they can't be preempted themselves. On-demand is both the price ceiling and the safety net.
• There is no order book. The one price the market shows per machine is min_bid, the amount a newcomer must bid to take it. Which is the current winner's bid, plus a hair.

That last rule is the trap, and it deserves its own section.

The moment you win, you go blind

While you're shopping, min_bid is real information: it's what the current holder is paying, plus epsilon. But the moment you take the machine, you are the top bid, so the number the market reports for your machine becomes your own bid plus epsilon. A rival quietly bidding up underneath you is completely invisible. The first time you learn about them is the moment your pod dies.

Now watch what that does to the "sensible" defensive strategy everyone writes first: if my margin over min_bid gets thin, raise my bid 10% above it. Live, your margin over your own reflection is always thin, it's negative epsilon, forever. So the rule fires every single cycle, raising your bid 10% above your own bid, again and again. Starting from a typical $0.94 opening bid, ten-percent compounding hits the $1.74 on-demand ceiling in seven ticks, under two minutes on our tick rate. After that you're just paying the ceiling price for a preemptible GPU, indefinitely.

This isn't hypothetical. We replayed our own old policy through the real control loop twice: once with the simulator's usual, secretly-leaky price feed, and once with the feed corrected to what a live bidder actually sees. Same policy, same market, same nine and a half hours:

A 2.4x cost distortion from one information leak. Every simulated result we'd celebrated was measuring a bidder that could see the future. Before asking "what's the best strategy," we had to rebuild the simulator so that every strategy sees exactly what a live bidder sees: real prices for machines you don't hold, your own reflection for the one you do, and the challenger's bid only at the moment it displaces you. The true prices still exist inside the simulator, but they're only used to decide outcomes and keep score, never shown to the strategy.

The data: our own price tracker

The dataset is the same one that powers the public tracker page: every ~30 seconds, the cheapest 1-GPU price per model and provider, spot and on-demand. The live run below uses the latest 9.5 hours of it (1,144 market rounds) with each H100 model tier (SXM, NVL, PCIE) treated as one "machine" whose price is the cheapest of its kind.

Three personalities emerge. SXM is the value tier and the knife fight: a median of $0.60 but swinging from $0.13 to $1.33, changing price 19 times an hour. PCIE is a boring $1.60 shelf. NVL steps between levels and sits there for an hour at a time. Everything left of the dotted line is what the model is allowed to learn from; everything right of it is the exam.

What the statistics actually say

Before fitting anything, we asked the data three questions that decide what kind of policy can work at all.

• Do preemptions arrive randomly? No, they arrive in gangs. If price spikes were independent events, the spike counts in half-hour windows would have a variance-to-mean ratio near 1. Measured: 7.2 on the SXM tier (p < 0.0001). One preemption is strong evidence another is seconds behind, which changes how you should behave right after winning your machine back.
• Does one tier warn you about another? This was our big hope for a legal early-warning system: maybe when PCIE and NVL heat up, SXM is about to. We test all 8 lead-lag pairs formally, on every load of this page. Not one is statistically significant once you correct for multiple testing. The pool-wide "surge detector" strategy we'd sketched has no reliable signal to detect, and in the original tournament it duly racked up 39 preemptions and got disqualified.
• Are there calm/contested regimes to exploit? Probably, but 9.5 hours can't prove it: regime-switching fits refuse to converge. So the policy doesn't get a regime knob yet.

Put together, the statistics close every clever door: you can't watch your own machine (reflection), you can't watch the neighbors (no cross-tier signal), and you can't switch modes (no identifiable regimes). What's left is pricing the risk correctly before you enter, which turns out to be enough.

Pricing the knife's edge

For every candidate bid level, the training data tells you three things directly: how often the price crosses above it (preemptions per hour), how long it stays above once it does (your outage length), and what the challenger typically pays (what a counter-raise costs you). Combine them and each bid level gets an honest price tag:

expected cost per served hour = (bid + storage + counter-premium) / uptime

, where uptime and the counter-premium both come from the measured crossing rates and outage lengths, not from assumptions. Then pick the bid that minimizes it, subject to one business constraint: modeled uptime must stay above 95%.

The curve has a story in it. The raw minimum sits at a bid of $0.21, the market spends most of its time below that, so it looks like a steal. But it concedes 18% downtime, which is why the uptime floor exists: the honest optimum on H100 SXM is currently $0.654, projecting $0.69 per served hour at 98.8% uptime with one preemption every three hours. On the original July 19 fit we bootstrap-resampled the whole thing 400 times and it landed on the same bid in over 90% of resamples, that cliff in the availability curve is real, not noise.

The same fit also answered a question we'd been hand-waving: after you fight off a challenger with a higher bid, when do you relax back down? You can't watch the market cool off, you're top bidder again, so you're blind again. The data's answer: excursions above the optimal bid last a known distribution of time, so relax on a timer set to its 90th percentile. Boldness from a stopwatch, not from a price feed.

One more thing the model handles automatically: tier selection with hard feasibility. On the July 19 window the H200 SXM tier's cheapest price walked above our $2.75 absolute bid cap and never came back, and every price-chasing heuristic followed it up to $2.34+ per hour; the model simply declared the tier unbiddable and held the NVL tier at a fitted $1.36 instead.

Fit on the past, scored only on the future

A model that grades itself on its own training data is marketing. So: every parameter above is fitted on the first 60% of the window, then frozen. Eight strategies, flat-margin bidders, quantile bidders, a pure reactive bidder, the sibling-surge detector, an adaptive optimizer, and the old margin-defense policy, plus the frozen model are all scored exclusively on the held-out remainder, under the honest visibility rules. This chart is recomputed from the newest data every time the page loads; the roster comes straight from the bidder's strategy file, so a new strategy shows up here the day we add it.

On the July 19 held-out window the fitted policy won outright at $0.76 per served hour (56% below on-demand) with the adaptive optimizer second at $0.84. The heuristics that look identical to each other in calm training windows tend to defend a stale price level into a hotter test window. And the old policy's honest number sits above the on-demand line, exactly where the self-ratchet arithmetic from earlier says it has to end up.

The final exam: through the real bidder

Backtest engines flatter policies in a hundred small ways, so the last gate is the real thing: our production bidder daemon (the actual state machine, guard rails, budget caps, and API client) driven against a mock exchange fed with the recorded market, on a virtual clock. Four configurations, both pools, full-window replay. This part can't run in a browser; the numbers below come from the most recent full-stack run (July 19, 2026) and refresh whenever we publish a new one:

Three readings. First, the honesty tax is a measured number: closing the information leak multiplies the old policy's cost by 2.4x on H100. Second, the fitted policy beats the best honest heuristic by 26% on H100 (and by 24% on H200, at $1.40/hr). Third (and this is the part we find genuinely satisfying) the fitted policy playing fair roughly matches the old policy playing with leaked information. The statistics recovered what the leak appeared to provide.

The whole policy, in four rules

1. Hold the tier with the lowest fitted cost, standing at its fitted bid. Never enter a tier whose price is above that bid, you'd rent a disk that never runs.
2. Never react to your own machine's price. It's a mirror.
3. When displaced, the challenger's bid finally becomes visible: outbid them by 2% if they're under 97% of on-demand, otherwise wait for free, the disk survives and resumes itself when the spike passes.
4. After winning a fight, relax back to the fitted bid on a timer (the 90th-percentile spike duration), not on a price signal you don't have.

Everything a first-year trader would recognize (position sizing from measured risk, no trading on reflections, time-based exits) and nothing else. This is the policy the production netra-bidder now runs by default: the same code path, keyed by tier, that scored $0.74/hr above.

Watch it run: the whole history, with real friction

Here is the fitted bidder replayed through the real production stack across the entire tracker history, from the very first sample the collector ever recorded, and this time with the physical friction a price model glosses over: real internet speed, download time, ingress cost, storage, and the two very different ways you can lose a machine.

The per-machine network here is not made up: it comes from real Vast API data, sampled live from current H100/H200 offers. So each machine's downlink (300–9,000 Mbps), its bandwidth price, and its storage rate ($0.36/GB·mo) are what real machines actually charge. Pulling the 46 GB image plus weights takes 7–20 min of cold-start downtime, and on a metered host it costs real money in ingress. That single fact (real bandwidth pricing) turned out to matter more than we expected.

And the two loss modes are not the same:

• Someone outbids you. Your processes stop but the disk stays. When the price dips back under your bid (usually within a couple of minutes on this market) the same machine resumes in place, no re-download. Cheap. We counted 13 of these; all resumed. A key tuning point falls out here: because a cold rebuild is now expensive, the bidder waits out the short outage for the free resume instead of eagerly rebuilding elsewhere.
• Someone rents the same machine on-demand. On-demand always wins and can't be preempted, so you're evicted and (this is the important part) you cannot get that machine back until they leave. There's no resume in place. This is the expensive one, and it is what the next idea is for.

Prepare the next disk before you need it

If losing a machine means re-downloading 46 GB somewhere else, the obvious defense is to have already downloaded it somewhere else. So the bidder keeps 3 pre-staged warm disks: cheap sibling machines where the image and weights are downloaded and then the instance is parked, GPU off, disk kept, billed only for storage (pennies an hour). When an on-demand renter takes our serving machine, recovery doesn't cold-rebuild; it wakes a warm disk that already has everything, and we're back in about a minute instead of fifteen. In this run, 2 on-demand takeovers were absorbed by 13 warm wakes, with 0 cold rebuilds.

The chart below is the whole run. The red bands are downtime, when and for how long the router had nothing healthy to serve; you can read the outages straight off the width of each band. Amber dots are outbids that resumed in place, red marks are on-demand takeovers, and aqua dots are recoveries that came from a pre-staged warm disk.

Fitted policy through the real bidder, honest live visibility. Hover for the exact state at any moment.

Keep the bid steady

One more thing you can see in the chart: left alone, the bid sawtooths. Every time the market brushes past the standing level, the resume race nudges the bid up to defend, then it relaxes back down, then the next brush nudges it up again, a jittery staircase that is both noisy and quietly expensive, because each defensive nudge ratchets the average bid higher. So the policy carries a small hysteresis band: it only moves the bid when the target leaves an 8% dead-band around the current bid, and never more than once every couple of minutes. In this window that cut bid changes by a third (from 201 to 135) and, more importantly, kept the effective cost from ratcheting: without the band the same run drifts to $2.38/served-hr, above on-demand; with it, the bid holds and the run comes in far cheaper.

So where does the whole thing land, with real machines, real bandwidth prices, real download times, both loss modes, warm-disk recovery, and a steady bid? On H100, $1.56/served-hr at 91% availability, 11% below the $1.75 on-demand price, saving $2.20 over the window. That's a long way below the idealized $0.74, and the reason is almost entirely $5.33 of ingress: on this window the cheapest H100 machines that were available charged for bandwidth, and re-downloading 46 GB adds up. It's the honest number, and it carries the real lesson, on real hardware, bandwidth pricing is a first-order cost, not a footnote. Filter to free-bandwidth hosts (the bidder prefers them, and Vast has plenty) and the ingress term collapses; the same policy on H200, where the machines that came up were cheaper-bandwidth, holds 43% below on-demand. The spot edge is real, it just has to survive contact with the network bill.

The math, if you want it

Everything above is driven by a handful of equations. None of them are complicated; the discipline is in feeding them only numbers a live bidder can actually see. Here they are.

1. Why margin-defense self-destructs

Let your bid be $b$ and the machine's reported min_bid be $m$. The naive rule raises to $r\cdot m$ (with raise factor $r>1$) whenever the margin $b-m$ gets thin. But once you hold the machine, the market reflects your own bid back: $m=b+\epsilon$. So the margin is always $-\epsilon$, the rule fires every tick, and the bid compounds:

It reaches any ceiling $C$ (the on-demand price) after

With the incumbent numbers ($r=1.10$, opening bid $b_0\approx \$0.94$, ceiling $\approx \$1.74$) that's $\lceil \ln (1.85)/\ln (1.10)\rceil =7$ ticks (under two minutes) and then you pay the ceiling forever. That single fact is why the policy must stand at a fixed, fitted level and never chase its own reflection.

2. How contested a bid is

From the training window we measure two things at every candidate bid level $b$. The exceedance, the fraction of time the market price sits above $b$ (how often you'd be outbid):

and the upcrossing rate $\lambda (b)$, how many times per hour the price crosses up through $b$ (how often you'd be preempted). Both are estimated directly, with no Poisson assumption, because preemptions arrive in bursts (we measured a variance-to-mean ratio of 7.2, p < 0.0001, nowhere near the value of 1 that independence would give). We smooth them to be monotone in $b$ with isotonic regression, since more headroom can only reduce contention.

3. Download time and the cost of losing a machine

A cold build re-downloads the image plus weights ($P$ gigabytes) over the machine's downlink $d$ (in Mbps). The downtime is just physics:

At $P=46$ GB and the 250 Mbps floor that's about 24 minutes; on a fast 5 Gbps machine, about 1½. A warm resume (disk kept) skips this entirely (roughly a minute) which is the entire reason resuming in place, and pre-staging a warm disk, matter so much.

4. Effective cost, and the bid that minimizes it

Put it together with the renewal reward theorem: the long-run cost per served hour is the cost of a preempt-and-recover cycle divided by the time you actually serve in it. Writing $s$ for storage, $\pi$ for the counter-premium you pay after a challenger appears, and $g$ for amortized ingress:

where availability falls with the preemption rate times the downtime each one costs:

and (this is the realism that raises the answer) a recovery is a cheap resume if the outage is short but a cold rebuild if it runs long, so $D_{\mathrm{recover}}$ blends the two by how often excursions outlast the resume window, and $g$ charges ingress on exactly those long ones. A too-low bid means a high $\lambda$, which means many cold rebuilds, which wrecks both terms. The bid we run is the one that minimizes cost subject to keeping the service level:

On H100 that lands at $0.654. The unconstrained minimum sits lower (around $0.21, where the market spends most of its time) but it concedes 18% downtime, which is exactly what the 95% constraint in (7) exists to reject. We attach a confidence interval to $b^{\ast }$ with a stationary bootstrap (400 resamples that preserve the burst structure), and every headline number is then scored on the held-out 40% the fit never saw, or replayed through the real bidder above.

How to trust these numbers

• Strict train/test discipline. Every fitted parameter comes from the first 60% of the window; every headline comparison is scored on data the model never saw, or on the full window through the production stack.
• The evaluator can't leak. True prices live only in the adjudication layer that decides preemptions and tallies costs; strategies receive reflections and reveals, exactly as live.
• Real code, not a toy. The final numbers come from the same daemon, guard rails, and API client we'd run with real money, exercised end-to-end over a mock exchange (41 tests green).
• Where it's weaker, we say so. Each tier's price is the cheapest of ~20 machines, which is smoother than any single machine, so our preemption rates are lower bounds, and the fitted bid will be re-estimated on per-machine data before real spend. The live window is still short (a day or two hours at a time), so held-out rankings can shuffle between visits, when they do, that's variance doing exactly what this page exists to show. And the mirror rule itself, while implied by the auction mechanics, hasn't yet been observed from a really-held instance, verifying it is the live pilot's first job.
• The one loss: on H200's held-out window the disqualified surge strategy nominally beat us, $1.32 vs $1.41, at borderline 95% uptime with 7 preemptions against our 1, riding a signal we showed doesn't statistically exist. We're comfortable calling that luck.

Why this matters beyond our GPU bill

At $0.74 per H100-hour versus $1.74 on-demand, every serving GPU costs 57% less, and that compounds directly into what inference costs on top of it. This is the supply-side twin of the runtime work we've published before: making each GPU do 3–4x more work, and now paying roughly half price for the GPU itself.

Next: the tracker's collector is being extended to record every machine's price individually (not just the cheapest per tier), the model refits continuously as data accumulates, and then a small live pilot verifies the one assumption a simulator can't, what the market really shows you about the machine you hold.

Run environment

Effective cost is total spend divided by hours actually served, warmups, storage, lost fights and idle disks all land in the numerator. On-demand baseline is the cheapest tracked on-demand price over the same window.

Learn more

• Live H100/H200 price tracker, the free tool this dataset comes from
• Up to 4x faster than vLLM: benchmarking Netra on real long-context workloads
• How much VRAM do you need to run an LLM?

Running long-context inference and curious what your workload would cost on infrastructure like this? Get in touch.