Price, customize, and provision compute for your AI agent.

Formula

input_tokens_per_day = requests_per_day x input_tokens; output_tokens_per_day = requests_per_day x output_tokens; avg_requests_per_s = requests_per_day / 86400; peak_requests_per_s = avg x peak_factor, or as given, or peak_concurrency / answer_time; for each replica of n GPUs (fewest that fit up to max_gpus): per_stream = fit throughput at c streams / c; answer = ttft(input_tokens) + output_tokens / per_stream; in_flight = peak_requests_per_s x answer (Little's law), c iterated until it settles; replicas = ceil(in_flight / (target_utilization x c)); answer_time = ttft + output_tokens / per_stream_tok_s; utilization_avg = avg_output_tok_s / (replicas x replica_capacity_tok_s); self_host_usd_per_day = total_gpus x 24 x gpu_usd_per_hour; api_usd_per_request = (input_tokens x ((1 - cached_share) + cached_share x (1 - discount)) x usd_per_m_input + output_tokens x usd_per_m_output) / 1e6; usd_per_month = usd_per_day x 730 / 24; break_even_requests_per_day = gpus_per_replica x 24 x gpu_usd_per_hour / api_usd_per_request, for the cheapest replica shape that meets the latency target with every slot busy and carries that volume at the target utilization

Assumptions

peak_source:

peak_factor

target_utilization:

0.7

always_on:

true

hours_per_month:

730

capacity:

fit estimate. Replicas of the fewest GPUs that fit up to max_gpus are tried. Streams per replica settle at the peak load; per-answer speed = replica throughput / streams, with every slot busy (on the slow side). Replica capacity = throughput at the most streams its GPUs hold within the latency target.

model:

Qwen/Qwen3.6-27B

precision:

fp8

weights:

FP8

kv:

bf16

context:

2048

prompt_tokens:

1500

max_gpus:

8

max_streams_per_replica:

256

streams_per_replica:

3

sized_for:

fewest_gpus

peak_factor:

3

Not checked

• · Engineering and operations time to run your own serving.
• · Redundancy: a spare replica (N+1) for failures and deploys.
• · Network egress and storage for weights and logs.
• · Idle scale-down: cold starts, minimum billing and load time.
• · Quality differences between the self-hosted model and the API model.
• · API rate limits and your provider's terms.
• · Measured speeds: fit speeds are bandwidth-bound upper bounds; confirm with a short load test.

Formula

units = max over gpus, vcpus and memory_gb the plan needs of ceil(plan / offer unit); 1 when the plan needs none. unit price = price_usd_per_hour, or price_usd_per_gpu_hour x offer gpus. Expected wall hours W = plan.hours; on an interruptible offer with mtbi_hours, Daly's W = (T / tau) x M x e^(R/M) x (e^((tau + C)/M) - 1), M = mtbi_hours / units, C = checkpoint hours, R = restart hours, tau = sqrt(2 x C x M) (Young); with no checkpoint_minutes, tau = T and C = 0. billed seconds per unit = max(minimum_billed_seconds, ceil(W x 3600 / increment) x increment). compute = units x unit price x billed seconds / 3600. storage = storage_gb x storage_usd_per_gb_month x W / 730. egress = egress_gb x egress_usd_per_gb. setup = setup_usd, once per run. total = compute + storage + egress + setup. effective per GPU-hour = total / (plan.gpus x plan.hours). monthly = total x runs_per_month. Break-even: the nearest run lengths under and over plan.hours (searched from 0 to 10x plan.hours) where the runner-up's total drops below the cheapest's, found exactly between billing steps.

Assumptions

billing_increment:

3600 s (by the hour) when an offer sends none; stated per offer.

missing_prices:

A cost item with no price is left out ($0) and listed under left_out_no_price and not_checked. An offer with no compute price is not ranked.

missing_sizes:

An offer that does not state a size the plan needs is not ranked (status unverified).

units_together:

All units of an offer run the plan together; any unit's interruption stalls the run.

interruptions:

Interruptions arrive at random at a steady rate. Restart time is billed. Time lost to interruptions is billed like any other time.

not_finishing:

An interruptible offer whose expected wall time passes 10x plan.hours is ineligible.

storage:

Pro-rated over the expected wall time, 730 hours to a month.

setup:

setup_usd is charged once per run.

interruptible:

An offer that does not send interruptible is treated as not interruptible.

Not checked

• · Offer A: no minimum_billed_seconds, priced with no minimum.
• · Offer C: no minimum_billed_seconds, priced with no minimum.
• · Whether each offer has capacity when you need it.
• · Taxes, commitments, discounts, credits and support fees.
• · Data transfer into the provider and between zones.
• · Billing restarts after interruptions: an interrupted run is billed as one continuous run.

Formula

calls_total = ceil(tasks x calls_per_task / (1 - failure_rate)); counted_input = input_tokens, or input_tokens - cached_prefix_tokens x cache_hit_rate when cached tokens don't count; ceiling (calls/min) = limit x headroom / per-call use: rpm 1; input tpm counted_input; output tpm output_tokens; total tpm counted_input + output_tokens; concurrency latency_s / 60; target_calls_per_min = smallest ceiling; concurrency = floor(target_calls_per_min / 60 x latency_s), at least 1, at most concurrency_limit (Little's law); reached_calls_per_min = min(target_calls_per_min, concurrency x 60 / latency_s); time_to_finish_min = (calls_total - 1) / reached_calls_per_min + latency_s / 60 (the last call starts, then runs one latency); cost_no_cache = (calls_total x input_tokens x usd_per_m_input + calls_total x output_tokens x usd_per_m_output) / 1e6; cost_with_cache = ((uncached + prefix x hit x read_mult + prefix x (1 - hit) x write_mult) x calls_total x usd_per_m_input + output x usd_per_m_output) / 1e6, write_mult = 1 when only read_mult is sent; cache_break_even_hit_rate = (write_mult - 1) / (write_mult - read_mult); batch_cost = cost_no_cache x (1 - batch_discount); suggested_hard_cap = min(expected x 1.2, budget_usd); tasks_affordable = floor(budget_usd / (expected / tasks))

Assumptions

headroom:

0.8

failure_rate:

0.03

retries:

every failed call is retried until it succeeds and is billed as a full call

cache_hit_rate:

0.9

cached_counts_toward_input_tpm:

true

cache_write:

a call that misses the cache writes the prefix at the write multiplier

cache_write_multiplier:

1.2

hard_cap:

expected cost plus 20%, never above budget_usd

steady_state:

calls run at a steady rate; ramp-up and bursts are not modeled

prices:

only the prices you sent; none are built in

batch:

the batch discount is applied to the no-cache cost; whether it stacks with caching depends on your terms

Not checked

• · Whether your limits are per model, per key or per account, and how they count bursts.
• · Whether output limits count max_tokens up front instead of tokens returned.
• · Whether failed calls are billed: they are counted here as full calls.
• · Whether cached tokens count toward your input limit: assumed yes. Set cached_counts_toward_input_tpm to false if not.

Formula

context(1) = system + tools + first_message; context(t) = context(t-1) + added_tokens_per_turn + output_tokens_per_turn; input billed per session = sum of context(t) over every call; without compaction = turns x context(1) + (added + output) x turns x (turns - 1) / 2; compaction: when context(t) would pass compact_at, one extra call reads it and writes a summary of (compact_to - system - tools) tokens; context(t) = compact_to; static cache: first call writes system + tools; later calls read them at hit rate, rewrite on a miss; rolling cache: first call writes the whole context; later calls read the previous call's context at hit rate and write the rest; input cost = (uncached + cache_read x read_mult + cache_write x write_mult) x usd_per_m_input / 1e6; output cost = output tokens x usd_per_m_output / 1e6; tool schema share = tool_schema_tokens x calls / input billed; cache_break_even_hit_rate = (write_mult - 1) / (write_mult - read_mult); per_month = per_day x 30

Assumptions

cache_hit_rate:

0.9

output_joins_context:

each turn's output is appended to the next turn's context

first_call_writes_cache:

the first call of a session always writes the cache

compaction_call:

a summary call reads the whole context and is billed like a turn; in rolling mode it writes no cache

tool_schema_what_if:

halving tool schemas also lowers compact_at and compact_to by the tokens cut, so the summary length stays the same

compaction_what_if:

the half-peak what-if keeps 20% of the conversation beyond system and tools

overflow_rule:

a call overflows when its input plus its output passes context_limit

days_per_month:

30

prices:

only the prices you sent; none are built in

Not checked

• · Exact token counts: they depend on the model's tokenizer.
• · Minimum cacheable prefix sizes, cache lifetimes and expiry between turns.
• · Long-context price tiers: one input price is used for every turn.

Formula

machines = ceil(gpus / gpus_per_machine) = ceil(8 / 8) = 1. Fleet MTBI M = mtbi_hours / machines = 4 / 1 = 4 h (any interruption stalls the job). Optimal interval (Young) tau = sqrt(2 x C x M), C = checkpoint hours. Daly: tau = sqrt(2 x C x M) - C. Checkpoint overhead = C / tau. Expected wall W = (T / tau) x M x e^(R/M) x (e^((tau + C)/M) - 1), R = restart hours (Daly's model for random interruptions). Past 10x the job hours the job is marked as not finishing. Expected interruptions = W / M. Expected lost hours = W - T x (1 + C/tau). Spot GPU-hours = gpus x W. Spot USD = GPU-hours x spot price. Pessimistic case: M / 2, same interval. On-demand wall = T (no checkpoints). Break-even MTBI: per-machine MTBI where spot W x spot price = on-demand wall x on-demand price (solved numerically). Worst case = gpus x hard_stop_hours x spot price. Largest stop in budget = budget / (gpus x spot price). Suggested stop = min(largest stop in budget, 1.5 x pessimistic W). Checkpoint size: weights = 2 bytes x params; full training state = 16 bytes x params (AdamW).

Assumptions

interruptions:

Interruptions arrive at random at a steady average rate (exponential).

lost_work_per_interruption:

The work since the last checkpoint plus the restart time. Interruptions can also land during checkpoint writes and restarts.

restart_billed:

Restart time is billed at the spot price.

interval_used:

optimal (Young)

machines_rounding:

Whole machines.

synchronous:

Any one machine's interruption stalls the whole job.

checkpoint_bytes_per_param:

{"weights":2,"full_state":16,"note":"Full state uses the upper end of 14-16 bytes per parameter for AdamW."}

on_demand:

On demand is assumed never interrupted.

on_demand_checkpoints:

None counted: on-demand wall time is the job hours, whatever checkpoint interval you send.

Not checked

• · capacity availability
• · data loading after restart
• · preemption notice handling
• · storage and egress

Formula

status = kept if id or label is in keep; else leftover if it has run_tag or its id is in planned_ids; else unknown (all matches case-insensitive); verdict = leftovers if any leftover; else unknown_resources if any unknown; else clean; hourly = hourly_usd, or monthly_usd / 730; monthly = monthly_usd, or hourly_usd x 730; burn per hour = sum of leftover hourly; per day = x 24; per month = sum of leftover monthly; age_hours = (now - started_at) / 1 hour, or age_hours as sent; accrued = sum of age_hours x hourly over leftovers with both; likely companion = an unknown volume, snapshot, ip or load_balancer whose id or label contains a leftover compute item's id or label as a whole word, or one of whose tag values equals it

Assumptions

hours_per_month:

730

accrued:

each leftover billed at one steady hourly price for its whole age; stopped time, free tiers and minimum charges are not modeled

keep_wins:

an item in keep is kept even when it carries the run tag

unknown:

unknown items may belong to someone else and are never put in the removal order

prices:

only the prices you sent; none are built in

clock:

now as sent; this tool never reads a clock

Not checked

• · Whether the list is complete: every account, project, region and zone the run used must be listed.
• · Charges that outlive a resource (transfer, requests, minimum storage durations, logs) and stopped resources that still bill for disks or addresses.
• · Whether the prices you sent are current, and whether they include tax, discounts or credits.
• · Resources the run created without its tag and outside planned_ids: they show as unknown.

Formula

weights = parameters x bytes per parameter (native: checkpoint size; fp8 and int8: 1; int4: 0.5625); kv = KV bytes per token x context x concurrency (sliding-window layers capped at the window); per_gpu = (weights + state) / gpus + kv x copies / gpus, where MLA caches are copied on every tensor-parallel GPU; fits when per_gpu <= memory x 0.9 - 1 GB (serving), memory x 0.95 - 1.5 GB (training), or 75% of a Mac's memory; precision = the first of native, 8-bit, int4 that fits; fastest_precision = the one that fits with the lowest answer_s (else the highest per_stream_tok_s); decode step = max(bytes read / (bandwidth x 0.7), 2 x active x streams / (TFLOPS x 0.5)) + 1 ms + layers x 20 us (40 us MoE) + 30 us per layer when split; decode_tok_s = 1 / step(1); throughput_tok_s = concurrency / step(concurrency); per_stream_tok_s = throughput_tok_s / concurrency; ttft_s = max((2 x active x prompt + attention FLOPs) / (TFLOPS x 0.4), weights / bandwidth) + step overhead; answer_s = ttft_s + output_tokens / per_stream_tok_s; copies = floor(count / gpus_used); throughput_all_copies_tok_s = throughput_tok_s x copies

Assumptions

gb:

GiB (2^30 bytes), as GPU drivers report memory

serve_usable_fraction:

0.9

serve_reserve_gb_per_gpu:

1

train_usable_fraction:

0.95

train_reserve_gb_per_gpu:

1.5

local_usable_fraction:

0.75

bandwidth_efficiency:

0.7

decode_compute_efficiency:

0.5

prefill_efficiency:

0.4

train_mfu:

0.35

int4_bytes_per_param:

0.5625

full_finetune_bytes_per_param:

16

lora_adapter_share:

0.004

qlora_slowdown:

1.3

unknown_tflops_per_gbs:

0.15

local_tflops_per_gbs:

0.035

multi_gpu_scaling:

fabric parts 0.9 within a machine, 0.7 across machines; bridged pairs 0.9 (2), 0.65 (4), 0.5 (8); PCIe-only parts 0.8 (2), 0.65 (4), 0.5 (8); bridged and PCIe parts one machine max

kv_context:

every sequence at the full context length (worst case)

gradient_checkpointing:

true

decode_step_overhead_ms:

1

decode_layer_overhead_us:

20

decode_layer_overhead_us_moe:

40

tensor_parallel_overhead_us_per_layer:

30

upper_bound_step_ms:

5

speed_basis:

decode step = max(bytes read / bandwidth, FLOPs / compute) + fixed overhead + per-layer overhead; conservative constants, not measured on your stack

pipeline_parallel:

past one machine the model runs in pipeline stages, one after another: one stream gets one machine's bandwidth, and throughput gains from more stages only when concurrency fills them

kv_tensor_parallel:

MLA and other single-latent caches are copied on every tensor-parallel GPU; GQA caches are copied when a machine's GPUs outnumber the KV heads; data-parallel attention is not assumed

prompt_tokens:

time to first token uses prompt_tokens, or the full context when not given

attention_flops:

prefill attention: 2 x prompt^2 x width per full-attention layer (causal), about 4 x prompt x window x width per sliding layer, at the BF16 rate; linear-attention and Mamba layers add none

training_sharding:

FSDP or ZeRO-3 style: weights, gradients and optimizer states split evenly across GPUs; with plain data parallel each GPU holds the whole model

kv_estimate:

no usable config.json: 64 KB x sqrt(params in billions) per token, low confidence; pass the attention shape for an exact figure

Not checked

• · prices: compute.md has none; use your own quotes
• · models outside the quick picks (use fit_model with any Hugging Face id)
• · quality loss from quantization
• · whether your serving software supports each precision on this part
• · real speed on your stack: measure with a short test