LLM VRAM Calculator
Estimate inference VRAM for model weights, quantization overhead, grouped-query KV cache, concurrency, and runtime reserve. Get an editable GB and GiB planning range.
Tell us more, and we'll get back to you.
Contact UsAbout LLM VRAM Calculator
Practical context, assumptions, examples, and next steps for using the result well.Estimate inference memory, not device compatibility
An LLM VRAM calculator estimates the memory components that must coexist while a language model performs inference. Model weights are the largest obvious component, but they are not the whole requirement. An autoregressive transformer can retain keys and values for previous tokens, while the runtime needs workspace and other allocations. A deployment that fits the weight file alone may still fail while processing a long prompt or serving several sequences at the same time.
This planner adds packed model weights, a user-selected weight-overhead allowance, a full-context KV cache, and an explicit runtime reserve. It then applies low and high headroom percentages to the complete subtotal. The two outputs are a planning range derived from your assumptions, not a statistical confidence interval or a guarantee from a hardware manufacturer. Both ends are shown in decimal GB and binary GiB so the unit choice does not remain hidden.
The default values describe an illustrative seven-billion-parameter model with editable cache dimensions. They are not a verified preset for a named model release. Replace the parameter count, layer count, number of key/value heads, and head dimension with the actual configuration you intend to run. This tool is designed for inference planning; it does not estimate training optimizer states, gradients, saved activations, device placement, tokens per second, electricity cost, or a complete hosted-service bill.
Calculate weights and account for quantization overhead
The ideal packed weight payload is parameter count multiplied by bits per weight, divided by eight bits per byte. If parameter count is entered in billions, seven billion parameters at four bits per weight occupy 3.5 billion bytes, or 3.5 decimal GB, before any additional allocation. At sixteen bits, the same parameter count has a 14 GB ideal payload. This simple equation is useful for comparisons, but an actual checkpoint is not necessarily one uniform, perfectly packed tensor.
Quantized formats can store scaling factors, zero points, grouping metadata, padding, and some tensors in a higher precision than the main weight format. Different implementations make different choices. A format described as four-bit does not by itself specify a universal overhead percentage. The weight-overhead input is therefore an editable planning allowance applied to the packed weight payload, rather than a claimed property of every quantization scheme. You can use a fractional bits-per-weight value if you have a measured effective payload estimate.
For example, 3.5 GB of packed weights with ten percent overhead adds 0.35 GB and gives 3.85 GB before cache or runtime reserve. If your effective bits-per-weight figure already includes metadata, avoid counting that same cost twice through an additional overhead allowance. Likewise, a checkpoint's file size may include serialization or tensors handled differently at runtime. Prefer measured loaded weight memory when it is available, and keep the interpretation of each input recorded beside the estimate.
Use KV heads in the cache formula
The modeled KV-cache byte count is 2 × layers × KV heads × head dimension × context tokens × concurrent sequences × cache bytes per element. Cache bytes per element equals cache precision in bits divided by eight. The factor of two accounts for the key cache and value cache. It does not mean multiplying by both a separate key-head count and a separate value-head count. The formula assumes both caches use the same head count, dimension, and element precision.
Use the key/value head count from the model configuration. In ordinary multi-head attention it can equal the query-head count. In grouped-query attention, several query heads share a smaller set of key/value heads, so copying the query-head count into this field can substantially overestimate memory. Multi-query attention can use a single key/value head. The calculator asks for KV heads explicitly because parameter count or a model's marketing name cannot reliably determine that architecture detail.
With thirty-two layers, eight KV heads, a head dimension of 128, 8,192 tokens, one sequence, and sixteen-bit cache entries, the formula gives 1,073,741,824 bytes. That is exactly 1 GiB, or 1.073741824 decimal GB. Changing eight KV heads to thirty-two multiplies this cache estimate by four without changing the weight estimate. Changing cache precision from sixteen to eight bits halves the modeled element payload, but actual support and additional quantized-cache allocations must be checked in the chosen runtime.
Keep decimal GB and binary GiB distinct
A decimal gigabyte, abbreviated GB, contains one billion bytes. A gibibyte, abbreviated GiB, contains 1,073,741,824 bytes. Memory tools, specifications, file managers, and cloud interfaces do not always use these labels consistently. A capacity advertised or displayed as a bare number without a clear byte convention is therefore ambiguous. Compare byte counts or explicitly documented units before deciding whether a planning range fits within a device's available memory.
This calculator converts every modeled component to bytes internally and displays the completed range using both conventions. For example, a 10 GB requirement is approximately 9.313 GiB. The smaller numerical GiB value represents the same memory requirement; conversion has not reduced the actual allocation. Similarly, the example cache in the previous section is 1 GiB and approximately 1.074 GB. Treat these as equivalent descriptions rather than two estimates with different accuracy.
The runtime reserve field uses decimal GB, and the component breakdown also uses decimal GB throughout. Headroom percentages are dimensionless and apply after all components have been combined in bytes. This consistent sequence avoids adding a binary value directly to a decimal value with the same-looking label. If a profiler reports GiB, convert that measurement to decimal GB before entering it as reserve, or work from its raw byte count. Keep display rounding separate from the underlying comparison, especially when an estimate lies close to a memory limit.
Context and concurrency grow the cache together
Context tokens means the number of tokens retained for each sequence at the point you want to plan for. Include the prompt and the generated continuation, rather than entering only the newly generated output. A request with a long prompt can already need substantial cache before it produces a useful response. This calculator assumes the complete entered context is resident for every concurrent sequence; it does not infer an average occupancy from request statistics.
For independent sequences with equal context lengths, cache memory scales directly with their count. Four sequences at 8,192 tokens require the same modeled cache payload as one sequence at 32,768 tokens, given identical layers, heads, dimensions, and precision. The shared weight copy does not multiply by sequence count in this model. This explains why serving concurrency can change the overall memory requirement substantially even when the selected model and weight quantization remain unchanged.
Actual servers can implement variable-length allocation, paged caches, prefix sharing, sliding windows, or static preallocation. Those strategies affect how the simple full-context estimate relates to observed allocation. Prefix sharing can avoid storing identical prefix content repeatedly, while a server configured to reserve a large cache pool can allocate more than the active sequences currently use. Use the same context policy when comparing scenarios. For a conservative request budget, consider the largest supported retained context and simultaneous occupancy you expect to permit, then verify that peak workload in the actual serving stack.
Separate runtime reserve from planning headroom
The runtime reserve represents expected memory beyond weights, weight overhead, and the explicitly modeled KV cache. It can cover kernel workspace, temporary activations, communication buffers, execution-graph allocations, and other backend-specific needs. These costs vary with implementation, batching, sequence length, and execution phase. The default reserve is an editable illustration, not a universal benchmark. Use measurements from the intended runtime and workload when you have them, especially for prompt-processing peaks.
Headroom is an additional percentage of the complete modeled subtotal. If weights, overhead, cache, and reserve total 8 GB, ten percent headroom gives 8.8 GB and twenty-five percent gives 10 GB. The low end uses the lower percentage and the high end uses the higher percentage; the calculator rejects a reversed range. Equal percentages are allowed if you want a single scenario displayed at both endpoints. Neither endpoint is automatically safer than the assumptions entered to produce it.
This definition differs from requiring a fraction of the device to remain free. Adding twenty-five percent to an 8 GB subtotal gives 10 GB. Requiring twenty-five percent of installed capacity to stay free would instead require dividing 8 GB by 0.75, or approximately 10.667 GB. The interface states which convention is used so these two policies are not confused. Do not disguise a known runtime allocation as vague headroom when it can be measured and entered explicitly; clear component assumptions make later revisions much easier.
Know when the architecture exceeds this model
The cache equation describes a conventional transformer with uniform key/value cache geometry across its entered layers. Some models use different layer patterns, different key and value dimensions, latent attention representations, recurrent state, cross-attention, or hybrid architectures. Sliding-window layers may retain fewer tokens than full-attention layers. Multimodal models may also allocate image, audio, or encoder-related tensors. For those cases, a single layer count and head geometry may not faithfully describe the actual memory requirement.
Mixture-of-experts parameter counts need careful interpretation. The number of parameters active for one token is not necessarily the number of parameters that must remain resident. If all expert weights are loaded, use the resident total rather than the active subset. Offloading weights or cache to host memory changes placement, transfer behavior, and performance, and is not modeled as a free memory reduction here. A lower local allocation is not by itself proof that the resulting deployment meets latency requirements.
The result is an aggregate planning quantity rather than a device-placement plan. Dividing it by the number of GPUs can miss uneven layer placement, duplicated allocations, communication buffers, or per-device cache behavior. A device may also lack the required kernels, instructions, software support, or contiguous free memory even when its nominal capacity exceeds the estimate. Training has additional large allocations and is outside this tool's scope. Treat the estimate as a transparent first pass that identifies likely constraints before a deployment-specific test.
Build a reproducible deployment check
Start with the actual model configuration and intended quantized artifact. Record resident parameter count, weight format, layer count, KV-head count, head dimension, cache precision, and the serving runtime version. Choose the maximum retained context and concurrent sequences you plan to support. Enter a runtime reserve based on measurement or an explicitly provisional assumption, then choose a headroom range that reflects how uncertain the workload and allocator behavior remain.
Compare scenarios by changing one assumption at a time. Reducing weight precision primarily affects the weight portion, while reducing retained context or concurrency primarily affects cache. A grouped-query configuration can reduce cache without proportionally reducing model weights. This decomposition helps avoid an expensive but ineffective change: buying more weight compression may not solve a deployment whose dominant allocation is a long-context cache. The result rows make these components visible rather than hiding them behind one unexplained total.
Finally, load the actual model on the intended stack and measure both steady allocation and peak allocation during representative prompt processing and generation. Test the longest permitted context, the planned concurrent load, and realistic scheduling behavior. Compare available device memory, not only installed capacity, and leave room for other processes if the device is shared. Update reserve and overhead from observed results, retain the assumptions with your deployment notes, and repeat after important software or model changes. The calculator can organize a hardware or hosting discussion, but measured behavior remains the final compatibility and capacity check.
Frequently Asked Questions
How much VRAM do seven billion parameters need at four bits?
The packed weight payload alone is 7 billion × 4 ÷ 8, or 3.5 decimal GB. Actual inference also needs quantization metadata or higher-precision tensors, a KV cache, and runtime workspace. The calculator adds these separately and then applies your chosen planning margins, so 3.5 GB is not a claim that the complete model will run on a device with that capacity.
Which attention head count belongs in the KV-cache formula?
Use the number of key/value heads from the model configuration, not automatically its query attention head count. Grouped-query attention shares key/value heads among multiple query heads, while multi-query attention can share a single key/value head. Substituting the larger query-head count can substantially overestimate the cache.
Does four-bit weight quantization also make the cache four-bit?
Not necessarily. Weight format and cache format are separate runtime settings with different support and accuracy tradeoffs. This planner accepts 8-, 16-, or 32-bit cache precision independently of weight precision; choosing a smaller cache setting is only meaningful if your actual backend implements it. Extra cache quantization metadata or residual buffers should be reflected in reserve and validated by measurement.
What changes when I serve several requests at once?
For independent sequences at the same context length, the modeled cache scales directly with sequence count while one shared copy of the model weights remains unchanged. Four concurrent sequences therefore require four times the single-sequence cache in this model. Prefix sharing, variable sequence lengths, batching policy, and preallocated cache pools can change real memory allocation.
Is the estimated range a guarantee that a particular GPU will work?
No. The range reflects your selected overhead, reserve, and headroom assumptions rather than certified device compatibility or a measured confidence interval. Kernel support, available rather than installed memory, fragmentation, prefill peaks, and software versions can still prevent execution. Confirm the model format and runtime support and measure peak allocation on the intended system.
Can this estimate training memory or multi-GPU placement?
It is an inference-only, aggregate memory estimate. Training adds gradients, optimizer states, saved activations, and often master weights, which are not modeled here. Dividing the result by a GPU count does not prove that tensor placement, cache partitioning, communication buffers, or interconnect requirements will work; use a deployment-specific profiler for that decision.