🧠 大模型显存估算（训练 / 推理 / LoRA）LLM GPU Memory Calculator (Training, Inference, LoRA)
估算大语言模型推理和训练需要多少显存：推理按权重精度（BF16/INT8/INT4）加 KV 缓存（考虑 GQA、序列长度和批大小）；训练区分全参微调（AdamW 混合精度约 16 字节/参数）、LoRA 和 QLoRA，可估算激活值和梯度检查点，并给出需要几张 A100、H100 或 RTX 4090，内置 Llama、Qwen、Mistral 常用配置。Estimate GPU memory for LLM inference and training: inference = weights at BF16/INT8/INT4 plus KV cache (GQA, sequence length, batch); training covers full fine-tuning (≈16 bytes/param for mixed-precision AdamW), LoRA and QLoRA, with optional activations and gradient checkpointing — and how many A100, H100 or RTX 4090 cards you need. Presets for Llama, Qwen and Mistral.
模型结构（用于 KV 缓存、LoRA 和激活值）Architecture (for KV cache, LoRA and activations)
| GPU | 显存Memory | 至少需要At least |
|---|
⚠️ 这是估算：实际占用还受框架、CUDA 上下文、显存碎片、并行策略（ZeRO / FSDP / 张量并行）、FlashAttention 等影响。多卡数量只按显存总量计算，需要配合分片（ZeRO-3 / FSDP）才能把模型分摊到多张卡上。This is an estimate: real usage also depends on the framework, CUDA context, fragmentation, parallelism (ZeRO / FSDP / tensor parallel) and FlashAttention. GPU counts only divide total memory and assume sharding (ZeRO-3 / FSDP) across cards.
使用说明与常见问题How-to & FAQ
「大模型显存估算（训练 / 推理 / LoRA）」是什么？What is the LLM GPU Memory Calculator (Training, Inference, LoRA)?
估算大语言模型推理和训练需要多少显存：推理按权重精度（BF16/INT8/INT4）加 KV 缓存（考虑 GQA、序列长度和批大小）；训练区分全参微调（AdamW 混合精度约 16 字节/参数）、LoRA 和 QLoRA，可估算激活值和梯度检查点，并给出需要几张 A100、H100 或 RTX 4090，内置 Llama、Qwen、Mistral 常用配置。Estimate GPU memory for LLM inference and training: inference = weights at BF16/INT8/INT4 plus KV cache (GQA, sequence length, batch); training covers full fine-tuning (≈16 bytes/param for mixed-precision AdamW), LoRA and QLoRA, with optional activations and gradient checkpointing — and how many A100, H100 or RTX 4090 cards you need. Presets for Llama, Qwen and Mistral.
如何使用「大模型显存估算（训练 / 推理 / LoRA）」？How do I use the LLM GPU Memory Calculator (Training, Inference, LoRA)?
选择推理、全参微调、LoRA 或 QLoRA，再选一个模型预设（或直接填参数量），设置序列长度和批大小。上方会直接给出“需要多少 GB、几张卡”的结论，下面列出权重、梯度、优化器状态、KV 缓存、激活值各占多少，以及每条计算假设。Pick inference, full fine-tuning, LoRA or QLoRA, choose a model preset (or type a parameter count) and set sequence length and batch size. The answer — GB and number of GPUs — appears on top, with the breakdown (weights, grads, optimizer, KV cache, activations) and every assumption listed below.
7B 模型 BF16 全参微调要多少显存？How much memory does 7B BF16 full fine-tuning need?
用 AdamW 混合精度时，每个参数需要 BF16 权重 2 字节 + 梯度 2 字节 + FP32 主权重 4 字节 + Adam 两个状态 8 字节 = 16 字节，7B 约 108–112 GB，再加激活值和开销，通常需要 2 张 A100 80GB 并配合 ZeRO/FSDP 分片，或改用 LoRA。With mixed-precision AdamW each parameter needs 2 (BF16 weights) + 2 (grads) + 4 (FP32 master) + 8 (Adam states) = 16 bytes, so 7B is about 108–112 GB before activations and overhead — typically 2× A100 80GB with ZeRO/FSDP sharding, or use LoRA instead.
为什么 Llama 3 的 KV 缓存比 Llama 2 小？Why is the Llama 3 KV cache smaller than Llama 2?
Llama 3 使用分组查询注意力（GQA），8B 模型只有 8 个 KV 头（Llama 2 7B 有 32 个），所以同样序列长度下 KV 缓存只有约四分之一。Llama 3 uses grouped-query attention: the 8B model has 8 KV heads versus 32 in Llama 2 7B, so its KV cache is about a quarter of the size at the same length.
估算结果准确吗？How accurate is it?
权重、梯度和优化器状态是精确计算；KV 缓存按公式计算；激活值是基于论文公式的近似。实际占用还受框架、FlashAttention、并行策略和显存碎片影响，建议预留 10%–20% 余量。Weights, grads and optimizer states are exact; the KV cache follows its formula; activations are a paper-based approximation. Frameworks, FlashAttention, parallelism and fragmentation also matter, so keep 10%–20% headroom.
「大模型显存估算（训练 / 推理 / LoRA）」免费吗？文件会上传吗？Is the LLM GPU Memory Calculator (Training, Inference, LoRA) free? Are my files uploaded?
「大模型显存估算（训练 / 推理 / LoRA）」完全免费，无需注册。你输入的内容只在本机浏览器里处理（并保存在本机方便下次继续），不会上传到任何服务器，适合处理未发表的论文和数据。The LLM GPU Memory Calculator (Training, Inference, LoRA) is free with no sign-up. Everything you enter is processed in this browser only (and kept locally so you can continue later); nothing is uploaded, so it is safe for unpublished papers and data.