⏱️ 大模型训练时长估算（6ND 算力公式）LLM Training Time Calculator (6ND FLOPs)
按 6 × 参数量 × token 数估算大模型训练的总计算量，再除以 GPU 峰值算力 × 数量 × MFU（模型算力利用率）得到训练时长，给出 GPU 小时、天数和估算费用；内置 A100、H100、H200、H800、B200、RTX 4090 等的 BF16 稠密峰值算力。Estimate LLM training compute as 6 × parameters × tokens, then divide by GPU peak × count × MFU to get wall-clock time, GPU-hours, days and cost; presets give dense BF16 peaks for A100, H100, H200, H800, B200, RTX 4090 and more.
公式：总计算量 = k × N × D（k 取 6 或 8）；训练时间 = 总计算量 ÷（单卡峰值算力 × GPU 数量 × MFU）。峰值为 BF16 稠密（非稀疏）Tensor Core 算力。MFU（模型 FLOPs 利用率）在大规模训练中通常为 30%–55%，小规模或通信受限时更低。结果是理想估算，未计入故障重启、评估和数据加载停顿。Formula: total compute = k × N × D (k = 6 or 8); time = compute ÷ (peak per GPU × GPUs × MFU). Peaks are dense (non-sparse) BF16 tensor-core figures. MFU (model FLOPs utilisation) is typically 30%–55% at scale and lower for small or communication-bound runs. This is an idealised estimate that ignores restarts, evaluation and data-loading stalls.
使用说明与常见问题How-to & FAQ
「大模型训练时长估算（6ND 算力公式）」是什么？What is the LLM Training Time Calculator (6ND FLOPs)?
按 6 × 参数量 × token 数估算大模型训练的总计算量，再除以 GPU 峰值算力 × 数量 × MFU（模型算力利用率）得到训练时长，给出 GPU 小时、天数和估算费用；内置 A100、H100、H200、H800、B200、RTX 4090 等的 BF16 稠密峰值算力。Estimate LLM training compute as 6 × parameters × tokens, then divide by GPU peak × count × MFU to get wall-clock time, GPU-hours, days and cost; presets give dense BF16 peaks for A100, H100, H200, H800, B200, RTX 4090 and more.
如何使用「大模型训练时长估算（6ND 算力公式）」？How do I use the LLM Training Time Calculator (6ND FLOPs)?
填写参数量（B）和训练 token 数（B 或 T），选择 GPU 型号和数量，设置 MFU（大规模训练常见 30%–55%）。结果显示总 FLOPs、训练时长和 GPU 小时；填写每 GPU 小时价格还能估算费用。Enter parameters (B) and training tokens (B or T), pick the GPU and count, and set MFU (30%–55% is typical at scale). You get total FLOPs, wall-clock time and GPU-hours; add a price per GPU-hour to estimate cost.
为什么是 6ND？Why 6ND?
训练时每个 token 前向传播约 2N 次浮点运算，反向传播约 4N 次，合计约 6N；乘以 token 数 D 就是总计算量（Kaplan 等 2020、Chinchilla 论文使用同样的近似）。开启全量激活重算时约为 8N。Per token the forward pass costs about 2N FLOPs and the backward pass about 4N, so ≈6N; times D tokens gives total compute (the approximation used by Kaplan et al. 2020 and Chinchilla). Full activation recomputation makes it ≈8N.
结果可信吗？Is the result realistic?
以 Llama 2 7B 为例：6.74B 参数、2T token、A100 在 40% MFU 下约 18 万 GPU 小时，与论文报告的 184,320 GPU 小时接近。实际时长还受故障、评估和数据管线影响。For Llama 2 7B (6.74B params, 2T tokens) on A100 at 40% MFU it gives about 180k GPU-hours, close to the 184,320 reported in the paper. Real runs also lose time to failures, evaluation and data pipelines.
「大模型训练时长估算（6ND 算力公式）」免费吗？文件会上传吗？Is the LLM Training Time Calculator (6ND FLOPs) free? Are my files uploaded?
「大模型训练时长估算（6ND 算力公式）」完全免费，无需注册。你输入的内容只在本机浏览器里处理（并保存在本机方便下次继续），不会上传到任何服务器，适合处理未发表的论文和数据。The LLM Training Time Calculator (6ND FLOPs) is free with no sign-up. Everything you enter is processed in this browser only (and kept locally so you can continue later); nothing is uploaded, so it is safe for unpublished papers and data.