NEURONIX // CODE LAB interactive compute bench
A live simulator for the LLM memory wall and kernel operators
Adjust Batch Size and Context Length to watch, in real time, how much VRAM MLA saves and how its throughput curve compares with classic MHA/GQA.
Hyperparameters
TRITON COMPATIBLE
Formula assumptions: MHA / GQA-8 use a "Dense-70B-class" reference architecture (80 layers, head_dim=128, 64 KV heads for MHA / 8 KV heads for GQA, bf16, 2 bytes); MLA uses a DeepSeek-V3-class reference architecture (61 layers, a joint latent vector of kv_lora_rank=512 + qk_rope_dim=64, bf16, 2 bytes). VRAM = layers × bytes per token per layer × batch × actual token count — a real calculation, not decorative scaling.
KV Cache VRAM footprint, live comparison (GB):
Multi-Head Attention (MHA, Dense-70B-class)-- GB
Grouped-Query Attention (GQA-8, Dense-70B-class)-- GB
DeepSeek MLA (low-rank joint latent compression)-- GB
neuronix_profiler_terminal.sh
STREAMING
Target Device: NVIDIA H100 SXM5 (80GB HBM3e)
CUDA Driver: 12.4 | Triton Version: 3.0.0
Click “Run GPU kernel simulation” on the left to watch low-level VRAM scheduling and the operator pipeline log.