LocalNodeOps

Synced from Hugging Face — 2026-09-14

How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need?

Exact on-disk weight sizes below are pulled directly from Hugging Face. Total VRAM figures also include an estimated KV cache and runtime overhead — see the breakdown per quantization, and the calculator below to adjust context length.

By quantization

Meta-Llama-3.1-8B-Instruct-IQ2_M

2.75 GB exact, on disk

~3.58 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-IQ3_M

3.52 GB exact, on disk

~4.42 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-IQ3_XS

3.28 GB exact, on disk

~4.16 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-IQ4_NL

4.36 GB exact, on disk

~5.35 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-IQ4_XS

4.14 GB exact, on disk

~5.10 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-Q2_K

2.96 GB exact, on disk

~3.81 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-Q2_K_L

3.44 GB exact, on disk

~4.33 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-Q3_K_L

4.03 GB exact, on disk

~4.98 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-Q3_K_M

3.74 GB exact, on disk

~4.66 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-Q3_K_S

3.41 GB exact, on disk

~4.30 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-Q3_K_XL

4.45 GB exact, on disk

~5.45 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-Q4_0_4_4

4.34 GB exact, on disk

~5.32 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-Q4_0_4_8

4.34 GB exact, on disk

~5.32 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-Q4_0_8_8

4.34 GB exact, on disk

~5.32 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-Q4_K_L

4.95 GB exact, on disk

~6.00 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-Q4_K_M

4.58 GB exact, on disk

~5.59 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-Q4_K_S

4.37 GB exact, on disk

~5.36 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-Q5_K_L

5.64 GB exact, on disk

~6.75 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-Q5_K_M

5.34 GB exact, on disk

~6.42 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-Q5_K_S

5.21 GB exact, on disk

~6.28 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-Q6_K

6.14 GB exact, on disk

~7.30 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-Q6_K_L

6.38 GB exact, on disk

~7.57 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-Q8_0

7.95 GB exact, on disk

~9.29 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

Meta-Llama-3.1-8B-Instruct-f32

29.92 GB exact, on disk

~33.46 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

VRAM calculator

Estimate total VRAM for a given model size, quantization, and context length. Estimates only — see the note at the bottom.

7B13B70B120B
2K4K8K16K32K64K128K
Model weights
—
KV cache
—
Runtime overhead (10%)
—
Total VRAM
—

Fits on:

    Live changes

    • No changes yet — adjust a control above.

    Generic estimate: weight size is bits-per-weight × parameter count; KV cache uses a reference architecture for the selected size bucket (7B/13B/70B: real published models; 120B: extrapolated, no real model exists at that exact size). Model-specific (HF-synced): weight size is the exact .gguf file size fetched from Hugging Face — no estimation. KV cache is still estimated: this schema doesn't carry layer count, head count, or head dimension, so the calculator parses an approximate parameter count from the model's title (e.g. "8B") and reuses the nearest size bucket's reference architecture for the KV math, same caveats as the generic mode. If no parameter count can be parsed from the title, it falls back to the 7B architecture and says so next to the quantization dropdown. Batch size is fixed at 1 in both modes. Treat all figures here as a starting estimate, not a guarantee. Full formula and known limitations: /methodology.

    FAQ

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-IQ2_M?

    The Meta-Llama-3.1-8B-Instruct-IQ2_M quantization is 2.75 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 3.58 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-IQ3_M?

    The Meta-Llama-3.1-8B-Instruct-IQ3_M quantization is 3.52 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 4.42 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-IQ3_XS?

    The Meta-Llama-3.1-8B-Instruct-IQ3_XS quantization is 3.28 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 4.16 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-IQ4_NL?

    The Meta-Llama-3.1-8B-Instruct-IQ4_NL quantization is 4.36 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 5.35 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-IQ4_XS?

    The Meta-Llama-3.1-8B-Instruct-IQ4_XS quantization is 4.14 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 5.10 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-Q2_K?

    The Meta-Llama-3.1-8B-Instruct-Q2_K quantization is 2.96 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 3.81 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-Q2_K_L?

    The Meta-Llama-3.1-8B-Instruct-Q2_K_L quantization is 3.44 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 4.33 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-Q3_K_L?

    The Meta-Llama-3.1-8B-Instruct-Q3_K_L quantization is 4.03 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 4.98 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-Q3_K_M?

    The Meta-Llama-3.1-8B-Instruct-Q3_K_M quantization is 3.74 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 4.66 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-Q3_K_S?

    The Meta-Llama-3.1-8B-Instruct-Q3_K_S quantization is 3.41 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 4.30 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-Q3_K_XL?

    The Meta-Llama-3.1-8B-Instruct-Q3_K_XL quantization is 4.45 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 5.45 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-Q4_0_4_4?

    The Meta-Llama-3.1-8B-Instruct-Q4_0_4_4 quantization is 4.34 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 5.32 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-Q4_0_4_8?

    The Meta-Llama-3.1-8B-Instruct-Q4_0_4_8 quantization is 4.34 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 5.32 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-Q4_0_8_8?

    The Meta-Llama-3.1-8B-Instruct-Q4_0_8_8 quantization is 4.34 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 5.32 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-Q4_K_L?

    The Meta-Llama-3.1-8B-Instruct-Q4_K_L quantization is 4.95 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 6.00 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-Q4_K_M?

    The Meta-Llama-3.1-8B-Instruct-Q4_K_M quantization is 4.58 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 5.59 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-Q4_K_S?

    The Meta-Llama-3.1-8B-Instruct-Q4_K_S quantization is 4.37 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 5.36 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-Q5_K_L?

    The Meta-Llama-3.1-8B-Instruct-Q5_K_L quantization is 5.64 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 6.75 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-Q5_K_M?

    The Meta-Llama-3.1-8B-Instruct-Q5_K_M quantization is 5.34 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 6.42 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-Q5_K_S?

    The Meta-Llama-3.1-8B-Instruct-Q5_K_S quantization is 5.21 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 6.28 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-Q6_K?

    The Meta-Llama-3.1-8B-Instruct-Q6_K quantization is 6.14 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 7.30 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-Q6_K_L?

    The Meta-Llama-3.1-8B-Instruct-Q6_K_L quantization is 6.38 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 7.57 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-Q8_0?

    The Meta-Llama-3.1-8B-Instruct-Q8_0 quantization is 7.95 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 9.29 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does Meta-Llama-3.1-8B-Instruct-GGUF need at Meta-Llama-3.1-8B-Instruct-f32?

    The Meta-Llama-3.1-8B-Instruct-f32 quantization is 29.92 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 33.46 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.