Hardware Status
Hardware not selected

Environment Profile

Local LLM fit analyzer

Configure hardware parameters, compare supported local models, and estimate inference performance under different quantization paths.

Primary Compute
Custom GPUNo GPU selected
Host Processor
Custom CPUManual
System Memory
Discrete GPU + system RAM0 GB RAM

Memory Allocation Matrix

Q4KVFREE
Model Weights (12.7GB)KV Cache (3.2GB)Free (0.0GB)
0GB1GB total memory

Model Discovery

Model explorer

Filter and compare local LLMs against the active hardware profile.
Model IDPrimary StrengthsAction
ZAYA1-8BQ4 quantizedZyphra0 mo8B / A760M MoE0.76B active for speed6.7 GB190% memory~3.9 t/s
Efficient local chat
CardZAYA1-8BA compact Zyphra MoE model with a sub-1B active path, useful for fast local tests when memory and throughput matter.8B / A760M MoEBenchmarks not listed
Grok 4.3Q4 quantizedxAI0 mo420B / A60B MoE60B active for speed235.4 GB190% memory~0.4 t/s
Efficient local chat
CardGrok 4.3A large Grok-family MoE model with a heavy active path, best treated as an aspirational local target for high-memory systems.420B / A60B MoEBenchmarks not listed
Qwen 3.6-27BQ4 quantizedAlibaba (Qwen)1 mo27B Dense17.6 GB190% memory~0.4 t/s
CodingMathReasoning
CardQwen 3.6-27BQwen 3.6-27B looks strongest for coding and math. Its 27B dense footprint makes hardware fit the main deciding factor.27B DenseMMLU Pro: 86.2% | AIME 2026: 94.1% | SWE-bench: 77.2%
Qwen 3.6-35B-A3BQ4 quantizedAlibaba (Qwen)1 mo35B / A3B MoE3B active for speed21.6 GB190% memory~1.4 t/s
CodingMathReasoning
CardQwen 3.6-35B-A3BQwen 3.6-35B-A3B looks strongest for coding and math. Its 35B MoE shape activates about 3B parameters per token, so it may feel lighter than the headline size suggests.35B / A3B MoEMMLU Pro: 85.2% | AIME 2026: 92.2% | SWE-bench: 73.4%
Gemma 4 31B DenseQ4 quantizedGoogle1 mo31B Dense19.9 GB190% memory~0.4 t/s
MathReasoningGeneral knowledge
CardGemma 4 31B DenseGemma 4 31B Dense looks strongest for math and reasoning. Its 31B dense footprint makes hardware fit the main deciding factor.31B DenseMMLU Pro: 85.2% | AIME 2026: 89.2% | SWE-bench: 52%
Gemma 4 26B MoEQ4 quantizedGoogle1 mo26B / A4B MoE4B active for speed16.7 GB190% memory~1.1 t/s
MathReasoningGeneral knowledge
CardGemma 4 26B MoEGemma 4 26B MoE looks strongest for math and reasoning. Its 26B MoE shape activates about 4B parameters per token, so it may feel lighter than the headline size suggests.26B / A4B MoEMMLU Pro: 82.6% | AIME 2026: 88.3% | SWE-bench: 48.5%
Gemma 4 7.9BQ4 quantizedGoogle1 mo7.9B Dense6.4 GB190% memory~0.6 t/s
Efficient local chat
CardGemma 4 7.9BGemma 4 7.9B looks strongest for efficient local chat. Its 7.9B dense footprint makes hardware fit the main deciding factor.7.9B DenseMMLU Pro: 69.4% | AIME 2026: 42.5% | SWE-bench: 23.6%
Gemma 4 5.1BQ4 quantizedGoogle1 mo5.1B Dense4.8 GB190% memory~0.8 t/s
Efficient local chat
CardGemma 4 5.1BGemma 4 5.1B looks strongest for efficient local chat. Its 5.1B dense footprint makes hardware fit the main deciding factor.5.1B DenseMMLU Pro: 60% | AIME 2026: 37.5%
Showing 1-8 of 24
1 / 3

Predicted inference speed

Selected model: Qwen 3.6-27B using Q4

0.4 tok/s
1.0x vs FP16 | 0% speed change

Memory footprint

Estimated allocation for local context

Required17.6 GB
Available: 0.0 GBUsage: 100%

Quantization Impact Matrix

Selected model: Qwen 3.6-27B. Click a row to update the active quantization and speed panels.

FormatFile SizeRequired MemoryPredicted SpeedQualityStatus
Q414.9 GB17.6 GB0.4 tok/s88%CompactMemory bound
Q518.4 GB21.1 GB0.4 tok/s93%BalancedMemory bound
Q622.1 GB24.9 GB0.4 tok/s96%High fidelityMemory bound
Q828.4 GB31.1 GB0.4 tok/s98%High fidelityMemory bound
FP1654.0 GB56.7 GB0.4 tok/s100%ReferenceMemory bound

Learning Center

Local inference guide

Understand hardware constraints, optimization strategies, and model fit.

What is VRAM?

Video RAM is the critical bottleneck for local LLMs. It stores model weights during inference; when a model cannot fit entirely into fast memory, layers spill into system RAM and tokens-per-second drops fast.

Example: 8B FP16~16GB required

Understanding Quantization

Reducing precision shrinks model weights, lowers memory bandwidth per token, and usually raises speed. The tradeoff is output fidelity: Q4 is fastest and smallest, while Q8 and FP16 preserve more detail but cost more memory.

Q4
28% size2.8x speed88% quality
Q5
34% size2.4x speed93% quality
Q6
41% size2.1x speed96% quality
Q8
53% size1.7x speed98% quality
FP16
100% size1.0x speed100% quality