Which Local LLM Is Most Useful With 36 GB VRAM and 128 GB RAM?

Qwen has less than half of GLM's total parameters and only a third as many active parameters. I expected GLM's extra capacity to help. Instead, Qwen solved 158 HumanEval problems while GLM solved 142.

Parameter counts are difficult to compare across mixture-of-experts architectures. For each token, the model uses only a selection of its experts. Their design, routing, training, and tuning can matter more than the number of active parameters.

There is also a much simpler explanation in the files I tested. Qwen’s UD-IQ4_XS model takes 87.25 GiB. GLM fits into 86.69 GiB only because it uses the much more aggressive UD-IQ1_S quantization. Both deployments demand almost the same amount of memory, but Qwen keeps more information per parameter. I count that difference as part of the result because choosing a quantization that fits is part of choosing a local model.

The paired results make the difference easier to see. Both models solved 140 HumanEval problems. Qwen alone solved 18, GLM alone solved two, and both missed four. GLM also averaged 258 output tokens per answer, compared with 81 for Qwen. Six GLM answers reached the output limit, and every one failed. Concise, well-formed output is part of being useful, even when the underlying problem is excessive generation rather than reasoning.

On this machine, this Qwen configuration is the better system. The test does not tell us whether the unquantized Qwen model is generally smarter than unquantized GLM. The sampled MMLU-Pro scores were also much closer: 176/210 for Qwen and 168/210 for GLM, with overlapping confidence intervals.

Qwen was the practical winner: the highest HumanEval score, an MMLU-Pro score within two questions of the best, 18.8 tok/s during MMLU-Pro, and the shortest total evaluation time. GLM proved that a 321B-parameter MoE model can still clear 10 tok/s on this split CPU/GPU setup, although its coding score and runtime were weaker. At UD-Q3_K_XL it became the strongest reasoner in the test, but it needs 50 GiB more memory than Qwen and runs at 10.18 tok/s, right at my target. DeepSeek remained capable, but its measured 9.53 tok/s narrowly missed my speed target.

I would use these models for drafting code, explaining an unfamiliar system, generating tests, and working through a contained problem. HumanEval cannot tell me whether they could maintain a real codebase on their own. It does tell me that I can get useful work from a local model without giving up privacy or a large context window.