Suggested MidGround LLM

[bdog4life]

Just looking for what everyone is using for the 'best' middle ground model.

Consider Low Power/offgrid/SHTF - what would be the 'best' per anyones experience

[chriscrosstalk]

Good question. It depends a lot on your hardware, but here's what I'd suggest based on community benchmark data:

• Budget hardware (no GPU, 8-16GB RAM): llama3.2:3b or qwen3:4b. Small enough to run on CPU at usable speeds. Don't expect long conversations, but solid for quick lookups and Q&A.
• Mid-range (AMD APU with Radeon 780M/890M, 32GB RAM): qwen3:8b is a great sweet spot. Fast enough on an iGPU to feel responsive, and smart enough for most tasks.
• If you have a dedicated GPU (RTX 3060+): qwen3:14b or even qwen3:30b if you have the VRAM for it.

For an off-grid/SHTF scenario where you want one model that covers the most ground on limited hardware, I'd go with qwen3:8b if your system can handle it, or llama3.2:3b if you're really constrained. The Qwen 3 models have been punching above their weight class.

You can check what kind of performance to expect on your hardware at the community benchmark leaderboard: https://benchmark.projectnomad.us

[dv0ps]

I'm taking a liking to cogito:3b‑v1‑preview‑llama‑q4_K_M as a daily driver compared to Llama 3.2 3B.
Qwen3.5‑4B is a good one too but on my old hardware I'm finding the larger cogito:8b‑v1‑preview‑llama‑q4_K_M is more performant than qwen3.5-4b and more helpful with answers taken from the documents I've uploaded to my NOMAD knowledge base.

For knowledge base queries on my added content, the Minstril-3:3b-instruct really pulls out good information and presents it well. I just wish thinking was disabled on it as it takes longer than I care for but when you get them they are very good.
llama3.2:1b is the only other LLM that seems to work well with my uploaded KB content

I don't know if a computer with any GPU qualifies for low power but they are hard to live without as even my old NVIDIA GeForce GTX 1060 3GB makes a big difference in processing AI prompts compared to doing it with CPUs only.

[CaptRudolf]

I've been thinking about this as well. Is there any hardware out there that allows one to disable the GPU for power savings, or some other means of power consumption reduction? The goal being hardware that has a 'low power' mode (when power is in short supply) and a 'high power' mode (when power is readily available).

I'd love to build a 'full-power' version, but if running on a smaller portable power station, it would be nice to have the option to reduce the load.

[gidkid]

easiest way run a raspberry pi computer for a low power desktop.

[CaptRudolf]

Just looking for what everyone is using for the 'best' middle ground model.

Consider Low Power/offgrid/SHTF - what would be the 'best' per anyones experience

This looks like a good fit: https://www.microcenter.com/product/703980/gmktec-nucbox-k12-mini-pc

[GeorgeWalker]

Just looking for what everyone is using for the 'best' middle ground model.
Consider Low Power/offgrid/SHTF - what would be the 'best' per anyones experience

This looks like a good fit: https://www.microcenter.com/product/703980/gmktec-nucbox-k12-mini-pc

That's very similar to the old laptop I am using - it works great with Nomad. 32 GB of RAM means you can run lots of Docker containers, and the Ryzen is a good CPU. Roughly 50 watts power at idle.

[Bradley-Radomski]

Truly considering getting an MS-02 or one of their other models. Have an MS-01 and it's epic without a gpu.

[fotomensch]

i took some time and created a testscenario with claude for several of the available LLMs in project nomad.

there where 3 tasks: a coding task, a reasoning task and a mixed task of knowledge and creativity. all models recieved the same prompt as input.
calude compared all the outputs and ranked the models.
so make of that what you want. a LLM rating other LLMs.

for comparisson i also included calude it self, perplexity.ai and chatgpt in the test but without telling claude what the models where to minimize bias.

test system:
RTX 5070ti
Ryzen 9 7950X 16-Core Processor
64GB RAM

attached you will find the claude-created PDF with the full results

[llm_benchmark_report_en.pdf](https://github.com/user-attachments/files/26335644/llm_benchmark_report_en.pdf)

[CaptRudolf]

How do we think this would work (possible to upgrade to 64GB DDR4 and 1 TB SSD)?