The local model deskCompare hardware

Best LLM to Run Locally

On an 8 GB phone, start with a small text model like Llama 3.2 1B. On a 16 GB laptop, try a quantized Qwen3.5-4B. Give it a task you know well before downloading anything larger.

Your hardware sets the starting point.

These are editorial starting ranges, not tested fit guarantees. Available memory matters more than the number printed on the box.

Choose a starting range, then check the exact model.
HardwareFirst choiceWatch
8 GB phoneApp recommendation; supported 1B to 3B-class text modelApp memory limit
16 GB laptopQuantized 3B to 8B-class modelOther apps and context
Apple SiliconSupported GGUF or MLX packageUnified memory use
NVIDIA GPUCompact quantized model firstVRAM and CUDA support
CPU-onlySmall quantized text modelYour task's response time

8 GB phone

Start with a small text model.

Try Llama 3.2 1B Instruct if your app offers a quantized copy. Give it a paragraph to summarize. If it misses a point and you have memory to spare, try the 3B Instruct variant on the same paragraph.

Phone memory is shared with the operating system. An app may have a lower memory limit than the device total. Keep the first conversation short and check that the model reloads without a connection.

Llama 3.2 model card

16 GB laptop

Try a compact model before a large download.

A quantized 3B to 8B-class model is a useful starting range. Qwen3.5-4B and Phi-4-mini are concrete options to investigate in a compatible runtime. Start with a modest context and leave memory for your browser and other work.

LM Studio recommends 16 GB or more of RAM. That is an application recommendation, not a guarantee for every model or context setting.

LM Studio system requirements

Apple Silicon

Choose by available unified memory.

On a Mac with 16 GB of unified memory, try a quantized Qwen3.5-4B with a short prompt. LM Studio supports GGUF through llama.cpp and MLX packages on Apple Silicon. Use the same model variant and prompt to compare the two.

A chip name alone does not tell you what fits. The operating system, GPU and applications share the same pool of unified memory.

MLX LM documentation

NVIDIA GPU

Budget VRAM for weights and context.

Check the runtime’s GPU compatibility, then start with a compact quantized model. A 4B-class model is a conservative first experiment. Increase size after you can see how much memory your actual prompt uses.

CPU offload can help a model fit, but it does not establish a useful response speed. Confirm that the runtime is using the intended backend.

Ollama GPU documentation

CPU-only

Keep the first task short and specific.

A GPU is not required by every local runtime. Start with a supported small quantized text model, such as Llama 3.2 1B, and a short rewriting or summarising task. Move up only if the wait and memory use suit your work.

llama.cpp supports CPU execution and mixed CPU/GPU inference. No single speed estimate applies to all processors, memory systems and model formats.

llama.cpp backends

Three different memory numbers

Download size
The files you store on disk. Quantization changes their size. Image-capable models may need an additional projector.
Working memory
The model's runtime footprint, including the conversation cache and temporary calculations.
Available memory
What remains after the operating system and other applications take their share. This is the budget you need to respect.

Longer conversations can use more memory. A model's advertised context window does not mean you can use the entire window on every device. See the download and format guide before treating a file-size figure as a RAM requirement.

Pick the runtime for the work

For everyday chat, we recommend Loci. It recommends a model for your device, taking its processor and available memory into account. You can start there without choosing a quantization yourself. For a local API or a Linux workflow, try Ollama. Choose LM Studio to manage model files on a desktop, or llama.cpp to set inference options from a script.

Check the model catalogue for named variants and licenses. A family name alone is not enough to select a file. Download a compatible model and any required runtime before disconnecting.

How we chose

We used official model cards and runtime documentation to identify cautious starting points. We did not benchmark these hardware/model combinations or certify an exact device fit. Test your own task and check the result before choosing a larger model.

Questions and answers

What is the best LLM to run locally on an 8 GB phone?

Start with the model your app recommends for that phone. For manual selection, a supported 1B to 3B-class quantized text model is a cautious starting point. The operating system limits usable app memory, so 8 GB of device RAM is not an 8 GB model budget.

Is 16 GB of RAM enough for a local LLM?

It can be enough for many compact quantized models. Start with a 3B to 8B-class model and a modest context. The model weights, context cache, application, operating system and other programs all need memory. This range is guidance, not a fit guarantee.

Do I need an NVIDIA GPU?

No. Local runtimes can use a CPU, Apple Silicon or a supported GPU. llama.cpp documents CPU, Metal and CUDA backends. Hardware support and speed depend on the runtime, model and configuration.

Should I use GGUF or MLX on a Mac?

Use a format your chosen app supports. GGUF works with llama.cpp-based runtimes, including Metal acceleration. MLX is designed for Apple Silicon. Compare compatible packages using your own task instead of assuming the format alone determines answer quality.

Why does a model need more memory than its download size?

The runtime needs space for the model, intermediate calculations and the conversation context cache. Long prompts and additional modalities can increase the working memory. A file-size figure is not a total RAM requirement.

Will a larger model always give a better answer?

No. Training, the task, quantization and the prompt all affect the result. Use a model that fits comfortably and check its answers. Moving up in size is useful only if the result improves enough for your work.