Best Local LLMs You Can Run on a Laptop

Something meaningful changed when capable language models became small enough to run on consumer hardware. The ability to run a genuinely useful AI model entirely on your own machine, with no API calls, no subscription fees, no data leaving your device, and no internet connection required, moved from research curiosity to practical reality for anyone with a reasonably modern laptop.

The reasons people run models locally vary. Privacy matters for some, particularly anyone working with sensitive documents, client data, or proprietary code they do not want passing through a third-party API. Cost matters for others, since local inference has zero marginal cost per query once the model is downloaded. Offline capability matters for developers building applications that need to work without connectivity. And for a growing number of people, the simple satisfaction of owning and controlling the tools they use is reason enough.

This guide covers the best local LLMs available today for laptop use, what hardware you actually need, how quantization makes larger models fit on smaller hardware, and which tools make the whole thing practical without requiring deep technical knowledge.

What Running Locally Actually Means

Running a language model locally means the model weights are stored on your machine and all inference computation happens on your hardware. No query leaves your device. No external server is involved. The model runs in your laptop’s memory and generates responses using your CPU or GPU.

The practical constraint is memory. Language model weights are large files. A 7 billion parameter model in full 32-bit precision requires roughly 28 gigabytes of memory. Most laptops do not have 28 gigabytes of RAM available for a single application. Quantization solves this by compressing the model weights to lower precision, reducing a 7 billion parameter model to 4 to 8 gigabytes depending on the quantization level, at a modest cost to output quality.

The distinction between RAM and VRAM matters here. Dedicated GPU inference is significantly faster than CPU inference, but most laptops either have no discrete GPU or have a GPU with limited VRAM. Apple Silicon Macs are an important exception because their unified memory architecture means the GPU and CPU share the same memory pool, allowing the GPU to use all available RAM for model inference. A MacBook Pro with 32 gigabytes of unified memory can run remarkably large models with GPU acceleration.

For Windows and Linux laptops with discrete NVIDIA GPUs, the GPU VRAM is the binding constraint. For laptops without discrete GPUs, models run on CPU, which works but is noticeably slower.

Hardware Tiers and What They Support

Entry level: 8GB RAM, no discrete GPU

CPU-only inference on models up to 3 to 4 billion parameters in 4-bit quantization. Expect one to three tokens per second generation speed. Responses take noticeably longer than cloud-hosted models but are usable for non-interactive tasks like document analysis, code review, and batch processing where speed is less critical. Suitable models include Phi-3 Mini, Gemma 2 2B, and Llama 3.2 3B.

Mid range: 16GB RAM, or 8GB VRAM discrete GPU

CPU inference on 7 billion parameter models in 4-bit quantization runs comfortably. With an 8GB VRAM discrete GPU, 7 billion parameter models run with GPU acceleration at three to eight tokens per second, which is fast enough for interactive use. Apple Silicon Macs with 16GB unified memory fall in this tier and perform significantly better than equivalent RAM on Windows laptops due to unified memory GPU access. Suitable models include Mistral 7B, Llama 3.1 8B, and Gemma 2 9B.

Capable: 32GB RAM or Apple Silicon with 32GB unified memory

The most practically useful tier for laptop use. CPU inference handles 13 billion parameter models in 4-bit quantization. Apple Silicon at this tier runs 13 billion parameter models with full GPU acceleration at speeds that feel genuinely responsive, typically six to fifteen tokens per second. Models at this size produce noticeably higher quality outputs than 7 billion parameter alternatives on complex reasoning, coding, and analytical tasks. Suitable models include Llama 3.1 13B, Mistral Nemo 12B, and Qwen 2.5 14B.

High end: 64GB RAM or Apple Silicon with 64GB unified memory

Models up to 30 to 34 billion parameters become accessible in 4-bit quantization. At this tier the gap between local model quality and frontier cloud models narrows significantly for many practical tasks. Suitable models include Llama 3.3 70B in very aggressive quantization, Qwen 2.5 32B, and Yi 34B.

The Best Models to Run Locally

Phi-3 Mini and Phi-3.5 Mini (Microsoft)

Phi-3 Mini is a 3.8 billion parameter model that punches significantly above its weight class on reasoning and coding tasks. Microsoft trained it on carefully curated high-quality data rather than raw internet scale, which produces a model that performs closer to 7 billion parameter models on structured tasks despite being half the size.

Phi-3 Mini fits comfortably in 2 to 3 gigabytes of memory in 4-bit quantization, making it the best choice for low-memory hardware and the fastest option for CPU-only inference. If you have an entry-level laptop and want a model that is genuinely useful rather than just technically running, Phi-3 Mini is the practical starting point. Phi-3.5 Mini is the updated version with stronger multilingual capability and improved instruction following.

Best for: low-memory hardware, fast responses on CPU, quick coding assistance, and users trying local models for the first time.

Llama 3.2 3B and Llama 3.1 8B (Meta)

Meta’s Llama 3 family represents the strongest open weights models available for local use across most capability benchmarks. The 3.2 3B model is competitive with Phi-3 Mini and offers a good balance of speed and quality for entry-level hardware. The 8B model is the most widely used local model in the community, with an enormous ecosystem of fine-tunes, tools, and documentation built around it.

Llama 3.1 8B in 4-bit quantization requires around 5 gigabytes of memory, runs on most 16 gigabyte laptops, and produces outputs that handle the majority of everyday tasks well, including code generation, summarization, question answering, and structured data extraction. The instruction-tuned variant follows directions reliably, which matters significantly for practical use.

Best for: general-purpose use, the widest ecosystem of tools and fine-tunes, reliable instruction following across a broad range of tasks.

Mistral 7B and Mistral Nemo 12B (Mistral AI)

Mistral 7B was the model that demonstrated convincingly that 7 billion parameters could produce genuinely high-quality outputs when architecture and training were done well. It remains highly competitive with newer models of similar size and has a particularly strong reputation for code generation and structured output.

Mistral Nemo 12B is the more recent release and represents a substantial step up in quality from the 7B model while still fitting in approximately 7 gigabytes in 4-bit quantization. It was trained with a 128,000 token context window, which is significantly larger than most models at this size and makes it useful for tasks involving long documents. For users with 16 gigabytes or more of RAM, Mistral Nemo offers some of the best quality per gigabyte available.

Best for: coding tasks, structured output generation, long document analysis, and users who have evaluated multiple models and found 7 billion parameter models slightly lacking.

Gemma 2 2B and Gemma 2 9B (Google)

Google’s Gemma 2 family is noteworthy for quality that exceeds expectations at its parameter counts. The 2B model is competitive with many 7 billion parameter models on instruction following and reasoning tasks, making it an exceptional choice for low-memory hardware. The 9B model is among the strongest performers in its size class.

Gemma 2’s training approach using knowledge distillation from larger models produces outputs with notably coherent reasoning and strong performance on analytical tasks. The 9B model in 4-bit quantization requires approximately 6 gigabytes and is one of the best options for users with 16 gigabytes of RAM who want the highest quality output available at that memory tier.

Best for: analytical tasks, instruction following on constrained hardware, users who want Google’s training quality in a locally runnable package.

Qwen 2.5 7B, 14B, and 32B (Alibaba)

The Qwen 2.5 family deserves more attention in Western communities than it typically receives. The 14B and 32B models in particular are exceptionally strong on coding tasks, mathematical reasoning, and multilingual capability, consistently appearing near the top of open model benchmarks at their respective parameter counts.

Qwen 2.5 Coder, a code-specialized variant, is arguably the best locally runnable model for software development tasks. The 7B coder variant fits on 16 gigabyte hardware and produces code quality that competes with much larger general-purpose models. For developers using local models primarily for coding assistance, Qwen 2.5 Coder is worth prioritizing over the general-purpose alternatives.

Best for: coding tasks where Qwen 2.5 Coder is the specific variant, mathematical reasoning, multilingual use cases, and users with 32 gigabyte or larger memory configurations.

Tools for Running Models Locally

Ollama

Ollama is the most practical tool for getting local models running quickly. It handles model downloading, quantization selection, and serving a local API compatible with the OpenAI API format. Installation is a single command and running a model is as simple as ollama run llama3.1. The local API means any application that supports OpenAI-compatible endpoints works with Ollama immediately.

bash

# Install Ollama (macOS/Linux)
curl -fsSL https://ollama.ai/install.sh | sh

# Pull and run a model
ollama run phi3

# Run with a specific quantization
ollama run llama3.1:8b-instruct-q4_0

# Use the API
curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1",
  "prompt": "Explain gradient descent simply"
}'

LM Studio

LM Studio provides a graphical interface for downloading, managing, and running local models. It is the easiest entry point for users who prefer not to use the command line. The built-in chat interface is polished and the model browser makes discovering and downloading quantized models from Hugging Face straightforward. LM Studio also serves a local API, making it compatible with the same ecosystem of tools as Ollama.

llama.cpp

llama.cpp is the underlying inference engine that most local model tools build on. Running it directly gives maximum control over quantization options, context length, hardware acceleration settings, and performance tuning. For users who want to understand exactly what is happening or who need configuration options that higher-level tools do not expose, llama.cpp is the authoritative option.

Understanding Quantization

Quantization reduces the numerical precision of model weights to shrink file size and memory requirements. The tradeoff is a small reduction in output quality that varies by quantization level.

The naming convention across most tools uses Q followed by a number indicating bits per weight. Q8 uses 8 bits per weight and is very close to full precision quality at roughly half the memory of float16. Q4 uses 4 bits and is the practical sweet spot for most use cases, retaining strong quality while halving memory requirements again. Q2 and Q3 are aggressive quantizations that fit larger models in very limited memory but produce noticeably lower quality outputs.

For most users, Q4 is the right choice. The quality difference from Q8 is small enough to be imperceptible for everyday tasks. The memory savings relative to Q8 are significant enough to matter when fitting models into laptop RAM.

Local LLM Cheat Sheet

Model	Size (Q4)	Min RAM	Best For
Phi-3 Mini	2.3GB	8GB	Low memory, fast CPU inference
Llama 3.2 3B	2.0GB	8GB	General use, entry hardware
Gemma 2 2B	1.6GB	8GB	Strong quality at tiny size
Mistral 7B	4.1GB	8GB	Coding, structured output
Llama 3.1 8B	4.9GB	16GB	General use, wide ecosystem
Gemma 2 9B	5.5GB	16GB	Analytical tasks
Mistral Nemo 12B	7.1GB	16GB	Long documents, high quality
Qwen 2.5 14B	8.9GB	16GB	Coding, multilingual
Qwen 2.5 32B	19.8GB	32GB	Near-frontier quality locally

FAQs