Skip to content

How-ToRAM

How to Run a Local LLM on an 8GB Graphics Card: Models, Settings and Limits

An 8GB graphics card can run useful local AI models if you choose carefully. Ollama lists qwen3:8b at 5.2GB and gemma3:4b at 3.4GB, both of which fit, while 12B to 14B models spill into system RAM. Use ollama ps to confirm the model shows 100% GPU, keep the context window modest and set the K/V cache to q8_0 to roughly halve its memory use. Forge PC has not tested these models.

By , Editorial team

5 min read

How to Run a Local LLM on an 8GB Graphics Card: Models, Settings and Limits

You don't need a 24GB graphics card to run an AI chatbot on your own PC. Plenty of capable models fit on an 8GB card such as an RTX 4060, RTX 5060 or Radeon RX 7600, as long as you choose the model size carefully and keep an eye on how much memory it really uses.

Forge PC has not tested these models for this guide. It is based on Ollama's documentation and model library, and LM Studio's documentation. Model sizes are as listed on Ollama's library pages on 15 November 2026.

Why VRAM matters so much

A language model runs fastest when all of it sits in your graphics card's memory (VRAM). Ollama's documentation says that when a model fits entirely on a single GPU, Ollama loads it there, and that this typically gives the best performance because it reduces data transfer across the PCI bus.

If the model is too big, Ollama splits it between the GPU and system RAM, and the part running on the CPU slows everything down. So on an 8GB card, the goal is a model that fits with some room to spare.

Remember that Windows, your browser and other apps use some VRAM too. Our guide to checking VRAM usage shows how much is free before you start.

Step 1: Install Ollama or LM Studio

If you haven't installed a local AI app yet, follow our Ollama and LM Studio setup guide. Not sure which to choose? See Ollama vs LM Studio.

Check your graphics card is supported. Ollama's GPU documentation says it supports NVIDIA GPUs with compute capability 5.0 or newer and driver version 550 or newer, and it supports many AMD Radeon cards. It also offers Vulkan support for other GPUs. AMD owners can also read our AMD Radeon local AI guide.

Step 2: Pick a model that fits

These are download sizes from Ollama's library for popular model families. The size on disk is a rough guide to how much memory the model needs, but the context window (see below) adds more on top.

Model (Ollama tag)

Download size

Fits in 8GB VRAM?

qwen3:4b

2.5GB

Yes, with lots of room

gemma3:4b

3.4GB

Yes, with room to spare

qwen3:8b

5.2GB

Yes, leaves about 2.8GB

gemma4:e2b

4.6GB to 7.5GB depending on version

Smaller versions, yes

gemma4:e4b

6.6GB to 9.5GB depending on version

Smaller versions only, and tight

gemma4:12b

7.7GB to 8.0GB

Not fully; it will spill into system RAM

gemma3:12b

8.2GB

No; it will spill into system RAM

qwen3:14b

9.3GB

No; it will spill into system RAM

For most 8GB cards, qwen3:8b is a sensible starting point: it's a capable general-purpose model at 5.2GB, which leaves room for the context window. If you need image input, gemma3:4b handles both text and images in 3.4GB, according to Ollama's library.

To download and run a model in Ollama, open a terminal and type, for example:

ollama run qwen3:8b

Bigger models can still run on an 8GB card with part of the model in system RAM. They'll be slower, but they may be good enough for occasional use if you have 32GB of RAM. Our guide to RAM for local AI covers this.

Step 3: Check the model is fully on the GPU

After loading a model, open another terminal and run:

ollama ps

Ollama's documentation explains the PROCESSOR column:

  • 100% GPU means the model is entirely in VRAM. This is what you want.

  • 100% CPU means it's entirely in system memory.

  • A split such as 48%/52% CPU/GPU means part of the model didn't fit on the GPU.

If you see a split, try a smaller model or a shorter context window.

Step 4: Keep the context window sensible

The context window is how much text the model can consider at once, including your conversation so far. A longer context uses more memory. Ollama's documentation says it uses a 4,096-token context window by default. You can change it:

  • in an ollama run session, with /set parameter num_ctx 8192

  • for every model, with the OLLAMA_CONTEXT_LENGTH environment variable

On an 8GB card, raise the context only when you need it, and check ollama ps afterwards to make sure the model is still 100% on the GPU.

Step 5: Shrink the context cache

Ollama can store the context cache (the K/V cache) at lower precision to save memory. Its documentation says:

  • Flash Attention, which significantly reduces memory use as the context grows, is used automatically when your hardware and backend support it. You can force it on with OLLAMA_FLASH_ATTENTION=1.

  • With Flash Attention enabled, setting OLLAMA_KV_CACHE_TYPE=q8_0 uses about half the memory of the default f16 cache, with a very small loss in precision that usually makes no noticeable difference.

  • q4_0 uses about a quarter of the memory, but the loss in precision may be more noticeable at larger context sizes.

For an 8GB card, q8_0 is a sensible setting. Ollama notes this is a global option, so it applies to every model. For more tuning tips, see our guide to speeding up local LLMs in Ollama.

Using LM Studio instead

LM Studio's documentation explains that you can download models from its Discover tab and choose load configuration parameters when loading a model. The same principles apply: pick a model that fits in your VRAM, and don't set the context length higher than you need. Our LM Studio model guide explains what the different model files and quantisation levels mean, and our GGUF quantisation explainer goes into more detail.

When to upgrade

If you find yourself wanting 12B to 14B models or long contexts, more VRAM is the real fix. A 16GB card roughly doubles the model sizes you can run fully on the GPU. Our best GPU for local LLMs guide and how much VRAM you need cover the options.

Our verdict

An 8GB graphics card is enough to run useful local AI models. Start with a model of about 5GB, such as qwen3:8b, or a 4B model if you want more room, then use ollama ps to confirm it shows 100% GPU. Keep the context window modest, and set the K/V cache to q8_0 to save memory. Larger 12B to 14B models will spill into system RAM and slow down, so that's the point at which a 16GB card starts to make sense. Forge PC has not tested these models; sizes are from Ollama's library on 15 November 2026.

The bottom line

An 8GB graphics card is enough to run useful local AI models. Start with a model of about 5GB, such as qwen3:8b, or a 4B model if you want more room, then use `ollama ps` to confirm it shows 100% GPU. Keep the context window modest, and set the K/V cache to q8_0 to save memory. Larger 12B to 14B models will spill into system RAM and slow down, so that's the point at which a 16GB card starts to make sense. Forge PC has not tested these models; sizes are from Ollama's library on 15 November 2026.

Sources

  1. 01Ollama, FAQ
  2. 02Ollama, GPU support
  3. 03Ollama, qwen3 model library
  4. 04Ollama, gemma3 model library
  5. 05Ollama, gemma4 model library
  6. 06LM Studio, Getting started

About the author

Forge PC Staff

Editorial team

The shared byline for articles researched, written and edited by the Forge PC editorial team. How we source, check and correct our work is set out in our editorial policy.

More from Forge PC Staff

Read next

How to Enable AMD Fluid Motion Frames (AFMF 2.1) on a Radeon Graphics Card

How-To

How to Enable AMD Fluid Motion Frames (AFMF 2.1) on a Radeon Graphics Card

To enable AMD Fluid Motion Frames, install Adrenalin 25.3.1 or newer, then select the HYPR-RX profile (RX 7000 and newer) or turn on AMD Fluid Motion Frames under Gaming > Graphics, globally or per game. AFMF 2.1 works on RX 6000 cards and newer in DirectX 11, DirectX 12, Vulkan and OpenGL games, with VSync off; RX 6000 cards need exclusive full screen.

Forge PC Staff2 min

Two vs Four RAM Sticks for DDR5: Why 2x32GB Usually Beats 4x16GB

Versus

Two vs Four RAM Sticks for DDR5: Why 2x32GB Usually Beats 4x16GB

Two DDR5 sticks are better for most builds. AMD lists DDR5-5600 as the AM5 maximum with two modules but DDR5-3600 with four, and XMP or EXPO speeds above that are overclocking and not guaranteed. Buy 2x16GB or 2x32GB for 32GB or 64GB, and use four sticks, ideally a matched kit, only if you need 128GB or more.

Forge PC Staff3 min

AMD's FSR 4.1.2 Unreal Engine 5 Plugin Brings ML Upscaling to RX 6000 Cards

News

AMD's FSR 4.1.2 Unreal Engine 5 Plugin Brings ML Upscaling to RX 6000 Cards

AMD's FSR Unreal Engine 5 plugin now uses FSR Upscaling 4.1.2, bringing ML-based upscaling to Radeon RX 6000 cards and select Ryzen integrated graphics as well as RX 7000 and RX 9000, in UE 5.3 to 5.8 games that adopt it. It's DirectX 12 only for now and adds Frame Generation 4.0.2 with driver-assisted frame pacing on RX 9000 cards.

Forge PC Staff2 min

NVIDIA's DGX Spark HPC Developer Kit Links Four Sparks Into a 512GB Desk-Side Cluster

News

NVIDIA's DGX Spark HPC Developer Kit Links Four Sparks Into a 512GB Desk-Side Cluster

NVIDIA's DGX Spark HPC Developer Kit, announced at SC26 on 16 November 2026, links four DGX Spark systems in a ring over ConnectX-7 with no network switch, for 512GB of combined memory and 20TB of NVMe storage. NVIDIA says it can spread models of up to 700B parameters across the nodes with NVFP4. No UK kit price yet; Scan sells two-Spark bundles at about £12,845.

Forge PC Staff3 min