You don't need a 24GB graphics card to run an AI chatbot on your own PC. Plenty of capable models fit on an 8GB card such as an RTX 4060, RTX 5060 or Radeon RX 7600, as long as you choose the model size carefully and keep an eye on how much memory it really uses.
Forge PC has not tested these models for this guide. It is based on Ollama's documentation and model library, and LM Studio's documentation. Model sizes are as listed on Ollama's library pages on 15 November 2026.
Why VRAM matters so much
A language model runs fastest when all of it sits in your graphics card's memory (VRAM). Ollama's documentation says that when a model fits entirely on a single GPU, Ollama loads it there, and that this typically gives the best performance because it reduces data transfer across the PCI bus.
If the model is too big, Ollama splits it between the GPU and system RAM, and the part running on the CPU slows everything down. So on an 8GB card, the goal is a model that fits with some room to spare.
Remember that Windows, your browser and other apps use some VRAM too. Our guide to checking VRAM usage shows how much is free before you start.
Step 1: Install Ollama or LM Studio
If you haven't installed a local AI app yet, follow our Ollama and LM Studio setup guide. Not sure which to choose? See Ollama vs LM Studio.
Check your graphics card is supported. Ollama's GPU documentation says it supports NVIDIA GPUs with compute capability 5.0 or newer and driver version 550 or newer, and it supports many AMD Radeon cards. It also offers Vulkan support for other GPUs. AMD owners can also read our AMD Radeon local AI guide.
Step 2: Pick a model that fits
These are download sizes from Ollama's library for popular model families. The size on disk is a rough guide to how much memory the model needs, but the context window (see below) adds more on top.
Model (Ollama tag) | Download size | Fits in 8GB VRAM? |
|---|
qwen3:4b | 2.5GB | Yes, with lots of room |
gemma3:4b | 3.4GB | Yes, with room to spare |
qwen3:8b | 5.2GB | Yes, leaves about 2.8GB |
gemma4:e2b | 4.6GB to 7.5GB depending on version | Smaller versions, yes |
gemma4:e4b | 6.6GB to 9.5GB depending on version | Smaller versions only, and tight |
gemma4:12b | 7.7GB to 8.0GB | Not fully; it will spill into system RAM |
gemma3:12b | 8.2GB | No; it will spill into system RAM |
qwen3:14b | 9.3GB | No; it will spill into system RAM |
For most 8GB cards, qwen3:8b is a sensible starting point: it's a capable general-purpose model at 5.2GB, which leaves room for the context window. If you need image input, gemma3:4b handles both text and images in 3.4GB, according to Ollama's library.
To download and run a model in Ollama, open a terminal and type, for example:
ollama run qwen3:8b
Bigger models can still run on an 8GB card with part of the model in system RAM. They'll be slower, but they may be good enough for occasional use if you have 32GB of RAM. Our guide to RAM for local AI covers this.
Step 3: Check the model is fully on the GPU
After loading a model, open another terminal and run:
ollama ps
Ollama's documentation explains the PROCESSOR column:
100% GPU means the model is entirely in VRAM. This is what you want.
100% CPU means it's entirely in system memory.
A split such as 48%/52% CPU/GPU means part of the model didn't fit on the GPU.
If you see a split, try a smaller model or a shorter context window.
Step 4: Keep the context window sensible
The context window is how much text the model can consider at once, including your conversation so far. A longer context uses more memory. Ollama's documentation says it uses a 4,096-token context window by default. You can change it:
in an ollama run session, with /set parameter num_ctx 8192
for every model, with the OLLAMA_CONTEXT_LENGTH environment variable
On an 8GB card, raise the context only when you need it, and check ollama ps afterwards to make sure the model is still 100% on the GPU.
Step 5: Shrink the context cache
Ollama can store the context cache (the K/V cache) at lower precision to save memory. Its documentation says:
Flash Attention, which significantly reduces memory use as the context grows, is used automatically when your hardware and backend support it. You can force it on with OLLAMA_FLASH_ATTENTION=1.
With Flash Attention enabled, setting OLLAMA_KV_CACHE_TYPE=q8_0 uses about half the memory of the default f16 cache, with a very small loss in precision that usually makes no noticeable difference.
q4_0 uses about a quarter of the memory, but the loss in precision may be more noticeable at larger context sizes.
For an 8GB card, q8_0 is a sensible setting. Ollama notes this is a global option, so it applies to every model. For more tuning tips, see our guide to speeding up local LLMs in Ollama.
Using LM Studio instead
LM Studio's documentation explains that you can download models from its Discover tab and choose load configuration parameters when loading a model. The same principles apply: pick a model that fits in your VRAM, and don't set the context length higher than you need. Our LM Studio model guide explains what the different model files and quantisation levels mean, and our GGUF quantisation explainer goes into more detail.
When to upgrade
If you find yourself wanting 12B to 14B models or long contexts, more VRAM is the real fix. A 16GB card roughly doubles the model sizes you can run fully on the GPU. Our best GPU for local LLMs guide and how much VRAM you need cover the options.
Our verdict
An 8GB graphics card is enough to run useful local AI models. Start with a model of about 5GB, such as qwen3:8b, or a 4B model if you want more room, then use ollama ps to confirm it shows 100% GPU. Keep the context window modest, and set the K/V cache to q8_0 to save memory. Larger 12B to 14B models will spill into system RAM and slow down, so that's the point at which a 16GB card starts to make sense. Forge PC has not tested these models; sizes are from Ollama's library on 15 November 2026.