Skip to content

How-ToRAM

How to Make Local AI Models Run Faster in Ollama: 7 Settings That Matter

The biggest speed-up for local AI in Ollama is keeping the whole model on your GPU: run ollama ps and aim for 100% GPU. Ollama's documented settings can help, including a smaller context window, flash attention, a quantised KV cache (q8_0 roughly halves its memory use), fewer parallel requests and keep_alive. Forge PC has not benchmarked these settings.

By , Editorial team

4 min read

How to Make Local AI Models Run Faster in Ollama: 7 Settings That Matter

A local AI model that answers in a blur on one PC can crawl on another with a similar graphics card. Usually the difference is not raw GPU power but memory: whether the whole model, plus its working memory for the conversation, fits in your card's VRAM. Once part of it spills into system RAM, speed drops sharply.

Forge PC has not benchmarked these settings, so we do not quote speed-up figures. The steps below follow Ollama's official documentation (its FAQ and context length guide). If you are new to Ollama, start with our guide to running AI models locally with Ollama or LM Studio.

1. Check whether the model fits on your GPU

Before changing anything, load your model and run:

ollama ps

Ollama's documentation explains the PROCESSOR column. 100% GPU means the model is loaded entirely into the graphics card. A split such as 48%/52% CPU/GPU means part of it is in system memory. Ollama's context length guide says that for best performance you should "avoid offloading the model to CPU" and use this command to verify the split. If you see anything other than 100% GPU, the rest of this guide is about getting there. To keep an eye on GPU memory while you work, see our HWiNFO and Afterburner monitoring guide.

2. Pick a model and quantisation that fit

The single biggest lever is the model file itself. A smaller model, or a more heavily quantised version of the same model, needs less memory. Our quantisation explainer explains the quality trade-off of formats such as Q4 and Q8, and our local AI GPU guide shows which sizes suit 8GB, 12GB, 16GB and 24GB cards.

3. Set the context length deliberately

Context length is how many tokens the model can hold in memory for a conversation. Ollama's guide says that "setting a larger context length will increase the amount of memory required to run a model", and its defaults depend on your VRAM:

Your VRAM

Ollama's default context

Under 24GB

4K tokens

24GB to 48GB

32K tokens

48GB or more

256K tokens

Source: Ollama context length documentation.

That default is a sensible starting point for chat. Ollama does recommend at least 64,000 tokens for tasks like web search, agents and coding tools, but on a 12GB or 16GB card that much context can push a model off the GPU. Raise it only as far as you need, and check ollama ps afterwards.

To change it, Ollama's documentation gives several options: the context slider in the Ollama app's settings, the OLLAMA_CONTEXT_LENGTH environment variable when serving, or, inside an ollama run session, /set parameter num_ctx 8192 (replace the number with the size you want). API users can pass num_ctx in the request options.

4. Make sure Flash Attention is on

Ollama's FAQ describes Flash Attention as a feature of most modern models that "can significantly reduce memory usage as the context size grows". Ollama now uses it automatically when your backend and GPU support it. You can force it on with the environment variable OLLAMA_FLASH_ATTENTION=1 (or off with 0) if you need to.

5. Quantise the KV cache

The KV cache is the model's working memory for your conversation, and it grows with context length. Ollama can store it at lower precision when Flash Attention is enabled, using the OLLAMA_KV_CACHE_TYPE environment variable:

Setting

Memory use

Ollama's description

f16 (default)

Full

High precision

q8_0

About half of f16

Very small loss in precision, usually no noticeable impact; recommended if not using f16

q4_0

About a quarter of f16

Small to medium loss in precision, more noticeable at higher context sizes

Source: Ollama FAQ.

q8_0 is the safe first step. Ollama warns that this is a global setting for all models, and that models with a high GQA count, such as Qwen2, may be affected more, so test the quality on your own tasks.

6. Keep one model loaded and limit parallel requests

Ollama can load several models at once and handle parallel requests, but both cost memory. The FAQ explains that parallel requests multiply the context: a 2K context with four parallel requests becomes an 8K context with the extra memory that implies. When using the GPU, new models must fit completely in VRAM to load alongside others.

If you only chat with one model at a time, setting OLLAMA_NUM_PARALLEL=1 keeps memory use predictable. The keep_alive setting (or OLLAMA_KEEP_ALIVE) controls how long a model stays loaded after use, which avoids the delay of reloading it from disk every time.

7. Set environment variables properly on Windows

Ollama's FAQ says that on Windows it inherits your user and system environment variables. To change one:

  1. Quit Ollama from the taskbar.

  2. Open Settings and search for environment variables.

  3. Choose Edit environment variables for your account.

  4. Add or edit the variable (for example OLLAMA_KV_CACHE_TYPE with the value q8_0).

  5. Save, then start Ollama again.

If it is still slow

If a model you need simply will not fit, more VRAM is the real fix. Our RAM for local AI guide explains when system memory helps, and our guide to running a local LLM across two GPUs covers splitting a model between cards.

Our verdict

Most slow local AI comes down to memory. Run ollama ps first: if it does not say 100% GPU, choose a smaller model or quantisation, keep the context length no bigger than you need, make sure Flash Attention is on and try OLLAMA_KV_CACHE_TYPE=q8_0, which Ollama says roughly halves KV cache memory with very little quality loss. Limiting parallel requests and keeping a model loaded also help. Forge PC has not benchmarked these settings; every step follows Ollama's own documentation.

The bottom line

Most slow local AI comes down to memory. Run `ollama ps` first: if it does not say 100% GPU, choose a smaller model or quantisation, keep the context length no bigger than you need, make sure Flash Attention is on and try `OLLAMA_KV_CACHE_TYPE=q8_0`, which Ollama says roughly halves KV cache memory with very little quality loss. Limiting parallel requests and keeping a model loaded also help. Forge PC has not benchmarked these settings; every step follows Ollama's own documentation.

Sources

  1. 01Ollama, FAQ
  2. 02Ollama, Context length

About the author

Forge PC Staff

Editorial team

The shared byline for articles researched, written and edited by the Forge PC editorial team. How we source, check and correct our work is set out in our editorial policy.

More from Forge PC Staff

Read next

How to Enable AMD Fluid Motion Frames (AFMF 2.1) on a Radeon Graphics Card

How-To

How to Enable AMD Fluid Motion Frames (AFMF 2.1) on a Radeon Graphics Card

To enable AMD Fluid Motion Frames, install Adrenalin 25.3.1 or newer, then select the HYPR-RX profile (RX 7000 and newer) or turn on AMD Fluid Motion Frames under Gaming > Graphics, globally or per game. AFMF 2.1 works on RX 6000 cards and newer in DirectX 11, DirectX 12, Vulkan and OpenGL games, with VSync off; RX 6000 cards need exclusive full screen.

Forge PC Staff2 min

Two vs Four RAM Sticks for DDR5: Why 2x32GB Usually Beats 4x16GB

Versus

Two vs Four RAM Sticks for DDR5: Why 2x32GB Usually Beats 4x16GB

Two DDR5 sticks are better for most builds. AMD lists DDR5-5600 as the AM5 maximum with two modules but DDR5-3600 with four, and XMP or EXPO speeds above that are overclocking and not guaranteed. Buy 2x16GB or 2x32GB for 32GB or 64GB, and use four sticks, ideally a matched kit, only if you need 128GB or more.

Forge PC Staff3 min

AMD's FSR 4.1.2 Unreal Engine 5 Plugin Brings ML Upscaling to RX 6000 Cards

News

AMD's FSR 4.1.2 Unreal Engine 5 Plugin Brings ML Upscaling to RX 6000 Cards

AMD's FSR Unreal Engine 5 plugin now uses FSR Upscaling 4.1.2, bringing ML-based upscaling to Radeon RX 6000 cards and select Ryzen integrated graphics as well as RX 7000 and RX 9000, in UE 5.3 to 5.8 games that adopt it. It's DirectX 12 only for now and adds Frame Generation 4.0.2 with driver-assisted frame pacing on RX 9000 cards.

Forge PC Staff2 min

NVIDIA's DGX Spark HPC Developer Kit Links Four Sparks Into a 512GB Desk-Side Cluster

News

NVIDIA's DGX Spark HPC Developer Kit Links Four Sparks Into a 512GB Desk-Side Cluster

NVIDIA's DGX Spark HPC Developer Kit, announced at SC26 on 16 November 2026, links four DGX Spark systems in a ring over ConnectX-7 with no network switch, for 512GB of combined memory and 20TB of NVMe storage. NVIDIA says it can spread models of up to 700B parameters across the nodes with NVFP4. No UK kit price yet; Scan sells two-Spark bundles at about £12,845.

Forge PC Staff3 min