llama.cpp is the open-source engine that runs large language models on ordinary PCs, and it sits underneath many popular local AI apps. It reached version 1.0 on 13 November 2026, and its developers now offer several easy ways to install it on Windows. Running it directly gives you the newest model support first, every setting, and a built-in chat page and OpenAI-compatible server.
Forge PC hasn't benchmarked llama.cpp for this guide. The steps are based on the official llama.cpp README, installation and server documentation, the llama.cpp release downloads on GitHub, and the Llama app's website, as checked on 17 November 2026.
What you need
Option 1: the Llama app (easiest)
The llama.cpp team and Hugging Face now offer Llama, a small app for Windows 11 and macOS at llama.app. On Windows it sits in the system tray. Its website says it checks your PC, recommends models that fit, picks settings such as quantisation and context size automatically, and lets you chat in your browser or connect other apps to a local API at localhost:9931/v1. It's free and open source. If you just want llama.cpp working with no command line, start here.
Option 2: winget
llama.cpp's installation guide lists winget for Windows. Open Terminal and run:
winget install llama.cpp
The guide says the winget package is updated automatically with new llama.cpp releases.
Option 3: a pre-built download for your GPU
For full control over which GPU back end you use, download a zip from the llama.cpp releases page on GitHub. The Windows x64 builds on 17 November 2026 included:
Your hardware | Build to download |
|---|
NVIDIA GeForce RTX | win-cuda-13.5-x64 or win-cuda-12.4-x64 (plus the matching cudart zip)
|
AMD Radeon | win-rocm or win-vulkan
|
Intel Arc | win-sycl, win-vulkan or win-openvino
|
Any GPU / no GPU | win-vulkan or win-cpu
|
Extract the zip to a folder such as C:\llama, then open Terminal in that folder. For AMD and Intel cards, see our guides to local AI on Radeon and local AI on Intel Arc.
Load a model and start chatting
The README's quick start downloads and runs a model straight from Hugging Face. With the pre-built Windows zip, the server is llama-server.exe. For example:
llama-server -hf ggml-org/gemma-3-4b-it-GGUF
The server documentation says -hf takes a Hugging Face repository with an optional quantisation tag (for example :Q4_K_M), and defaults to Q4_K_M when there is one. Once it's running, open http://127.0.0.1:9931 in your browser to use Llama Chat, the built-in chat page. To use a model file you've already downloaded, use -m with its path instead.
Useful settings
These are documented options for llama-server:
Option | What it does |
|---|
-c 16384
| Sets the context size. The default, 0, uses the model's own value, which can use a lot of memory. |
-ngl auto
| How many layers go into VRAM. The default is auto; use all to force everything onto the GPU. |
--port 9931
| The port; 9931 is the default. |
--host 127.0.0.1
| Which address to listen on; the default only accepts connections from your own PC. |
--api-key yourkey
| Requires a key for every request. |
If a model runs slowly, the usual cause is that it doesn't fit in VRAM and part of it is running from system memory. Try a smaller model or quantisation, or a shorter context. Our guide to speeding up local LLMs explains the same trade-offs in Ollama.
Connect other apps
llama-server provides OpenAI-compatible chat completions, responses and embeddings routes, plus an Anthropic Messages-compatible route, so many apps and coding tools can use it. Point them at http://localhost:9931/v1. The setup is similar to using Ollama's OpenAI API.
Stay safe on your network
By default, llama-server only listens on 127.0.0.1, so other devices can't reach it. If you change --host to share it on your network, set --api-key as well, and don't expose it to the internet. The 1.0 release notes include a change making server-side tools on a non-loopback address an explicit opt-in.
Our verdict
The easiest way to run llama.cpp on Windows is the Llama app, which picks models and settings for you. Use winget if you want command-line tools that stay updated, or a pre-built zip if you want to choose the CUDA, ROCm, Vulkan or SYCL build for your GPU. Then run llama-server -hf with a GGUF model, open the built-in chat at port 9931, and keep the server on 127.0.0.1 unless you add an API key. If you prefer a simpler manager, see llama.cpp vs Ollama. Forge PC hasn't benchmarked llama.cpp for this guide.