Skip to content

How-ToGPU

How to Run llama.cpp on Windows: The Llama App, Winget and llama-server

The easiest way to run llama.cpp on Windows is the free Llama app from the llama.cpp team and Hugging Face. Alternatively install it with 'winget install llama.cpp' or download a CUDA, ROCm, Vulkan or SYCL build from GitHub, then run llama-server -hf with a GGUF model and open the built-in chat at 127.0.0.1:9931. Add --api-key before sharing it on a network.

By , Editorial team

4 min read

How to Run llama.cpp on Windows: The Llama App, Winget and llama-server

llama.cpp is the open-source engine that runs large language models on ordinary PCs, and it sits underneath many popular local AI apps. It reached version 1.0 on 13 November 2026, and its developers now offer several easy ways to install it on Windows. Running it directly gives you the newest model support first, every setting, and a built-in chat page and OpenAI-compatible server.

Forge PC hasn't benchmarked llama.cpp for this guide. The steps are based on the official llama.cpp README, installation and server documentation, the llama.cpp release downloads on GitHub, and the Llama app's website, as checked on 17 November 2026.

What you need

Option 1: the Llama app (easiest)

The llama.cpp team and Hugging Face now offer Llama, a small app for Windows 11 and macOS at llama.app. On Windows it sits in the system tray. Its website says it checks your PC, recommends models that fit, picks settings such as quantisation and context size automatically, and lets you chat in your browser or connect other apps to a local API at localhost:9931/v1. It's free and open source. If you just want llama.cpp working with no command line, start here.

Option 2: winget

llama.cpp's installation guide lists winget for Windows. Open Terminal and run:

winget install llama.cpp

The guide says the winget package is updated automatically with new llama.cpp releases.

Option 3: a pre-built download for your GPU

For full control over which GPU back end you use, download a zip from the llama.cpp releases page on GitHub. The Windows x64 builds on 17 November 2026 included:

Your hardware

Build to download

NVIDIA GeForce RTX

win-cuda-13.5-x64 or win-cuda-12.4-x64 (plus the matching cudart zip)

AMD Radeon

win-rocm or win-vulkan

Intel Arc

win-sycl, win-vulkan or win-openvino

Any GPU / no GPU

win-vulkan or win-cpu

Extract the zip to a folder such as C:\llama, then open Terminal in that folder. For AMD and Intel cards, see our guides to local AI on Radeon and local AI on Intel Arc.

Load a model and start chatting

The README's quick start downloads and runs a model straight from Hugging Face. With the pre-built Windows zip, the server is llama-server.exe. For example:

llama-server -hf ggml-org/gemma-3-4b-it-GGUF

The server documentation says -hf takes a Hugging Face repository with an optional quantisation tag (for example :Q4_K_M), and defaults to Q4_K_M when there is one. Once it's running, open http://127.0.0.1:9931 in your browser to use Llama Chat, the built-in chat page. To use a model file you've already downloaded, use -m with its path instead.

Useful settings

These are documented options for llama-server:

Option

What it does

-c 16384

Sets the context size. The default, 0, uses the model's own value, which can use a lot of memory.

-ngl auto

How many layers go into VRAM. The default is auto; use all to force everything onto the GPU.

--port 9931

The port; 9931 is the default.

--host 127.0.0.1

Which address to listen on; the default only accepts connections from your own PC.

--api-key yourkey

Requires a key for every request.

If a model runs slowly, the usual cause is that it doesn't fit in VRAM and part of it is running from system memory. Try a smaller model or quantisation, or a shorter context. Our guide to speeding up local LLMs explains the same trade-offs in Ollama.

Connect other apps

llama-server provides OpenAI-compatible chat completions, responses and embeddings routes, plus an Anthropic Messages-compatible route, so many apps and coding tools can use it. Point them at http://localhost:9931/v1. The setup is similar to using Ollama's OpenAI API.

Stay safe on your network

By default, llama-server only listens on 127.0.0.1, so other devices can't reach it. If you change --host to share it on your network, set --api-key as well, and don't expose it to the internet. The 1.0 release notes include a change making server-side tools on a non-loopback address an explicit opt-in.

Our verdict

The easiest way to run llama.cpp on Windows is the Llama app, which picks models and settings for you. Use winget if you want command-line tools that stay updated, or a pre-built zip if you want to choose the CUDA, ROCm, Vulkan or SYCL build for your GPU. Then run llama-server -hf with a GGUF model, open the built-in chat at port 9931, and keep the server on 127.0.0.1 unless you add an API key. If you prefer a simpler manager, see llama.cpp vs Ollama. Forge PC hasn't benchmarked llama.cpp for this guide.

The bottom line

The easiest way to run llama.cpp on Windows is the Llama app, which picks models and settings for you. Use winget if you want command-line tools that stay updated, or a pre-built zip if you want to choose the CUDA, ROCm, Vulkan or SYCL build for your GPU. Then run `llama-server -hf` with a GGUF model, open the built-in chat at port 9931, and keep the server on 127.0.0.1 unless you add an API key. If you prefer a simpler manager, see llama.cpp vs Ollama. Forge PC hasn't benchmarked llama.cpp for this guide.

Sources

  1. 01llama.cpp (GitHub), README
  2. 02llama.cpp (GitHub), Install pre-built version of llama.cpp
  3. 03llama.cpp (GitHub), LLaMA.cpp HTTP Server documentation
  4. 04llama.cpp (GitHub), Releases
  5. 05Hugging Face, llama.cpp v1.0.0: A stronger foundation for the local AI ecosystem
  6. 06Llama, Your AI, on your computer

About the author

Forge PC Staff

Editorial team

The shared byline for articles researched, written and edited by the Forge PC editorial team. How we source, check and correct our work is set out in our editorial policy.

More from Forge PC Staff

Read next

How to Enable AMD Fluid Motion Frames (AFMF 2.1) on a Radeon Graphics Card

How-To

How to Enable AMD Fluid Motion Frames (AFMF 2.1) on a Radeon Graphics Card

To enable AMD Fluid Motion Frames, install Adrenalin 25.3.1 or newer, then select the HYPR-RX profile (RX 7000 and newer) or turn on AMD Fluid Motion Frames under Gaming > Graphics, globally or per game. AFMF 2.1 works on RX 6000 cards and newer in DirectX 11, DirectX 12, Vulkan and OpenGL games, with VSync off; RX 6000 cards need exclusive full screen.

Forge PC Staff2 min

How to Use System Restore in Windows 11: Turn It On, Create a Restore Point and Roll Back

How-To

How to Use System Restore in Windows 11: Turn It On, Create a Restore Point and Roll Back

Microsoft says System Protection isn't on by default. Search for Create a restore point, select your C: drive, choose Configure and turn it on, then create a restore point before installing drivers or tools. To roll back, run rstrui.exe, or use Troubleshoot > Advanced options > System Restore in Windows RE. Personal files aren't affected, but it isn't a backup.

Forge PC Staff3 min

How to Use Live Captions and Translation in Windows 11

How-To

How to Use Live Captions and Translation in Windows 11

Press Windows + Ctrl + L to turn on live captions in Windows 11 22H2 or later, download the language files, then dock or float the captions and pick a style. Captions are generated on the device and work offline. On a Copilot+ PC with 24H2 or later, live captions can translate from over 40 languages into English and 27 into Chinese (Simplified).

Forge PC Staff3 min

How to Manage Ollama Models: List, Remove and Move Them to Another Drive

How-To

How to Manage Ollama Models: List, Remove and Move Them to Another Drive

Run ollama ls to list your models, ollama ps to see what's loaded, ollama stop to unload one and ollama rm to delete it. On Windows, models are stored in C:\Users\%username%\.ollama\models; to move them, quit Ollama, set an OLLAMA_MODELS user environment variable to a folder on a bigger drive and restart Ollama.

Forge PC Staff3 min