Skip to content

NewsGPU

vLLM 0.34.0 Released: Bigger KV Cache by Default, Model Runner V1 Removed and ROCm 10 Only

vLLM 0.34.0, released on 16 November 2026, sizes its KV cache from actually free GPU memory by default, adds ten new models and quotes big speed-ups on data-centre GPUs. It also removes Model Runner V1, raises the default block size to 64, drops ROCm 7.2 builds and needs huggingface_hub 2.0, so check your setup before upgrading.

By , Editorial team

3 min read

vLLM 0.34.0 Released: Bigger KV Cache by Default, Model Runner V1 Removed and ROCm 10 Only

vLLM, the open-source inference server used to run large language models on NVIDIA and AMD GPUs, released version 0.34.0 on 16 November 2026. The project's release notes describe it as 1,061 commits from 412 contributors, and the headline changes are a larger KV cache by default, ten new models and the removal of the old Model Runner V1. There are also several breaking changes, so if you run vLLM at home or on a workstation, read the upgrade notes before you update. This summary is based on the official GitHub release notes and PyPI listing. Forge PC hasn't tested vLLM 0.34.0, and the speed-ups quoted below are the project's own figures on data-centre hardware.

What's new

Extensible KV cache, on by default. On CUDA and ROCm, vLLM now sizes the KV cache (the memory that holds the conversation context) from the GPU memory that is actually free after warm-up, instead of from an estimate. The release notes say most models get a larger cache and more throughput as a result, and that "in most cases you should now remove gpu_memory_utilization from your configs". There's a trade-off: the project says per-token decode latency can rise on some models at high concurrency. You can opt out with --gpu-memory-utilization 0.9 or --no-enable-extensible-kv-cache. The notes also say an explicit gpu_memory_utilization setting is the way to share a GPU with other processes, which matters if your vLLM box also drives a desktop or runs other AI apps.

Ten new models. The release lists ten new models, including Mistral Large 4, MiniMax-M3.1-Flash and NVIDIA's Nemotron-3.5-Transcribe speech model. Ricursive's Ripple-65B Preview is also listed. Some of these are far too large for a single consumer GPU: vLLM describes Mistral Large 4 as 1T total parameters with 52B active.

Speed-ups on data-centre GPUs. vLLM quotes gains of up to 1.95x decode on GB200 systems and 1.6x output throughput for one model. These figures are from Blackwell data-centre and AMD MI355X hardware, not GeForce or Radeon cards, so don't expect the same results at home. The full changelog does mention faster weight loading on integrated GPUs.

Breaking changes to check before you upgrade

The release notes have a "Before you upgrade" section. The changes most likely to affect home users are:

  • Model Runner V1 is removed. Model Runner V2 is now the only GPU runner. If you set VLLM_USE_V2_MODEL_RUNNER=0, remove it: vLLM says it now fails at startup.

  • Default block size is now 64, up from 16. Backends with a preferred block size still override it, and an explicit --block-size still wins.

  • ROCm 7.2 builds are gone. AMD users should use the ROCm 10.0 wheels and images; the -rocm72 images are no longer published. If you're on Windows with a Radeon card, AMD's latest Adrenalin driver adds ROCm 10.2 support (see our Adrenalin 26.11.2 news); check vLLM's install notes for which platforms its ROCm 10.0 builds cover.

  • New dependency minimums. huggingface_hub 2.0.0 or newer is required, along with newer tokenizers and timm versions.

  • Some quantisation paths were removed, including Marlin W4A8, which the notes say should be replaced with the new humming option.

How to install it

The release notes list pip install vllm for the default CUDA 13.0 build, a separate ROCm 10.0 wheel index, and Docker images such as vllm/vllm-openai:v0.34.0 (CUDA 13.0) and vllm/vllm-openai-rocm:v0.34.0. If you want to try it on a Windows gaming PC, our guide to setting up WSL2 for GPU local AI covers running Linux AI tools under Windows.

Should you use vLLM at home?

vLLM is built for serving many requests at once, which is why so many of its changes focus on throughput and big multi-GPU systems. For a single user chatting with one model, tools such as Ollama or LM Studio are simpler; our vLLM vs Ollama comparison explains when each makes sense. If you do run vLLM, the larger default KV cache is relevant for long contexts: see our explainer on context length and VRAM, and how much VRAM you need for local models.

Our verdict

vLLM 0.34.0 is a significant release: the extensible KV cache should give most setups more context capacity without manual tuning, and ten new models are supported. But it's also a breaking upgrade. Remove old Model Runner V1 settings, update huggingface_hub, check your block size and, on AMD, move to the ROCm 10.0 builds. If your current setup works and you share the GPU with other apps, test the new memory behaviour before relying on it, or keep an explicit gpu_memory_utilization value.

The bottom line

vLLM 0.34.0 is a significant release: the extensible KV cache should give most setups more context capacity without manual tuning, and ten new models are supported. But it's also a breaking upgrade. Remove old Model Runner V1 settings, update huggingface_hub, check your block size and, on AMD, move to the ROCm 10.0 builds. If your current setup works and you share the GPU with other apps, test the new memory behaviour before relying on it, or keep an explicit gpu_memory_utilization value.

Sources

  1. 01vLLM project, v0.34.0 release notes on GitHub (16 November 2026)
  2. 02PyPI, vllm 0.34.0 (released 16 November 2026)

About the author

Forge PC Staff

Editorial team

The shared byline for articles researched, written and edited by the Forge PC editorial team. How we source, check and correct our work is set out in our editorial policy.

More from Forge PC Staff

Read next

How to Check Your Wi-Fi Signal Strength in Windows 11

How-To

How to Check Your Wi-Fi Signal Strength in Windows 11

Check Wi-Fi signal strength in Windows 11 from the taskbar bars, Settings > Network & internet, or exactly with netsh wlan show interfaces (the Signal percentage). Windows Central treats 80-100% as reliable and below 60% as weak. netsh wlan show wlanreport summarises recent wireless sessions.

Forge PC Staff2 min

How to Turn Thinking On or Off in Ollama (DeepSeek-R1, Qwen 3, gpt-oss)

How-To

How to Turn Thinking On or Off in Ollama (DeepSeek-R1, Qwen 3, gpt-oss)

Ollama lets you control reasoning models' thinking: --think and --think=false on the command line, /set think and /set nothink in chats, --hidethinking for scripts and a think field in the API. Check /api/show first, as models such as gpt-oss use low, medium and high levels instead.

Forge PC Staff2 min

How to Use Embedding Models in Ollama for Local Semantic Search and RAG

How-To

How to Use Embedding Models in Ollama for Local Semantic Search and RAG

Embedding models turn text into vectors for semantic search and RAG. Pull one such as embeddinggemma in Ollama, generate vectors with ollama run or the /api/embed endpoint, store them, then embed questions with the same model and compare using cosine similarity, all locally.

Forge PC Staff3 min

How to Fix Windows Search Not Working in Windows 11

How-To

How to Fix Windows Search Not Working in Windows 11

If Windows 11 search stops finding files or apps, follow Microsoft's fixes in order: end SearchHost.exe in Task Manager, install updates, run the Search and Indexing troubleshooter, check Classic or Enhanced indexed locations, then rebuild the index and allow up to 24 hours.

Forge PC Staff3 min