vLLM, the open-source inference server used to run large language models on NVIDIA and AMD GPUs, released version 0.34.0 on 16 November 2026. The project's release notes describe it as 1,061 commits from 412 contributors, and the headline changes are a larger KV cache by default, ten new models and the removal of the old Model Runner V1. There are also several breaking changes, so if you run vLLM at home or on a workstation, read the upgrade notes before you update. This summary is based on the official GitHub release notes and PyPI listing. Forge PC hasn't tested vLLM 0.34.0, and the speed-ups quoted below are the project's own figures on data-centre hardware.
What's new
Extensible KV cache, on by default. On CUDA and ROCm, vLLM now sizes the KV cache (the memory that holds the conversation context) from the GPU memory that is actually free after warm-up, instead of from an estimate. The release notes say most models get a larger cache and more throughput as a result, and that "in most cases you should now remove gpu_memory_utilization from your configs". There's a trade-off: the project says per-token decode latency can rise on some models at high concurrency. You can opt out with --gpu-memory-utilization 0.9 or --no-enable-extensible-kv-cache. The notes also say an explicit gpu_memory_utilization setting is the way to share a GPU with other processes, which matters if your vLLM box also drives a desktop or runs other AI apps.
Ten new models. The release lists ten new models, including Mistral Large 4, MiniMax-M3.1-Flash and NVIDIA's Nemotron-3.5-Transcribe speech model. Ricursive's Ripple-65B Preview is also listed. Some of these are far too large for a single consumer GPU: vLLM describes Mistral Large 4 as 1T total parameters with 52B active.
Speed-ups on data-centre GPUs. vLLM quotes gains of up to 1.95x decode on GB200 systems and 1.6x output throughput for one model. These figures are from Blackwell data-centre and AMD MI355X hardware, not GeForce or Radeon cards, so don't expect the same results at home. The full changelog does mention faster weight loading on integrated GPUs.
Breaking changes to check before you upgrade
The release notes have a "Before you upgrade" section. The changes most likely to affect home users are:
Model Runner V1 is removed. Model Runner V2 is now the only GPU runner. If you set VLLM_USE_V2_MODEL_RUNNER=0, remove it: vLLM says it now fails at startup.
Default block size is now 64, up from 16. Backends with a preferred block size still override it, and an explicit --block-size still wins.
ROCm 7.2 builds are gone. AMD users should use the ROCm 10.0 wheels and images; the -rocm72 images are no longer published. If you're on Windows with a Radeon card, AMD's latest Adrenalin driver adds ROCm 10.2 support (see our Adrenalin 26.11.2 news); check vLLM's install notes for which platforms its ROCm 10.0 builds cover.
New dependency minimums. huggingface_hub 2.0.0 or newer is required, along with newer tokenizers and timm versions.
Some quantisation paths were removed, including Marlin W4A8, which the notes say should be replaced with the new humming option.
How to install it
The release notes list pip install vllm for the default CUDA 13.0 build, a separate ROCm 10.0 wheel index, and Docker images such as vllm/vllm-openai:v0.34.0 (CUDA 13.0) and vllm/vllm-openai-rocm:v0.34.0. If you want to try it on a Windows gaming PC, our guide to setting up WSL2 for GPU local AI covers running Linux AI tools under Windows.
Should you use vLLM at home?
vLLM is built for serving many requests at once, which is why so many of its changes focus on throughput and big multi-GPU systems. For a single user chatting with one model, tools such as Ollama or LM Studio are simpler; our vLLM vs Ollama comparison explains when each makes sense. If you do run vLLM, the larger default KV cache is relevant for long contexts: see our explainer on context length and VRAM, and how much VRAM you need for local models.
Our verdict
vLLM 0.34.0 is a significant release: the extensible KV cache should give most setups more context capacity without manual tuning, and ten new models are supported. But it's also a breaking upgrade. Remove old Model Runner V1 settings, update huggingface_hub, check your block size and, on AMD, move to the ROCm 10.0 builds. If your current setup works and you share the GPU with other apps, test the new memory behaviour before relying on it, or keep an explicit gpu_memory_utilization value.