Skip to content

How-ToGPU

How to Run a Local AI Model Across Two GPUs with Ollama, LM Studio or llama.cpp

Two graphics cards can hold a larger local AI model than one. Ollama spreads a model across GPUs automatically only when it will not fit on one card; LM Studio 0.3.14 and later let you choose which cards to use and how to fill them; llama.cpp gives full control with --split-mode and --tensor-split. This guide follows each app's official documentation.

By , Editorial team

4 min read

How to Run a Local AI Model Across Two GPUs with Ollama, LM Studio or llama.cpp

If a local language model is too big for one graphics card, a second card can give you more total VRAM to hold it. Ollama, LM Studio and llama.cpp can all split a model across two GPUs, but they do it differently, and none of them turns two cards into one faster card.

Forge PC has not benchmarked a two-GPU setup for this guide, so there are no speed figures here. The behaviour and settings below come from Ollama's official FAQ and GPU documentation, LM Studio's 0.3.14 release notes and llama.cpp's official server documentation, all read on 14 November 2026.

First, what two GPUs do and don't do

  • They add room. With the model split across two cards, the combined VRAM can hold a larger model or a less compressed version of it. Our GGUF and quantisation explainer covers why that matters.

  • They don't simply double speed. By default, llama.cpp splits a model by layers, which its documentation describes as "pipelined". In a pipelined split, the cards largely take turns rather than both working flat out on the same token.

  • One card is often better if the model fits. Ollama's own FAQ says loading a model on a single GPU "typically provides the best performance" because it reduces data transfers across the PCIe bus.

If you are still choosing hardware, start with our best GPU for local LLMs guide and, for second-hand cards, our used GPU for local AI guide.

Ollama: automatic, with a way to limit it

Ollama needs no special setting. According to its FAQ, when you load a model it checks the VRAM available. If the model fits entirely on any single GPU, Ollama loads it on that GPU. If it does not fit on one, Ollama spreads it across all available GPUs.

To stop Ollama using a card, set an environment variable before starting the Ollama server:

  • NVIDIA: set CUDA_VISIBLE_DEVICES to a comma-separated list of the GPUs you want Ollama to use. Ollama's GPU documentation says numeric IDs work, but ordering can vary, so UUIDs are more reliable. Run nvidia-smi -L to list them.

  • AMD: the equivalent variable is ROCR_VISIBLE_DEVICES.

If you are new to Ollama, set it up first with our Ollama and LM Studio starter guide.

LM Studio: the multi-GPU controls

LM Studio added dedicated multi-GPU controls in version 0.3.14. According to its release notes:

  1. Press Ctrl+Shift+H on Windows (Cmd+Shift+H on a Mac) to open the GPU controls.

  2. Use the switch next to each GPU to enable or disable it. A disabled GPU is not used at all.

  3. Choose an allocation strategy: split evenly, or use priority order, which fills the GPUs in the order you set.

  4. Optionally turn on Limit Model Offload to Dedicated GPU Memory, so model weights do not spill into slower shared GPU memory. LM Studio says that in its testing, splitting weights between dedicated VRAM and system RAM was faster than using shared memory.

LM Studio's release notes say priority order and the dedicated-memory limit were CUDA-only (NVIDIA) at launch, with AMD support being worked on. Check the current version's settings if you use Radeon cards.

llama.cpp: full manual control

llama.cpp's server documentation lists the options that control a multi-GPU split:

  • --list-devices prints the GPUs llama.cpp can see, and -dev (or --device) chooses which to use.

  • -sm (or --split-mode) sets how the model is split: none (one GPU only), layer (the default: layers and KV cache split across GPUs, pipelined), row (weights split by rows, parallelised) or tensor (marked experimental).

  • -ts (or --tensor-split) sets the proportion of the model each GPU gets, for example 3,1 to put three-quarters on the first card. This is useful when your cards have different amounts of VRAM.

  • -mg (or --main-gpu) picks the GPU used with split mode none, or for intermediate results and the KV cache with split mode row.

So a server running a model split 3:1 across two cards by layer would use something like llama-server -m your-model.gguf -sm layer -ts 3,1, with your other usual options added. If you are deciding between tools, see our llama.cpp vs Ollama comparison.

Hardware checks before you add a second card

  • Power supply. Two GPUs can draw a lot of power. Add up both cards' official board power and the rest of the system, then size the PSU with our power supply buying guide.

  • Slots and spacing. Check your motherboard manual for which slot the second card can use and how many PCIe lanes it gets. Many consumer boards run the second full-length slot at fewer lanes, often through the chipset. Also check the cards physically fit with room to breathe.

  • Cooling. A second card in the slot below the first can starve it of air. Watch temperatures with our HWiNFO monitoring guide.

Our verdict

If a model fits on one card, keep it on one card; Ollama does that automatically and says it is usually faster. When a model needs more VRAM than one GPU has, Ollama will spread it across all your cards without any setup, LM Studio 0.3.14 and later let you choose which cards to use and how to fill them, and llama.cpp gives full control with --split-mode and --tensor-split. Check your PSU, slots and airflow before buying a second card. Forge PC has not benchmarked two-GPU setups; this guide follows the official documentation for each app.

The bottom line

If a model fits on one card, keep it on one card; Ollama does that automatically and says it is usually faster. When a model needs more VRAM than one GPU has, Ollama will spread it across all your cards without any setup, LM Studio 0.3.14 and later let you choose which cards to use and how to fill them, and llama.cpp gives full control with `--split-mode` and `--tensor-split`. Check your PSU, slots and airflow before buying a second card. Forge PC has not benchmarked two-GPU setups; this guide follows the official documentation for each app.

Sources

  1. 01Ollama, FAQ: How does Ollama load models on multiple GPUs?
  2. 02Ollama, GPU documentation (GPU selection)
  3. 03LM Studio, LM Studio 0.3.14: Multi-GPU Controls (27 March 2025)
  4. 04llama.cpp, llama-server README (ggml-org/llama.cpp)

About the author

Forge PC Staff

Editorial team

The shared byline for articles researched, written and edited by the Forge PC editorial team. How we source, check and correct our work is set out in our editorial policy.

More from Forge PC Staff

Read next

How to Enable AMD Fluid Motion Frames (AFMF 2.1) on a Radeon Graphics Card

How-To

How to Enable AMD Fluid Motion Frames (AFMF 2.1) on a Radeon Graphics Card

To enable AMD Fluid Motion Frames, install Adrenalin 25.3.1 or newer, then select the HYPR-RX profile (RX 7000 and newer) or turn on AMD Fluid Motion Frames under Gaming > Graphics, globally or per game. AFMF 2.1 works on RX 6000 cards and newer in DirectX 11, DirectX 12, Vulkan and OpenGL games, with VSync off; RX 6000 cards need exclusive full screen.

Forge PC Staff2 min

AMD's FSR 4.1.2 Unreal Engine 5 Plugin Brings ML Upscaling to RX 6000 Cards

News

AMD's FSR 4.1.2 Unreal Engine 5 Plugin Brings ML Upscaling to RX 6000 Cards

AMD's FSR Unreal Engine 5 plugin now uses FSR Upscaling 4.1.2, bringing ML-based upscaling to Radeon RX 6000 cards and select Ryzen integrated graphics as well as RX 7000 and RX 9000, in UE 5.3 to 5.8 games that adopt it. It's DirectX 12 only for now and adds Frame Generation 4.0.2 with driver-assisted frame pacing on RX 9000 cards.

Forge PC Staff2 min

NVIDIA's DGX Spark HPC Developer Kit Links Four Sparks Into a 512GB Desk-Side Cluster

News

NVIDIA's DGX Spark HPC Developer Kit Links Four Sparks Into a 512GB Desk-Side Cluster

NVIDIA's DGX Spark HPC Developer Kit, announced at SC26 on 16 November 2026, links four DGX Spark systems in a ring over ConnectX-7 with no network switch, for 512GB of combined memory and 20TB of NVMe storage. NVIDIA says it can spread models of up to 700B parameters across the nodes with NVFP4. No UK kit price yet; Scan sells two-Spark bundles at about £12,845.

Forge PC Staff3 min

Best Free Benchmark Software for PC in 2026: CPU, GPU, SSD and AI Tests

Best Picks

Best Free Benchmark Software for PC in 2026: CPU, GPU, SSD and AI Tests

The best free PC benchmarks in 2026: Cinebench 2026 for CPUs, Unigine Superposition Basic or 3DMark's free Steel Nomad test for graphics cards, CrystalDiskMark for SSDs, Geekbench 7 for cross-platform scores, Geekbench AI for NPUs, Blender Benchmark for creators and OCCT for stability. Run each test twice and download only from official sites.

Forge PC Staff5 min