Skip to content

VersusGPU

LoRA vs QLoRA: Which Fine-Tuning Method Fits Your GPU?

LoRA vs QLoRA for local LLM fine-tuning: both train small adapters on a frozen model, but QLoRA loads the base model in 4-bit, which Unsloth says cuts VRAM use about four times for a marginal accuracy cost. QLoRA suits most home GPUs.

By , Senior Hardware Editor

3 min read

LoRA vs QLoRA: Which Fine-Tuning Method Fits Your GPU?

If you want to teach a local AI model a new style, format or task, fine-tuning is how you do it, and on a home PC that almost always means LoRA or QLoRA. Both avoid retraining the whole model. The difference is how much graphics memory (VRAM) they need, and that decides what you can fine-tune on your card. This explainer is based on the original QLoRA research paper and documentation from Hugging Face and Unsloth.

If you're not sure fine-tuning is what you need at all, read our RAG vs fine-tuning comparison first: for answering questions about your own documents, RAG is usually the better starting point.

The short version

LoRA

QLoRA

What it trains

Small low-rank adapter layers; the base model is frozen

The same adapters, on a frozen base model quantised to 4-bit

Base model precision during training

16-bit

4-bit (NF4)

VRAM use

Higher: Unsloth says about 4 times more than QLoRA

Lower: Unsloth says about 4 times less than LoRA

Speed and accuracy

Unsloth says slightly faster and slightly more accurate

Unsloth says slightly slower and marginally less accurate

Best for

Cards with plenty of VRAM, or when maximum accuracy matters

Most home GPUs, and larger models than would otherwise fit

How LoRA works

LoRA (Low-Rank Adaptation) freezes the original model's weights and trains small extra matrices, called adapters, that adjust the model's behaviour. Because only the adapters are trained, it needs far less memory than full fine-tuning, and the result is a small adapter file rather than a whole new copy of the model. Unsloth says that, done well, LoRA can match full fine-tuning performance while using about 4 times less VRAM. Unsloth describes LoRA as 16-bit fine-tuning: the frozen base model is held in 16-bit precision.

How QLoRA works

QLoRA, introduced in a 2023 research paper by Tim Dettmers and colleagues, keeps the LoRA adapters but loads the frozen base model in 4-bit precision. The paper says it reduces memory use enough to fine-tune a 65-billion-parameter model on a single 48GB GPU while preserving full 16-bit fine-tuning task performance. It introduced three memory-saving techniques:

  • 4-bit NormalFloat (NF4), a data type designed for the way model weights are typically distributed;

  • Double quantisation, which also compresses the quantisation constants themselves;

  • Paged optimisers, which manage memory spikes during training.

Hugging Face's PEFT documentation describes QLoRA the same way, as quantising a model to 4-bit and then training it with LoRA, and notes that combining quantisation with adapters can be a good strategy for training even the largest models on a single GPU. Unsloth says a 70B Llama model fits in under 48GB of VRAM with QLoRA in its tools.

Which should you use?

  • Choose QLoRA if VRAM is your limit, as it is on most gaming graphics cards. It lets you fine-tune a bigger model than LoRA would on the same card, and Unsloth's figures suggest the accuracy cost is marginal.

  • Choose LoRA if your GPU has plenty of VRAM for the model you're training, or you want the slightly better accuracy and speed Unsloth describes.

  • Start small either way. Fine-tuning a small model first is the quickest way to learn whether your dataset works. Our how-to on fine-tuning an LLM locally with Unsloth walks through the process.

The settings that drive VRAM use most aren't only LoRA vs QLoRA. Unsloth's guide says batch size is the primary driver of VRAM usage, and suggests gradient accumulation to simulate a larger batch while conserving memory. It recommends a LoRA rank of 16 or 32 as a starting point, and a learning rate of 2e-4 for normal LoRA or QLoRA fine-tuning.

What it means for your hardware

VRAM is the limit for local fine-tuning, just as it is for running models. If you're choosing a card with AI in mind, our best GPU for local LLMs guide and how much VRAM you need explain the trade-offs. The 4-bit idea behind QLoRA is related to the quantised models you run day to day; our LLM quantisation explainer covers that side.

Our verdict

QLoRA is the right default for fine-tuning on a home PC: by loading the base model in 4-bit, it cuts VRAM use by around three-quarters compared with LoRA, according to Unsloth, with only a marginal accuracy cost. Use plain LoRA if you have VRAM to spare or need every last bit of accuracy. Either way, the adapter you produce is small, and you can start with a small model and a modest dataset.

The bottom line

QLoRA is the right default for fine-tuning on a home PC: by loading the base model in 4-bit, it cuts VRAM use by around three-quarters compared with LoRA, according to Unsloth, with only a marginal accuracy cost. Use plain LoRA if you have VRAM to spare or need every last bit of accuracy. Either way, the adapter you produce is small, and you can start with a small model and a modest dataset.

About the author

Callum Reid

Senior Hardware Editor

Callum leads GPU, CPU and platform architecture coverage at Forge PC. Before joining the editorial desk, he spent nine years specifying, assembling and stress-testing custom gaming and workstation systems in the UK, with a focus on PCIe bandwidth, VRM thermal design, DDR5 memory tuning and 1440p/4K frame-time consistency.

More from Callum Reid

Read next

RAG vs Fine-Tuning for Local AI: Which Should You Use on Your PC?

Versus

RAG vs Fine-Tuning for Local AI: Which Should You Use on Your PC?

RAG vs fine-tuning for local AI: RAG feeds a model relevant chunks of your documents for each question, while fine-tuning changes the model's weights. For most home PCs, start with RAG; fine-tune for style, format or specialised tasks.

Callum Reid4 min