Skip to content

VersusGPU

FP4 vs FP8 vs FP16: What AI Precision Formats Mean for Your GPU and VRAM

FP16, FP8 and FP4 describe how many bits store each value in an AI model: roughly 2GB, 1GB and 0.5GB per billion parameters for weights. Lower precision saves memory and raises headline performance but risks accuracy. NVIDIA's Blackwell NVFP4 adds an FP8 scale per 16 values; NVIDIA says it cuts memory about 3.5x versus FP16.

By , Editorial team

4 min read

FP4 vs FP8 vs FP16: What AI Precision Formats Mean for Your GPU and VRAM

Read about any AI graphics card or mini PC and you'll see performance quoted "at FP4" or "at FP8", and model downloads labelled FP16, FP8 or NVFP4. These are number formats: how many bits are used to store each value in an AI model. Fewer bits mean a smaller model that needs less memory and moves less data, which is why headline AI figures are often quoted at the lowest precision a chip supports.

Forge PC hasn't benchmarked these formats for this explainer. The technical details come from NVIDIA's documentation, and the memory maths is simple arithmetic.

The short version

Format

Bits per value

Approx. memory per billion parameters (weights only)

Typical use

FP16 / BF16

16

about 2GB

Training and full-quality inference

FP8

8

about 1GB

Efficient inference and some training on newer GPUs

FP4 (e.g. NVFP4)

4 (NVFP4: about 4.5 with scaling data)

about 0.5-0.6GB

Very efficient inference on Blackwell-generation GPUs

These figures are for model weights only. Real memory use is higher, because apps also need room for the context (KV cache) and working memory; our context length explainer covers that.

What "floating point" precision means

A floating-point number stores a sign, an exponent (the scale) and a mantissa (the precise digits). With 16 bits you get a lot of range and detail; with 4 bits you get very little. NVIDIA says the 4-bit format it uses has 1 sign bit, 2 exponent bits and 1 mantissa bit (E2M1), so each value can only be one of a handful of numbers between roughly -6 and 6, such as 0, 0.5, 1, 1.5, 2, 3, 4 and 6.

That's why quantising a model from 16 bits to 4 bits can lose accuracy, and why the clever part of modern 4-bit formats is scaling.

FP16 and BF16

These 16-bit formats are the traditional baseline for AI. They give the best accuracy of the three, but a model needs about 2 bytes per parameter for its weights, so even an 8-billion-parameter model needs around 16GB for weights alone. That's more than many gaming cards have.

FP8

FP8 halves the memory of FP16. NVIDIA says FP8 is supported by its Hopper and Blackwell GPUs and brought significant memory, latency and throughput benefits over 16-bit formats. It's a common format for efficient inference on newer hardware.

FP4, MXFP4 and NVFP4

NVIDIA's fifth-generation Tensor Cores in its Blackwell architecture support several 4-bit floating-point formats. NVIDIA's technical blog compares three:

  • FP4: plain 4-bit values.

  • MXFP4: 4-bit values with one shared power-of-two scale for every block of 32 values.

  • NVFP4: NVIDIA's format, with one shared FP8 (E4M3) scale for every 16 values, plus a second FP32 scale per tensor.

The smaller blocks and more precise scales let NVFP4 match the data more closely. NVIDIA says NVFP4 stores about 4.5 bits per value including the scale overhead, cutting model memory by about 3.5x compared with FP16 and about 1.8x compared with FP8. In NVIDIA's own tests on DeepSeek-R1-0528, quantising from FP8 to NVFP4 cost 1% or less accuracy on key language tasks. That's NVIDIA's result for one model, not a guarantee for every model.

NVIDIA also says lower-precision maths uses less energy for data movement and arithmetic, and its Tensor Core page says FP4 support in its Transformer Engine doubles performance and efficiency for supported models.

What this means for your PC

  • Check the precision behind AI performance claims. A figure quoted at FP4 will be larger than the same chip's FP8 or FP16 figure. Compare like with like.

  • Hardware support matters. NVFP4 is a Blackwell-generation feature, so it's relevant to RTX 50-series and newer NVIDIA hardware. Older cards can still run models stored in fewer bits, but without native FP4 maths.

  • Most local LLM apps use GGUF. Tools like Ollama and LM Studio typically run GGUF files with quantisation levels such as Q4 or Q8, which are block-based quantisation formats rather than NVFP4. Our GGUF quantisation explainer covers those.

  • VRAM still decides what fits. Halving the bits roughly halves the weights' memory, which is why 4-bit models let far larger models run on a single card. See how much VRAM you need and our best GPU for local LLMs.

Our verdict

FP16, FP8 and FP4 describe how many bits store each value in an AI model: about 2GB, 1GB and 0.5GB per billion parameters for weights. Lower precision saves memory and boosts headline performance, but risks accuracy. NVIDIA's NVFP4 uses an FP8 scale for every 16 values plus a per-tensor scale, and NVIDIA says it cuts memory by about 3.5x versus FP16 with 1% or less accuracy loss versus FP8 on its DeepSeek tests. When comparing AI hardware, always check which precision a figure uses. Forge PC hasn't benchmarked these formats.

The bottom line

FP16, FP8 and FP4 describe how many bits store each value in an AI model: about 2GB, 1GB and 0.5GB per billion parameters for weights. Lower precision saves memory and boosts headline performance, but risks accuracy. NVIDIA's NVFP4 uses an FP8 scale for every 16 values plus a per-tensor scale, and NVIDIA says it cuts memory by about 3.5x versus FP16 with 1% or less accuracy loss versus FP8 on its DeepSeek tests. When comparing AI hardware, always check which precision a figure uses. Forge PC hasn't benchmarked these formats.

Sources

  1. 01NVIDIA Technical Blog, Introducing NVFP4 for Efficient and Accurate Low-Precision Inference
  2. 02NVIDIA, Tensor Cores

About the author

Forge PC Staff

Editorial team

The shared byline for articles researched, written and edited by the Forge PC editorial team. How we source, check and correct our work is set out in our editorial policy.

More from Forge PC Staff

Read next

How to Check Your CUDA Version on Windows (Driver, Toolkit and PyTorch)

How-To

How to Check Your CUDA Version on Windows (Driver, Toolkit and PyTorch)

Run nvidia-smi: its CUDA Version is the highest CUDA version your NVIDIA driver supports, which is what most local AI apps need. nvcc -V shows the CUDA Toolkit version, but only if you've installed it. In PyTorch, torch.cuda.is_available() confirms its CUDA build can see your GPU. If an app needs newer CUDA than nvidia-smi shows, update the driver.

Forge PC Staff3 min

RTX 3090 vs RTX 5070 Ti for Local AI: 24GB Old or 16GB New?

Versus

RTX 3090 vs RTX 5070 Ti for Local AI: 24GB Old or 16GB New?

For local AI, the RTX 3090's 24GB of GDDR6X beats the RTX 5070 Ti's 16GB of GDDR7 whenever models and their context need more than 16GB, which usually decides it. The 5070 Ti is a current Blackwell card with 5th-gen Tensor Cores, FP4 support, 300W power against 350W and a new-card warranty, so it's the safer buy if your models fit in 16GB.

Forge PC Staff3 min

How to Use DLSS Override in the NVIDIA App (DLSS 4.5 Presets)

How-To

How to Use DLSS Override in the NVIDIA App (DLSS 4.5 Presets)

In the NVIDIA app's Graphics tab, choose a game or the global settings and set DLSS Override - Model Presets to Recommended. That uses DLSS 4.5's Preset M in Performance mode, Preset L in Ultra Performance and Preset K elsewhere. NVIDIA says RTX 20 and 30 owners may prefer Model K, because those cards lack native FP8. Check with the Alt+Z overlay.

Forge PC Staff3 min

Dell Creator Edition Desktop: Dell's 1.2-Litre RTX Spark PC for Local AI

News

Dell Creator Edition Desktop: Dell's 1.2-Litre RTX Spark PC for Local AI

Dell's Creator Edition Desktop went on sale in the US on 18 November 2026: a 1.2-litre Windows 11 on Arm PC with NVIDIA's RTX Spark chip, combining a 6,144-core Blackwell GPU, a 20-core Grace CPU and 128GB of unified memory for local AI. Dell claims models of up to 200B parameters run on-device. UK pricing and availability haven't been announced.

Forge PC Staff2 min