Skip to content

NewsRAM

FreeToken Promises 284B AI Models on a Gaming PC: What You Actually Need

FreeToken is a new open-source engine from UC Berkeley and MIT researchers that splits huge mixture-of-experts AI models across an NVIDIA RTX graphics card, the CPU and system RAM. IFAI reports 22 to 25 tokens per second on the 284B DeepSeek-V4-Flash, but that used an RTX 5090 with 192GB of RAM, the figures are unverified and AMD support is experimental. Forge PC has not tested FreeToken.

By , Editorial team

4 min read

FreeToken Promises 284B AI Models on a Gaming PC: What You Actually Need

A new open-source AI engine called FreeToken is getting attention for one bold claim: that a gaming PC can run "frontier" mixture-of-experts (MoE) models with hundreds of billions of parameters at usable speeds. It does this by treating your graphics card, CPU, system memory and PCIe bus as one pool, rather than insisting the whole model fits in video memory.

Forge PC has not tested FreeToken. This story is based on the project's GitHub page and documentation and on independent coverage from IFAI, all checked on 16 November 2026. Speed figures below are the developers' and IFAI's, not ours.

What FreeToken is

According to the project page, FreeToken is an "edge-native" MoE serving engine released under the Apache 2.0 licence, and IFAI reports it comes from researchers at UC Berkeley and MIT. The code is on GitHub under FlashML-org, and there is a desktop app for Windows and Linux at flashml.ai that sets the engine up and adds a graphical interface for running models and chatting.

The trick is in how MoE models work. As we explained in our MoE vs dense models guide, an MoE model only activates a fraction of its "experts" for each token. FreeToken keeps frequently used experts cached on the GPU and decides, layer by layer, whether the rest should be computed on the CPU or streamed over PCIe to the graphics card, based on measured transfer and memory speeds. The documentation lists several strategies (fused, offload, cpu and hybrid, plus auto).

Other features the developers list:

  • Model support: DeepSeek-V4-Flash, Qwen3.6-35B-A3B, GLM-5.2 and other MoE models, in formats including MXFP4, NVFP4, FP8, BF16 and llama.cpp's GGUF files.

  • Agent support: OpenAI- and Anthropic-compatible local APIs, with a ft launch claude command to connect Claude Code, plus Codex, OpenCode and others.

  • Smarter caching for agents: the developers say "semantic anchor" checkpoints avoid recomputing the whole context when a coding agent edits its thinking blocks or adds tool results.

The claimed speeds, and the hardware behind them

IFAI's write-up of the project's results says a gaming PC with an RTX 5090 (32GB of video memory) and 192GB of system RAM reached 22 to 25 tokens per second on the 284-billion-parameter DeepSeek-V4-Flash. It also reports about 39.3 tokens per second for Qwen3.6 35B on an 8GB RTX 4060 laptop, and says overall decode speed is roughly 1.3x to 2.1x faster than existing local engines. We haven't been able to verify these numbers.

The important detail is the memory. IFAI says FreeToken starts at 32GB of system RAM, but models in the 284B class need 64GB to 192GB. That is a lot of memory in late 2026, when RAM prices are high (see why RAM and GPUs are so expensive). For a broader view of what local AI needs, our how much RAM for local AI guide is a good starting point.

What you need to run it

Requirement

What the documentation says

Graphics card

Native support for NVIDIA RTX 30, 40 and 50 series

AMD graphics

Experimental, work-in-progress support for RDNA 3 and RDNA 4 via ROCm 7.14

Command-line version

Linux x86_64, NVIDIA driver r580 or newer (CUDA 13), Python 3.10 or newer

Windows

Desktop app download from flashml.ai

Local API address

127.0.0.1:1919 by default

IFAI's article says CUDA 12 is needed, but the project's own install guide asks for driver r580+ and CUDA 13, so follow the official documentation.

For most Windows gamers the desktop app is the sensible route. The command-line package (uv pip install "freetoken[accel]") is aimed at Linux, and its CUDA kernels compile on first use. If you want a Linux environment on a Windows PC, our WSL2 local AI guide covers GPU setup.

Why it matters

Until now, running models with hundreds of billions of parameters at home has generally meant an expensive workstation card, a Mac with lots of unified memory, or accepting very slow CPU offloading in tools like llama.cpp and Ollama. If FreeToken's approach holds up, the bottleneck shifts from video memory to system RAM and PCIe bandwidth, which favours desktops with lots of DIMM slots and a full-speed x16 graphics slot.

There are caveats. The project is young (the GitHub repository was created in July 2026), the headline speeds come from the developers, and IFAI notes that complex agent contexts can still take tens of seconds before the first token appears.

Our verdict

FreeToken is one of the more interesting local AI projects of 2026 because it targets the hardware many enthusiasts already own: an RTX graphics card and a desktop with room for lots of RAM. But the headline 284B result relies on an RTX 5090 and 192GB of memory, the speed claims are not independently verified, and AMD support is experimental. If you have a high-end NVIDIA PC and want to try big MoE models, the Windows desktop app is the easy way in. Everyone else should stick with smaller models in Ollama or LM Studio for now. Forge PC has not tested FreeToken.

The bottom line

FreeToken is one of the more interesting local AI projects of 2026 because it targets the hardware many enthusiasts already own: an RTX graphics card and a desktop with room for lots of RAM. But the headline 284B result relies on an RTX 5090 and 192GB of memory, the speed claims are not independently verified, and AMD support is experimental. If you have a high-end NVIDIA PC and want to try big MoE models, the Windows desktop app is the easy way in. Everyone else should stick with smaller models in Ollama or LM Studio for now. Forge PC has not tested FreeToken.

In this article

Sources

  1. 01FlashML-org, FreeToken GitHub repository
  2. 02FreeToken documentation, Install
  3. 03FreeToken documentation, AMD ROCm installation (WIP)
  4. 04FreeToken documentation, Quick start
  5. 05IFAI, FreeToken: local MoE inference engine

About the author

Forge PC Staff

Editorial team

The shared byline for articles researched, written and edited by the Forge PC editorial team. How we source, check and correct our work is set out in our editorial policy.

More from Forge PC Staff

Read next

How to Enable AMD Fluid Motion Frames (AFMF 2.1) on a Radeon Graphics Card

How-To

How to Enable AMD Fluid Motion Frames (AFMF 2.1) on a Radeon Graphics Card

To enable AMD Fluid Motion Frames, install Adrenalin 25.3.1 or newer, then select the HYPR-RX profile (RX 7000 and newer) or turn on AMD Fluid Motion Frames under Gaming > Graphics, globally or per game. AFMF 2.1 works on RX 6000 cards and newer in DirectX 11, DirectX 12, Vulkan and OpenGL games, with VSync off; RX 6000 cards need exclusive full screen.

Forge PC Staff2 min

Two vs Four RAM Sticks for DDR5: Why 2x32GB Usually Beats 4x16GB

Versus

Two vs Four RAM Sticks for DDR5: Why 2x32GB Usually Beats 4x16GB

Two DDR5 sticks are better for most builds. AMD lists DDR5-5600 as the AM5 maximum with two modules but DDR5-3600 with four, and XMP or EXPO speeds above that are overclocking and not guaranteed. Buy 2x16GB or 2x32GB for 32GB or 64GB, and use four sticks, ideally a matched kit, only if you need 128GB or more.

Forge PC Staff3 min

AMD's FSR 4.1.2 Unreal Engine 5 Plugin Brings ML Upscaling to RX 6000 Cards

News

AMD's FSR 4.1.2 Unreal Engine 5 Plugin Brings ML Upscaling to RX 6000 Cards

AMD's FSR Unreal Engine 5 plugin now uses FSR Upscaling 4.1.2, bringing ML-based upscaling to Radeon RX 6000 cards and select Ryzen integrated graphics as well as RX 7000 and RX 9000, in UE 5.3 to 5.8 games that adopt it. It's DirectX 12 only for now and adds Frame Generation 4.0.2 with driver-assisted frame pacing on RX 9000 cards.

Forge PC Staff2 min

NVIDIA's DGX Spark HPC Developer Kit Links Four Sparks Into a 512GB Desk-Side Cluster

News

NVIDIA's DGX Spark HPC Developer Kit Links Four Sparks Into a 512GB Desk-Side Cluster

NVIDIA's DGX Spark HPC Developer Kit, announced at SC26 on 16 November 2026, links four DGX Spark systems in a ring over ConnectX-7 with no network switch, for 512GB of combined memory and 20TB of NVMe storage. NVIDIA says it can spread models of up to 700B parameters across the nodes with NVFP4. No UK kit price yet; Scan sells two-Spark bundles at about £12,845.

Forge PC Staff3 min