Ollama is one of the easiest ways to run large language models locally, and on Linux it installs with a single command. It's a good fit for a dual-boot gaming PC, a home server or an old PC with a decent graphics card. This guide follows Ollama's official Linux documentation.
What you need
A 64-bit Linux install. Ollama publishes packages for x86-64 (amd64) and ARM64.
Optional: an NVIDIA or AMD graphics card. Ollama's Linux guide covers optional CUDA (NVIDIA) and ROCm (AMD) driver setup for GPU acceleration. Our guide to how much VRAM you need for local AI explains sizing, and our best GPU for local LLMs guide covers what to buy.
Terminal access with sudo rights.
If you're still choosing a Linux distribution, our guide to dual-booting Windows 11 and Linux covers setting one up alongside Windows.
Option 1: Install with the official script (easiest)
Ollama's documentation gives one command to install it:
curl -fsSL https://ollama.com/install.sh | sh
As with any script piped into a shell, you can download it first and read it before running it if you prefer.
Option 2: Manual install
If you'd rather not run the script, Ollama documents a manual install. If you're upgrading from an older version, first remove the old libraries with sudo rm -rf /usr/lib/ollama. Then download and extract the package:
curl -fsSL https://ollama.com/download/ollama-linux-amd64.tar.zst | sudo tar x -C /usr
For an AMD graphics card, Ollama says to also extract the ROCm package:
curl -fsSL https://ollama.com/download/ollama-linux-amd64-rocm.tar.zst | sudo tar x -C /usr
ARM64 systems use the ollama-linux-arm64.tar.zst package instead.
Start Ollama with ollama serve, then in another terminal check it's working with ollama -v.
Run it as a service (recommended for manual installs)
Ollama recommends adding it as a startup service. Create a user and group for it:
sudo useradd -r -s /bin/false -U -m -d /usr/share/ollama ollama
sudo usermod -a -G ollama $(whoami)
Then create /etc/systemd/system/ollama.service with the service definition from Ollama's documentation (it runs /usr/bin/ollama serve as the ollama user and restarts automatically), and enable it:
sudo systemctl daemon-reload
sudo systemctl enable ollama
Step 2: Set up GPU drivers
NVIDIA: Ollama's documentation says to download and install CUDA, then check the driver works by running nvidia-smi, which should print details about your GPU.
AMD: Ollama says to install ROCm v7. It notes that the amdgpu driver in the mainline Linux kernel is older and may not support all ROCm features, and recommends installing the latest driver from AMD's Linux drivers page for the best support of Radeon GPUs.
Our CUDA vs ROCm vs Vulkan explainer covers the differences between the GPU back ends.
Step 3: Start Ollama and run a model
Start the service and check its status:
sudo systemctl start ollama
sudo systemctl status ollama
Then run a model. Ollama's CLI reference uses this example:
ollama run gemma4
Models take up disk space, so check you have room first (our guide to how much storage local AI needs helps), and you can download a model in advance with ollama pull. Other useful commands from the CLI reference are ollama pull to download a model, ollama ls to list installed models, ollama ps to list running models, ollama stop to stop one and ollama rm to remove one. Our guide to managing Ollama models covers these in more detail.
Step 4: Customise, update and check logs
Settings: Ollama is configured with environment variables. Run sudo systemctl edit ollama to edit the service, or create /etc/systemd/system/ollama.service.d/override.conf with lines such as Environment="OLLAMA_DEBUG=1" under a [Service] heading.
Updating: run the install script again, or re-download the package as in the manual install.
Specific versions: set OLLAMA_VERSION when running the install script, for example curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.5.7 sh, which is the example version in Ollama's documentation.
Logs: journalctl -e -u ollama shows the logs for Ollama running as a startup service, which is a good place to start if something isn't working.
Uninstalling
Ollama's documentation lists the steps: stop and disable the service, remove the service file, remove the Ollama libraries and binary, and delete the ollama user, group and /usr/share/ollama folder (which holds downloaded models). Follow the exact commands on Ollama's Linux page.
Our verdict
On Linux, the official install script is the quickest way to get Ollama running, and the manual route gives you more control with a systemd service. Install CUDA or ROCm first so Ollama can use your graphics card, then use ollama run to download and chat with a model. If you want to use it from other PCs on your network, see our guide to sharing Ollama on your local network.