Skip to content

How-ToGPU

How to Use Ollama with Python: Run Local AI Models From Your Own Scripts

How to use Ollama from Python: install the official library with pip, chat with a local model in a few lines, stream replies, manage models with list and pull, handle errors and connect to Ollama on another PC with a Client.

By , Systems & Technical Editor

3 min read

How to Use Ollama with Python: Run Local AI Models From Your Own Scripts

Once you've got Ollama running local AI models on your PC, the next step is often to use those models from your own scripts: summarising files, tagging photos, sorting emails or building a simple chatbot. Ollama's official Python library makes that straightforward. This guide follows the library's official documentation.

What you need

  • Ollama installed and running. If you haven't set it up yet, see how to run AI models locally with Ollama and LM Studio.

  • Python 3.8 or newer. The library's documentation says it supports Python 3.8+.

  • A model. Pull one in a terminal with ollama pull followed by the model name. The library's examples use gemma4; use any model you've downloaded. If your graphics card has limited memory, our guide to running a local LLM on 8GB of VRAM explains how to pick a model that fits.

Step 1: Install the library

Open a terminal (ideally inside a Python virtual environment) and run:

pip install ollama

Step 2: Send your first prompt

Create a file called ask.py with this code, based on the library's own example:

from ollama import chat

response = chat(
    model='gemma4',
    messages=[{'role': 'user', 'content': 'Why is the sky blue?'}],
)
print(response.message.content)

Run it with python ask.py. The messages list works like a conversation: each entry has a role, such as "user", and some content. To continue a conversation, append the model's reply and your next question to the list and call chat again.

Step 3: Stream the reply as it's written

Waiting for a long answer to finish can feel slow. Set stream=True and the library returns the reply in chunks that you can print as they arrive:

from ollama import chat

stream = chat(
    model='gemma4',
    messages=[{'role': 'user', 'content': 'Why is the sky blue?'}],
    stream=True,
)
for chunk in stream:
    print(chunk['message']['content'], end='', flush=True)

Step 4: Use the other built-in functions

The library mirrors Ollama's command-line tools, so you can manage models from Python too. According to the documentation, the main functions include:

Function

What it does

ollama.generate()

Single prompt and reply, without a message history

ollama.list()

Lists the models you've downloaded

ollama.pull() / ollama.delete()

Downloads or removes a model

ollama.show()

Shows details of a model

ollama.ps()

Shows which models are loaded

ollama.embed()

Creates embeddings, for search or document Q&A projects

Embeddings are the building block for chatting with your own documents; our RAG vs fine-tuning explainer covers how that works.

Step 5: Handle errors

The library raises an error if a request fails. Its documentation shows catching ollama.ResponseError and pulling the model automatically if the error code is 404 (model not found):

import ollama

model = 'gemma4'
try:
    ollama.chat(model)
except ollama.ResponseError as e:
    print('Error:', e.error)
    if e.status_code == 404:
        ollama.pull(model)

Step 6: Connect to Ollama on another PC

By default the library talks to Ollama on your own PC. To use a different machine, such as a desktop with a bigger graphics card, create a Client with that machine's address. The documentation's example uses http://localhost:11434, Ollama's local address; replace localhost with the other PC's name or IP address. That PC must be set up to accept network connections; see how to share Ollama on your local network. There's also an AsyncClient for asynchronous code.

Prefer OpenAI-style code?

If you already have scripts written for OpenAI's API, Ollama also offers OpenAI compatibility, so you can point those at your local models instead. Our guide to using Ollama's OpenAI-compatible API explains how.

Our verdict

Install the library with pip install ollama, make sure Ollama is running with a model pulled, and you can chat with local models in about five lines of Python. Use stream=True for responsive output, ollama.list() and ollama.pull() to manage models, and a Client with a host address to use a more powerful PC on your network. It's the simplest way to build your own local AI tools without sending data to the cloud.

The bottom line

Install the library with pip install ollama, make sure Ollama is running with a model pulled, and you can chat with local models in about five lines of Python. Use stream=True for responsive output, ollama.list() and ollama.pull() to manage models, and a Client with a host address to use a more powerful PC on your network. It's the simplest way to build your own local AI tools without sending data to the cloud.

Sources

  1. 01Ollama, Ollama Python library (GitHub)
  2. 02Ollama, API documentation

About the author

Marcus Thorne

Systems & Technical Editor

Marcus writes Forge PC's step-by-step build, troubleshooting and system optimisation guides. His background spans Windows kernel and power-state diagnostics, UEFI/BIOS configuration, NVMe storage arrays, ATX 3.1 power delivery and running local AI and LLM workloads on consumer graphics cards.

More from Marcus Thorne

Read next

LoRA vs QLoRA: Which Fine-Tuning Method Fits Your GPU?

Versus

LoRA vs QLoRA: Which Fine-Tuning Method Fits Your GPU?

LoRA vs QLoRA for local LLM fine-tuning: both train small adapters on a frozen model, but QLoRA loads the base model in 4-bit, which Unsloth says cuts VRAM use about four times for a marginal accuracy cost. QLoRA suits most home GPUs.

Callum Reid3 min