Local AI is not just for chat. Many of the open models you can run in Ollama are vision models: Ollama's documentation describes them as models that "accept images alongside text so the model can describe, classify, and answer questions about what it sees". That means you can ask a model on your own PC to describe a photo, read a screenshot or explain a chart, without uploading the image anywhere.
This guide follows Ollama's official documentation and model library. Forge PC has not benchmarked these models, and answers from any AI model can be wrong, so check anything important. If you have not installed Ollama yet, start with our guide to running AI models locally with Ollama and LM Studio.
Step 1: Pick a vision model that fits your GPU
In Ollama's model library, models that understand images are tagged vision, and each model page lists Image as an input. Google's Gemma 4 family is a good starting point, and it is the model Ollama uses in its own vision examples. These are the download sizes Ollama's library lists for each version:
Ollama model | Listed size | Context window | Inputs |
|---|
gemma4:e2b | 4.6GB to 7.5GB | 128K | Text, image |
gemma4:e4b (the default gemma4) | 6.6GB to 9.5GB | 128K | Text, image |
gemma4:12b | 7.7GB to 8.0GB | 256K | Text, image |
gemma4:26b | 16GB to 19GB | 256K | Text, image |
gemma4:31b | 19GB to 20GB | 256K | Text, image |
qwen3.8:27b | 18GB | 256K | Text, image |
As a rule of thumb, you want the model to fit in your graphics card's memory with some room to spare for the context. An 8GB card suits the smaller Gemma 4 versions, while the 26B and 31B versions and Qwen3.8 27B need a 24GB-class card or a split across two GPUs. Our guide to how much VRAM you need explains the maths, and our Gemma 4 guide covers the family in more detail.
Download the default model with:
ollama pull gemma4
Step 2: Ask about an image from the command line
Ollama's quick-start example puts the image path straight into the prompt:
ollama run gemma4 ./image.png what is in this image?
You can also start a chat with ollama run gemma4 and paste or drag an image file's path into your message. Ollama's announcement of its multimodal engine (15 May 2025) shows several images given in one message, followed by questions about how they relate, so you can compare two screenshots or photos in the same conversation.
Step 3: Use the Ollama app instead
If you prefer not to use a terminal, the Ollama app for Windows and macOS lets you download and chat with models. Ollama's July 2025 announcement says the app supports file drag and drop and that images can be sent to models that support them. Pick a vision model such as Gemma 4 from the model menu, drop in an image and ask your question.
For a more feature-rich chat interface in your browser, see our guide to installing Open WebUI for Ollama.
Step 4: Send images through the API
To use a vision model from your own scripts, send an images array with your message to Ollama's local API at http://localhost:11434. Ollama's documentation says the REST API expects base64-encoded image data, while the official Python and JavaScript libraries also accept file paths, URLs or raw bytes.
In Python, using the official ollama library:
from ollama import chat
response = chat(
model='gemma4',
messages=[{
'role': 'user',
'content': 'What is in this image? Be concise.',
'images': ['photo.jpg'],
}],
)
print(response.message.content)
With curl, encode the image as base64 first and send it to /api/chat in the images field, as shown in Ollama's vision documentation. If you already use other tools that expect OpenAI's API format, our guide to Ollama's OpenAI-compatible API explains how to point them at your local server.
Tips for better results
Be specific. "List the error messages in this screenshot" works better than "what is this?".
Crop out what you do not need. A smaller, focused image gives the model less to get confused by.
Try a bigger model for hard tasks. Small models are quick but more likely to miss detail. If a larger model will not fit your card, our best GPU for local LLMs guide covers upgrade options.
Keep it private. Everything runs on your PC, which is the main reason to use a local model for personal photos or work documents.
Our verdict
Using a vision model in Ollama is as simple as pulling a model tagged vision, such as Gemma 4, and including an image in your prompt, whether in the app, at the command line or through the images field of the API. Pick a model size your graphics card can hold: the default Gemma 4 is listed at 6.6GB to 9.5GB, while the 26B and 31B versions want a 24GB-class card. These steps follow Ollama's official documentation; Forge PC has not benchmarked the models, so double-check anything that matters.
The bottom line
Using a vision model in Ollama is as simple as pulling a model tagged vision, such as Gemma 4, and including an image in your prompt, whether in the app, at the command line or through the images field of the API. Pick a model size your graphics card can hold: the default Gemma 4 is listed at 6.6GB to 9.5GB, while the 26B and 31B versions want a 24GB-class card. These steps follow Ollama's official documentation; Forge PC has not benchmarked the models, so double-check anything that matters.