Skip to content

How-ToGPU

How to Use Vision Models in Ollama: Ask a Local AI About Your Images

Ollama can run vision models such as Gemma 4 that answer questions about images on your own PC. Pick a size your graphics card can hold (the default Gemma 4 is listed at 6.6GB to 9.5GB), then add an image in the Ollama app, at the command line or in the API's images field. Steps follow Ollama's documentation; Forge PC has not benchmarked the models.

By , Editorial team

4 min read

How to Use Vision Models in Ollama: Ask a Local AI About Your Images

Local AI is not just for chat. Many of the open models you can run in Ollama are vision models: Ollama's documentation describes them as models that "accept images alongside text so the model can describe, classify, and answer questions about what it sees". That means you can ask a model on your own PC to describe a photo, read a screenshot or explain a chart, without uploading the image anywhere.

This guide follows Ollama's official documentation and model library. Forge PC has not benchmarked these models, and answers from any AI model can be wrong, so check anything important. If you have not installed Ollama yet, start with our guide to running AI models locally with Ollama and LM Studio.

Step 1: Pick a vision model that fits your GPU

In Ollama's model library, models that understand images are tagged vision, and each model page lists Image as an input. Google's Gemma 4 family is a good starting point, and it is the model Ollama uses in its own vision examples. These are the download sizes Ollama's library lists for each version:

Ollama model

Listed size

Context window

Inputs

gemma4:e2b

4.6GB to 7.5GB

128K

Text, image

gemma4:e4b (the default gemma4)

6.6GB to 9.5GB

128K

Text, image

gemma4:12b

7.7GB to 8.0GB

256K

Text, image

gemma4:26b

16GB to 19GB

256K

Text, image

gemma4:31b

19GB to 20GB

256K

Text, image

qwen3.8:27b

18GB

256K

Text, image

As a rule of thumb, you want the model to fit in your graphics card's memory with some room to spare for the context. An 8GB card suits the smaller Gemma 4 versions, while the 26B and 31B versions and Qwen3.8 27B need a 24GB-class card or a split across two GPUs. Our guide to how much VRAM you need explains the maths, and our Gemma 4 guide covers the family in more detail.

Download the default model with:

ollama pull gemma4

Step 2: Ask about an image from the command line

Ollama's quick-start example puts the image path straight into the prompt:

ollama run gemma4 ./image.png what is in this image?

You can also start a chat with ollama run gemma4 and paste or drag an image file's path into your message. Ollama's announcement of its multimodal engine (15 May 2025) shows several images given in one message, followed by questions about how they relate, so you can compare two screenshots or photos in the same conversation.

Step 3: Use the Ollama app instead

If you prefer not to use a terminal, the Ollama app for Windows and macOS lets you download and chat with models. Ollama's July 2025 announcement says the app supports file drag and drop and that images can be sent to models that support them. Pick a vision model such as Gemma 4 from the model menu, drop in an image and ask your question.

For a more feature-rich chat interface in your browser, see our guide to installing Open WebUI for Ollama.

Step 4: Send images through the API

To use a vision model from your own scripts, send an images array with your message to Ollama's local API at http://localhost:11434. Ollama's documentation says the REST API expects base64-encoded image data, while the official Python and JavaScript libraries also accept file paths, URLs or raw bytes.

In Python, using the official ollama library:

from ollama import chat

response = chat(
    model='gemma4',
    messages=[{
        'role': 'user',
        'content': 'What is in this image? Be concise.',
        'images': ['photo.jpg'],
    }],
)
print(response.message.content)

With curl, encode the image as base64 first and send it to /api/chat in the images field, as shown in Ollama's vision documentation. If you already use other tools that expect OpenAI's API format, our guide to Ollama's OpenAI-compatible API explains how to point them at your local server.

Tips for better results

  • Be specific. "List the error messages in this screenshot" works better than "what is this?".

  • Crop out what you do not need. A smaller, focused image gives the model less to get confused by.

  • Try a bigger model for hard tasks. Small models are quick but more likely to miss detail. If a larger model will not fit your card, our best GPU for local LLMs guide covers upgrade options.

  • Keep it private. Everything runs on your PC, which is the main reason to use a local model for personal photos or work documents.

Our verdict

Using a vision model in Ollama is as simple as pulling a model tagged vision, such as Gemma 4, and including an image in your prompt, whether in the app, at the command line or through the images field of the API. Pick a model size your graphics card can hold: the default Gemma 4 is listed at 6.6GB to 9.5GB, while the 26B and 31B versions want a 24GB-class card. These steps follow Ollama's official documentation; Forge PC has not benchmarked the models, so double-check anything that matters.

The bottom line

Using a vision model in Ollama is as simple as pulling a model tagged vision, such as Gemma 4, and including an image in your prompt, whether in the app, at the command line or through the images field of the API. Pick a model size your graphics card can hold: the default Gemma 4 is listed at 6.6GB to 9.5GB, while the 26B and 31B versions want a 24GB-class card. These steps follow Ollama's official documentation; Forge PC has not benchmarked the models, so double-check anything that matters.

Sources

  1. 01Ollama Docs, Vision
  2. 02Ollama Blog, Ollama's new engine for multimodal models (15 May 2025)
  3. 03Ollama Blog, Ollama's new app (30 July 2025)
  4. 04Ollama Library, gemma4
  5. 05Ollama Library, qwen3.8

About the author

Forge PC Staff

Editorial team

The shared byline for articles researched, written and edited by the Forge PC editorial team. How we source, check and correct our work is set out in our editorial policy.

More from Forge PC Staff

Read next

How to Enable AMD Fluid Motion Frames (AFMF 2.1) on a Radeon Graphics Card

How-To

How to Enable AMD Fluid Motion Frames (AFMF 2.1) on a Radeon Graphics Card

To enable AMD Fluid Motion Frames, install Adrenalin 25.3.1 or newer, then select the HYPR-RX profile (RX 7000 and newer) or turn on AMD Fluid Motion Frames under Gaming > Graphics, globally or per game. AFMF 2.1 works on RX 6000 cards and newer in DirectX 11, DirectX 12, Vulkan and OpenGL games, with VSync off; RX 6000 cards need exclusive full screen.

Forge PC Staff2 min

AMD's FSR 4.1.2 Unreal Engine 5 Plugin Brings ML Upscaling to RX 6000 Cards

News

AMD's FSR 4.1.2 Unreal Engine 5 Plugin Brings ML Upscaling to RX 6000 Cards

AMD's FSR Unreal Engine 5 plugin now uses FSR Upscaling 4.1.2, bringing ML-based upscaling to Radeon RX 6000 cards and select Ryzen integrated graphics as well as RX 7000 and RX 9000, in UE 5.3 to 5.8 games that adopt it. It's DirectX 12 only for now and adds Frame Generation 4.0.2 with driver-assisted frame pacing on RX 9000 cards.

Forge PC Staff2 min

NVIDIA's DGX Spark HPC Developer Kit Links Four Sparks Into a 512GB Desk-Side Cluster

News

NVIDIA's DGX Spark HPC Developer Kit Links Four Sparks Into a 512GB Desk-Side Cluster

NVIDIA's DGX Spark HPC Developer Kit, announced at SC26 on 16 November 2026, links four DGX Spark systems in a ring over ConnectX-7 with no network switch, for 512GB of combined memory and 20TB of NVMe storage. NVIDIA says it can spread models of up to 700B parameters across the nodes with NVFP4. No UK kit price yet; Scan sells two-Spark bundles at about £12,845.

Forge PC Staff3 min

Best Free Benchmark Software for PC in 2026: CPU, GPU, SSD and AI Tests

Best Picks

Best Free Benchmark Software for PC in 2026: CPU, GPU, SSD and AI Tests

The best free PC benchmarks in 2026: Cinebench 2026 for CPUs, Unigine Superposition Basic or 3DMark's free Steel Nomad test for graphics cards, CrystalDiskMark for SSDs, Geekbench 7 for cross-platform scores, Geekbench AI for NPUs, Blender Benchmark for creators and OCCT for stability. Run each test twice and download only from official sites.

Forge PC Staff5 min