You have a folder of receipt or product-label photos and you want rows, not prose. Ollama's structured outputs let a vision model answer in JSON that matches a schema you give it, so the output can go straight into Pydantic validation. This post builds that as a small script you can run on a laptop.
One constraint up front: Ollama's documentation states that its cloud does not currently support structured outputs, so this works with a model running locally.
What the feature does
The Ollama structured-outputs page says vision models accept the same format parameter as text models, which gives deterministic descriptions of images. You pass a JSON schema as format, attach images to the message, and the reply content is JSON constrained to that schema. The docs also advise lowering the temperature, for example to 0, for more deterministic completions.
Constrained output guarantees shape, not truth. A model can return a perfectly valid receipt with a wrong total. So the pattern has three layers: schema-constrained generation, Pydantic validation, and your own sanity checks.
Setup
pip install ollama pydantic
ollama pull gemma4
Any vision-capable model you have pulled will do; the model name in the script is the only thing to change. Smaller models read blurry photos worse, so test on your own images before trusting a number.
The schema
Describe the record you want. Keep it flat and small: every extra field is another chance for the model to guess.
from typing import Optional
from pydantic import BaseModel, Field, field_validator
class LineItem(BaseModel):
description: str
quantity: float = Field(ge=0)
price: float = Field(ge=0)
class Receipt(BaseModel):
merchant: str
date: Optional[str] = None
currency: Optional[str] = None
items: list[LineItem]
total: float = Field(ge=0)
@field_validator("items")
@classmethod
def at_least_one_item(cls, v):
if not v:
raise ValueError("no line items found")
return v
The extraction script
The script sends one image, asks for the schema, and validates the reply. If validation fails it retries once with the error message added to the prompt, then gives up and reports the file.
import sys
from pathlib import Path
from ollama import chat
from pydantic import ValidationError
from schema import Receipt # the models above, saved as schema.py
MODEL = "gemma4"
PROMPT = (
"Read this receipt. Return merchant, date, currency, every line item "
"and the total exactly as printed. Use null for anything you cannot read."
)
def extract(path: Path, retries: int = 1) -> Receipt:
prompt = PROMPT
last_error = None
for _ in range(retries + 1):
response = chat(
model=MODEL,
messages=[{"role": "user", "content": prompt, "images": [str(path)]}],
format=Receipt.model_json_schema(),
options={"temperature": 0},
)
try:
return Receipt.model_validate_json(response.message.content)
except ValidationError as err:
last_error = err
prompt = PROMPT + "\nYour previous answer was invalid: " + str(err)
raise ValueError(f"{path.name}: {last_error}")
if __name__ == "__main__":
for name in sys.argv[1:]:
try:
receipt = extract(Path(name))
print(receipt.model_dump_json())
except ValueError as err:
print("FAILED", err, file=sys.stderr)
Run it over a folder and collect one JSON object per line:
python extract.py receipts/*.jpg > receipts.jsonl
