Reliable JSON from a local model: schema-constrained output with Ollama and Pydantic

Ask a local model for JSON in plain English and you will eventually get a markdown fence, a friendly preamble, or a trailing comma. Ollama can instead constrain the output to a JSON schema, so the reply always has the shape you declared. Pair that with Pydantic and you get typed data out of a model running on your own machine. This post walks through the setup from Ollama's structured-outputs documentation, then adds the validation loop you still want in production.

The idea

Ollama's chat API accepts a format field. Instead of the string "json", you can pass a full JSON schema. The model's output is then restricted to text that matches it. Because the schema comes from a Pydantic model, the same class that defines the shape also validates the reply.

Setup

You need a running local Ollama server, the Python client and Pydantic, and a model pulled. The docs use gpt-oss for text; any model you have pulled works the same way.

pip install ollama pydantic
ollama pull gpt-oss

Plain HTTP first

The raw API call shows what is really happening. The schema goes straight into format:

curl -X POST http://localhost:11434/api/chat -H "Content-Type: application/json" -d '{
  "model": "gpt-oss",
  "messages": [{"role": "user", "content": "Tell me about Canada."}],
  "stream": false,
  "format": {
    "type": "object",
    "properties": {
      "name": {"type": "string"},
      "capital": {"type": "string"},
      "languages": {
        "type": "array",
        "items": {"type": "string"}
      }
    },
    "required": ["name", "capital", "languages"]
  }
}'

The same thing with Pydantic

In Python you rarely write that schema by hand. Define a model and pass its schema:

from ollama import chat
from pydantic import BaseModel

class Country(BaseModel):
    name: str
    capital: str
    languages: list[str]

response = chat(
    model='gpt-oss',
    messages=[{'role': 'user', 'content': 'Tell me about Canada.'}],
    format=Country.model_json_schema(),
)

country = Country.model_validate_json(response.message.content)
print(country)

model_validate_json parses and validates in one step. If the content does not fit the model, Pydantic raises a ValidationError instead of handing you a half-right dictionary.

Two tips from the docs

• Put the schema in the prompt too. The documentation recommends including the JSON schema as a string in the prompt as well as in format, to ground the model's response. The constraint controls the shape; the prompt helps the model understand what each field means.
• Use temperature 0. Setting it to 0 makes completions more deterministic, which is what you want for extraction.

Here is a helper that applies both:

import json
from ollama import chat
from pydantic import BaseModel, ValidationError

def extract(model_cls, text, model='gpt-oss'):
    schema = model_cls.model_json_schema()
    prompt = (
        'Extract the data from the text below. '
        'Reply only with JSON matching this schema:\n'
        + json.dumps(schema)
        + '\n\nText:\n' + text
    )
    response = chat(
        model=model,
        messages=[{'role': 'user', 'content': prompt}],
        format=schema,
        options={'temperature': 0},
    )
    return model_cls.model_validate_json(response.message.content)

class Invoice(BaseModel):
    vendor: str
    total: float
    currency: str

print(extract(Invoice, 'Invoice from Acme Ltd, total due 1,250.00 EUR.'))

A validation loop

Constrained output makes the shape reliable, not the meaning. The model can still put the wrong number in a valid field, and a schema that is too loose will accept nonsense. Keep a validation and retry layer, and add your own checks beyond types:

def extract_checked(model_cls, text, attempts=3, **kwargs):
    last_error = None
    for _ in range(attempts):
        try:
            result = extract(model_cls, text, **kwargs)
            return result
        except ValidationError as err:
            last_error = err
    raise RuntimeError(f'No valid output after {attempts} attempts: {last_error}')

With temperature 0 a retry of the identical prompt will often return the same answer, so in practice change something on the retry: append the validation error to the prompt, or fall back to a larger model. Add semantic checks in Pydantic itself, for example Field(gt=0) on a total or a Literal for a fixed set of values.

Vision works the same way

Vision models accept the same format parameter. The documentation's example describes an image into a typed structure:

from ollama import chat
from pydantic import BaseModel
from typing import Literal, Optional

class Object(BaseModel):
    name: str
    confidence: float
    attributes: str

class ImageDescription(BaseModel):
    summary: str
    objects: list[Object]
    scene: str
    colors: list[str]
    time_of_day: Literal['Morning', 'Afternoon', 'Evening', 'Night']
    setting: Literal['Indoor', 'Outdoor', 'Unknown']
    text_content: Optional[str] = None

response = chat(
    model='gemma4',
    messages=[{
        'role': 'user',
        'content': 'Describe this photo and list the objects you detect.',
        'images': ['path/to/image.jpg'],
    }],
    format=ImageDescription.model_json_schema(),
    options={'temperature': 0},
)

image_description = ImageDescription.model_validate_json(response.message.content)
print(image_description)

Note the Literal fields: they turn open-ended answers into a closed set, which is the easiest way to make downstream code simpler. Treat the confidence value as the model's guess, not a calibrated probability.

Caveats

• Ollama cloud. The documentation says Ollama's cloud does not currently support structured outputs, so this setup needs a local model.
• Small models. A schema guarantees syntax, not understanding. Test your extraction on a handful of real examples and count how many fields are right.
• Schema design. Keep schemas flat and give fields clear names. Optional fields with defaults, like text_content above, stop the model from inventing values for things that are not there.
• OpenAI-compatible API. The documentation notes structured outputs also work through Ollama's OpenAI-compatible endpoint using response_format, if you already have code written against that client.

Checklist

1. Define the output as a Pydantic model.
2. Pass Model.model_json_schema() as format.
3. Include the schema in the prompt as a string.
4. Set temperature to 0.
5. Parse with model_validate_json, and retry with changed input on ValidationError.
6. Measure field accuracy on real samples.

Sources

• Ollama documentation: Structured outputs