Structured Output & Tools

Free-text is for humans. Machines need schemas. This reel turns LLM output from prose into data.

▶ Watch this reel

What you'll learn

JSON / structured outputs
Function / tool calling
Pydantic + LLM end-to-end
Prompt caching
Multimodal inputs

Remember this

response_format + JSON Schema guarantees shape — Pydantic is the single source of truth
Tool calling = model proposes, your code disposes — the atomic unit of agents
Stable prompt prefix earns cache discounts; multimodal input + structured output = document superpowers

Structured outputs

Pass a JSON Schema as response_format → the API constrains generation to valid schema.
Loop: Pydantic model → model_json_schema() → API → Model.model_validate_json().
Guarantees shape, not truth — value checks still need validators + grounding.

Function / tool calling

1. Register tools: name + description + parameter schema. 2. Model replies with a structured call request (tool_calls). 3. Your code executes — validate args (Pydantic), authorize, run. 4. Append result as a role: "tool" message; model continues.

Model proposes, code disposes. The atomic unit of agents (Stage 6).

Pydantic end-to-end

One class: schema export · constraint · parse · value validation · test fixtures.

Prompt caching

Stable byte-identical prefix (system, docs, examples) → cached at a steep discount.
Static first, dynamic last — a deliberate cost architecture.

Multimodal inputs

Images (base64/URL) and PDFs ride along as message parts.
Multimodal in + structured out collapses OCR/extraction pipelines into one schema'd call.
Verify critical numbers on high-stakes documents.

Code: Structured output + tool calling, end to end

from pydantic import BaseModel, Field
from openai import OpenAI

client = OpenAI()

# --- 1. Structured output ---------------------------------------
class Invoice(BaseModel):
    vendor: str
    total: float = Field(ge=0)
    date: str

resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Extract the invoice from this text: 'Acme Corp — total $1,240.00 — 2026-03-01'"}],
    response_format={
        "type": "json_schema",
        "json_schema": {"name": "invoice", "schema": Invoice.model_json_schema()},
    },
)
invoice = Invoice.model_validate_json(resp.choices[0].message.content)

# --- 2. Tool calling ---------------------------------------------
def get_weather(city: str) -> str:
    return f"Sunny, 22°C in {city}"          # your real code runs here

weather_tool = {
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}

r = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Weather in Paris?"}],
    tools=[weather_tool],
)
call = r.choices[0].message.tool_calls[0]
args = eval(call.function.arguments)          # in prod: json.loads + validate
result = get_weather(**args)
print(result)                                  # → 'Sunny, 22°C in Paris'