Check the capability first
Vision is a per-model capability, and today it is a short list. Read it from the catalog rather than from a model’s name or reputation:
Sending an image to a text-only model is a
400, not a silent drop. That is deliberate —
a request whose image was ignored produces a confidently wrong answer, which is worse than
an error.
Sending an image
https://
URL in image_url.url — but a base64 data URL is the more reliable choice, because it does
not depend on the image being publicly reachable.
Documents and PDFs
The Messages dialect carries binary documents as their own content block:capabilities.documents, a document block is not
an error — it becomes a deterministic text placeholder, so the turn still completes and the
model is told a document was present rather than being handed silence. If the document’s
content is what matters, either pin a document-capable model or extract the text yourself
and send it as text.
Cost and context
- Images are input tokens. A large screenshot can be worth thousands of them, drawn from your plan’s included usage at the model’s input rate (Plans & limits). Down-scale before sending — most chart and UI questions are answered as well at 1024px wide as at 4096.
- Images sit in the conversation. In an agent loop, every subsequent turn re-sends them. On a model that publishes a cached input rate, a stable image prefix is cheaper on repeat; where no cached rate is published, you pay full input every turn (Prompt caching).
- Budget
max_tokensfor the answer, not the image. Input size does not consume the output ceiling — but on a reasoning model, thinking does (Reasoning).