Skip to main content

Check the capability first

Vision is a per-model capability, and today it is a short list. Read it from the catalog rather than from a model’s name or reputation:
The Vision tag on the Models page covers preview ids too — models whose name is accepted but whose placement is not routed yet. GET /v1/models lists only the routed models, so it is the list to trust when you need image input to actually work today. Document input is narrower still: one routed model reads binary documents.
Sending an image to a text-only model is a 400, not a silent drop. That is deliberate — a request whose image was ignored produces a confidently wrong answer, which is worse than an error.

Sending an image

Both dialects accept data URLs. The Chat Completions dialect also accepts a plain https:// URL in image_url.url — but a base64 data URL is the more reliable choice, because it does not depend on the image being publicly reachable.

Documents and PDFs

The Messages dialect carries binary documents as their own content block:
Where the serving model does not declare capabilities.documents, a document block is not an error — it becomes a deterministic text placeholder, so the turn still completes and the model is told a document was present rather than being handed silence. If the document’s content is what matters, either pin a document-capable model or extract the text yourself and send it as text.

Cost and context

  • Images are input tokens. A large screenshot can be worth thousands of them, drawn from your plan’s included usage at the model’s input rate (Plans & limits). Down-scale before sending — most chart and UI questions are answered as well at 1024px wide as at 4096.
  • Images sit in the conversation. In an agent loop, every subsequent turn re-sends them. On a model that publishes a cached input rate, a stable image prefix is cheaper on repeat; where no cached rate is published, you pay full input every turn (Prompt caching).
  • Budget max_tokens for the answer, not the image. Input size does not consume the output ceiling — but on a reasoning model, thinking does (Reasoning).

Failure modes