OpenAI Decisions API: Calibrate Thresholds on Labeled Data

On October 6, OpenAI opened the Decisions API in public beta. It is a new endpoint, POST /v1/decisions, that does not generate text at all. You hand it an input and a list of questions, and it returns typed answers: a probability for a yes/no check, a pick from options you define, or a position on an ordered scale. OpenAI says it is about 10x faster than the Responses API, and input costs $0.10 per million tokens with no charge for output or cache.

The part that makes it interesting for builders is that the answers come with probabilities. A normal LLM-as-judge call gives you a word. This gives you a number you can threshold, which means you can measure how often it is wrong and tune the cutoff for your own app, the way you would tune any classifier.