OpenAI Decisions API vs Structured Outputs: When to Use Each

Most LLM features hide at least one classification call: route this ticket, flag this upload, decide whether this message needs a human. The usual way to build it is to ask a chat model for a JSON label with Structured Outputs. OpenAI’s Decisions API, now in public beta, does the same job differently. It returns a probability for each option instead of a generated label, and the choice between the two APIs comes down to whether you need that number.

What the Decisions API returns

The Decisions API is a single endpoint, POST /v1/decisions. You send an input (text, or text with inline base64 images) and a list of named questions. Each question has one of three types:

• predicate: a yes/no condition. The answer is a probability between 0 and 1 that the condition is true.
• choice: pick one of the choices you supply. The answer is the chosen value, a probabilities array across all options, and a confidence field.
• score: rate the input on ordered levels, such as severity from low to critical. The score is a probability-weighted average of the level indices, so it can land between two levels.

Nothing is generated, and the billing reflects that. According to OpenAI’s Decisions guide (opens in a new tab), gpt-6-luna, the only model listed, costs $0.10 per million input tokens, with no charges for output, cache reads or cache writes. The same guide says it returns answers about 10x faster than the Responses API, without giving millisecond figures.

The guide is explicit about what the API is not for. If you need an object that follows your own schema, like extracted fields or a written explanation, use Structured Outputs. If the model needs to call a tool with arguments, use function calling.

Decisions API vs Structured Outputs: the probability is the difference

With Structured Outputs and strict: true, the response always matches your schema. The model can only produce one of your enum values, and you get a valid label every time. What you don’t get is any sense of how close the call was. A message that is plainly about billing and one that’s a 50/50 between billing and shipping come back looking identical.

The common workaround is to add a confidence field to the schema and ask the model to fill it in. That number is generated text, not a measurement. Research on chat models found that they “tend to be overconfident” when they state their own confidence (Xiong et al., ICLR 2024 (opens in a new tab)). That study used 2023 models, but the field is still produced the same way, as text the model writes about its own answer.

A decision model gives you the distribution directly. You can see that a message scored 0.48 billing and 0.44 shipping, and treat that differently from 0.97 billing. Once you have that number, three things that were awkward with a JSON label become straightforward:

• Abstaining. Send anything below a threshold to a person or a bigger model, instead of acting on a coin flip.
• Tuning by cost. Set a stricter threshold where a wrong answer is expensive (auto-closing a ticket) and a looser one where it’s cheap (choosing a queue).
• Monitoring drift. A falling share of high-probability answers can be an early sign that inputs have changed, before anyone notices misrouted tickets.

A routing request with a fallback option

The shape matters here, so this is the one example worth showing. One request can carry several independent questions about the same input:

curl https://api.openai.com/v1/decisions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-6-luna",
    "input": "I was charged twice for my order and the parcel never arrived.",
    "questions": [
      {
        "name": "department",
        "type": "choice",
        "instructions": "Which team should handle this customer message?",
        "choices": [
          { "value": "billing", "description": "Charges, refunds, invoices" },
          { "value": "shipping", "description": "Delivery status, lost or damaged parcels" },
          { "value": "account", "description": "Login, password, profile changes" },
          { "value": "other", "description": "Anything the other teams do not cover" }
        ]
      },
      {
        "name": "urgent",
        "type": "predicate",
        "instructions": "The customer has lost money or cannot use the product right now."
      }
    ]
  }'

Two details in it come straight from how classification fails in practice. The other choice is there because, without it, every message the categories don’t cover gets forced into the nearest wrong one. That’s how a complaint about a data breach ends up in the shipping queue. OpenAI’s guide recommends the same fallback.

The example message is also deliberately ambiguous, since it’s both a billing and a shipping problem. A JSON label picks one and hides the tie. The probabilities array shows it, and your code can route the message to both teams or to a person.

Check each answer’s type before reading it. Answers can come back as "refusal", and code that assumes a choice field will throw on the first one.

Set thresholds from your own labelled data

OpenAI’s guide tells you to pick thresholds “based on the cost of false positives and false negatives”. It doesn’t say the probabilities are calibrated, meaning that answers at 0.8 are right about 80% of the time. Cloudflare describes its own Clef decision models as calibrated, but it publishes no calibration data either. Treat calibration as something you measure.

That takes a few hundred real inputs with correct labels, not a research project. Run them through the API and group the answers by probability. If the 0.9-and-above group is right 98% of the time and the 0.6–0.9 group only 75%, the threshold is a business decision you can now make with numbers. The band below it is your abstain zone. Routing that band to a stronger model is a model cascade, with the escalation signal measured instead of guessed.

Rerun the same set whenever the model behind the endpoint changes. Thresholds tuned on one model don’t carry over to the next, for the same reasons prompts don’t. A regression suite pinned to your current behaviour catches the shift before your users do.

What it costs next to a JSON label

Take an illustrative million customer messages a month at 300 tokens each, so 300 million input tokens. Count only the message, since instructions and choice descriptions add to every request. With the Decisions API on gpt-6-luna, that’s $30. The same model on the Responses API, returning a 20-token JSON label, costs $30 of input plus $10 of output, so $40. On Cloudflare’s Clef-flash (opens in a new tab), at $0.038 per million input tokens, it’s about $11. On a mid-tier model such as GPT-6.1 Sol ($2 input, $10 output per million (opens in a new tab)), it’s about $800.

So at the cheapest tier, price isn’t the reason to switch: the probabilities and the latency are. The large saving comes when a team is classifying with a general mid-tier model because it was already wired in. That’s a common setup, and at volume it’s a line on the bill worth checking.

When Structured Outputs or function calling is still the right tool

Keep Structured Outputs when the output has more than one field, or any free text: an extracted address, a summary plus a category, a reason a reviewer will read. The Decisions API can’t write anything, and a classification you need to audit later is better with a one-line justification beside it. Keep function calling when the model is choosing an action with arguments, not a label. And keep a chat model when the set of categories isn’t fixed: a decision model can only pick from options you wrote down.

The beta caveats are real. One model is listed. There are no documented limits on questions, choices or input size. Images must be inline data URLs, not hosted links. Field names can change before general availability, which OpenAI says is weeks away.

Pick by what you’ll do with the answer

Use the rule this way. If your code branches on the answer and a wrong branch has a cost, use a decision model and a threshold you measured. If a person or another system reads the answer, use Structured Outputs. If the model takes an action, use function calling.

Start with the classification call that runs most often. Move it to the Decisions API behind a feature flag, label a few hundred of its real inputs, and set the threshold from that data. Then log every answer’s probability next to the outcome. Within a week you’ll know how many of those calls were close, which the JSON label never showed you.

Sources

• OpenAI Decisions guide (opens in a new tab)
• OpenAI Structured Outputs guide (opens in a new tab)
• OpenAI API pricing (opens in a new tab)
• Cloudflare: Clef decision models, faster and cheaper (opens in a new tab)
• Xiong et al., Can LLMs Express Their Uncertainty? (arXiv 2306.13063) (opens in a new tab)