# Jev vs structured outputs: when to use each

> Compare Jev, JSON mode, and schema-constrained LLM outputs. Learn when a typed answer is enough and when decision probabilities justify another model.

Author: [Flavio Copes](https://flaviocopes.com/about/) | Published: 2026-10-21 | Topics: [AI](https://flaviocopes.com/tags/ai/) | Canonical: https://flaviocopes.com/jev-structured-outputs/

A language model can return a category in JSON. Jev can return a category too. That overlap makes it reasonable to ask why an application would need both.

I would start with the answer the application needs. If a model is already reading a support ticket to draft a reply, adding a category to its structured response may be enough. If the application repeatedly needs a small decision and a distribution over its possible answers, [Jev](https://flaviocopes.com/jev/) deserves a separate evaluation.

The useful comparison includes the work around the call. An extra model means another integration to maintain. A general model that writes a long reply for every tiny routing decision can create unnecessary work too.

We're going to use a hypothetical support inbox to compare the approaches. The examples illustrate contracts and application policy. They are not measurements of either model's accuracy or speed.

## JSON mode and structured outputs solve different problems

Consider this response:

```json
{"department":"billing"}
```

It is valid JSON. That tells us a parser can read it. It doesn't tell us whether `department` is a supported property or whether `billing` is an allowed value.

A different valid JSON response could contain `{"team":"accounts"}`. Your application might have no handler for it.

**JSON mode** asks a model to produce JSON. **Schema-constrained structured outputs** additionally constrain the response to a declared shape, within the provider's supported schema features. OpenAI's [structured output guide](https://developers.openai.com/api/docs/guides/structured-outputs) makes this distinction explicitly.

For a routing answer, the schema could look like this:

```json
{
  "type": "object",
  "properties": {
    "department": {
      "type": "string",
      "enum": ["billing", "technical", "other"]
    }
  },
  "required": ["department"],
  "additionalProperties": false
}
```

The schema says which answers the application accepts. The prompt still needs to explain what those categories mean.

For instance, a request to change a payment card belongs to billing. An error while opening the payment settings page could belong to technical support. The field name alone doesn't settle that boundary.

You also need to handle unsuccessful requests and provider-specific refusal or incomplete-response states. A schema describes the successful answer; it doesn't remove the rest of the API lifecycle.

## A correct shape can contain a wrong decision

Suppose the customer writes:

> I was charged twice, and now I can't sign in to download either receipt.

Every value in our enum is structurally acceptable. Choosing `technical` would pass schema validation. Whether it is the right route depends on the support policy.

Perhaps billing should own duplicate-charge complaints and coordinate access recovery. Perhaps your system assigns an initial owner and allows more than one issue tag. Decide that before comparing models.

Otherwise, a disagreement can mean your instructions are ambiguous rather than that one model failed.

I would write down how the team handles mixed requests, messages containing no question, and requests outside its scope. Those cases become evaluation examples. Changing from JSON mode to structured outputs helps with malformed answers. It doesn't write the routing policy for us.

## What Jev adds to the contract

[Jev's Choice primitive](https://docs.typesafe.ai/primitives/choice) takes a set of options and descriptions. Its answer includes the selected option and a probability for every option. Choice also includes a confidence value derived from that distribution.

Here is a request body for the same inbox, using the [TypeSafe evaluation endpoint](https://docs.typesafe.ai/api):

```json
{
  "model": "jev-latest",
  "state": {
    "message": "I was charged twice, and now I can't sign in to download either receipt."
  },
  "questions": {
    "department": {
      "type": "choice",
      "instructions": "Choose the initial owner. Billing owns payment disputes even when the customer also reports an access problem.",
      "criteria": {
        "billing": "Payments, duplicate charges, invoices, and refunds",
        "technical": "Application failures without a payment dispute",
        "other": "Requests outside these categories or without enough information"
      }
    }
  }
}
```

If you want to run that request, save it as `routing.json` and use the credentials from the [Jev API setup guide](https://flaviocopes.com/jev-api-key/):

```bash
curl --fail-with-body https://api.typesafe.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" \
  -H 'Content-Type: application/json' \
  --data-binary @routing.json
```

Set the key in your terminal environment first. Keep it on the server, outside browser JavaScript and source control.

A Choice gives the application more information than a single winning label. A result narrowly split between billing and technical support can be handled differently from a clear billing result. The model still needs to make the right judgment.

Jev also has other typed questions, explained in the [main tutorial](https://flaviocopes.com/jev/). We only need Choice for this comparison.

## A generated confidence field is a separate design choice

You could add a numeric `confidence` field to an LLM schema. That constrains the output to contain a number. It doesn't establish how that number relates to real mistakes.

Jev's [confidence documentation](https://docs.typesafe.ai/confidence) defines Choice confidence as a calculation from the returned probabilities. It measures how far the top option is above an even split, adjusted for the number of options.

That is why I wouldn't compare `confidence: 0.9` from an arbitrary LLM prompt with Jev's `confidence: 0.9` as if they were the same measurement.

Nor would I treat either as proof that an action is safe. A model can be confidently wrong. A missing category can force it to prefer an unsuitable answer. Poor instructions can consistently produce the wrong policy.

I would evaluate the score against labeled cases from the actual workflow. If high-confidence billing assignments often need manual correction, raising a threshold alone may not fix the underlying question.

Adding an `other` category helps represent out-of-scope messages. Uncertainty and category membership remain different things. A model can be certain that a message belongs outside the supported departments.

## Put the action policy in ordinary code

The model should propose a department. Application code decides whether to assign it automatically.

For illustration, suppose our trial only allows automatic routing when the chosen option has at least 0.9 probability. Everything else goes to review. That threshold is invented for the example, not a recommendation derived from a production test.

A small policy function could be:

```javascript
export const route = answer => {
  const allowed = ['billing', 'technical', 'other']
  if (!allowed.includes(answer.choice)) return 'review'
  const probability = answer.probabilities?.[answer.choice]
  if (!Number.isFinite(probability)) return 'review'
  if (probability < 0.9 || probability > 1) return 'review'
  return answer.choice === 'other' ? 'review' : answer.choice
}
```

Pass one question's answer to it, as `route(response.answers.department)`.

This is the policy layer after validating the complete API response. It doesn't replace schema validation or verification that every probability is well formed.

Its input can also come from a different classifier with an appropriately evaluated probability contract. Keeping this function separate makes it possible to test routing without making a model call every time.

An API timeout should create a review item or leave the ticket waiting. It should never silently become the default department. Log the failure separately from a valid but uncertain answer so you can investigate each problem.

Routing into a support queue is a reversible action. Issuing a refund is a different permission. Even a very confident billing label doesn't authorize payment operations.

## When I would keep the existing LLM

Suppose the inbox already asks a language model to summarize a message and draft a response. The output includes a department, a short summary, and the proposed reply.

I would first test a structured response from that existing call. It keeps the integration small and uses the context the model already needs to produce the reply.

An extra classification request may add little if a human reviews every draft anyway. It can also fail independently, making a workflow wait for a decision that was only useful as a suggestion.

A general LLM is also useful when the answer includes arbitrary text from the message. Extracting a customer-provided reference and drafting a reply calls for different output than selecting a department from a fixed list.

Jev has documented extraction patterns, but I wouldn't introduce a second model merely to recreate an extraction workflow that already works well. The replacement needs an observable benefit.

For a small internal tool, the simplest useful result may be an enum with human correction. Start there if it meets the requirements.

## When I would evaluate a decision model

Now imagine most incoming messages don't need a generated reply. The application only needs to route them, detect whether a person should review them, and choose a processing path.

A narrow decision model fits that boundary more naturally. I would compare it against the existing implementation using the same messages and the same label descriptions.

I would also consider it when the application can use the full distribution. Perhaps ambiguous messages go into a shared queue while clear requests go directly to a specialist. That is a concrete use for uncertainty beyond displaying a confidence badge.

The reason to add Jev would be a useful improvement in the complete workflow. It could involve operating cost, response time, or fewer incorrect automatic assignments. I would measure those outcomes rather than assuming the specialized model wins every task.

Separate the cases where it should abstain from the cases it should answer. A system that routes everything automatically can look convenient while sending difficult work to the wrong people.

## Compare coverage as well as accuracy

I would assemble examples that reflect the inbox: short messages, mixed complaints, unfamiliar product names, and requests the team doesn't handle. Avoid sending private customer data to a new provider without permission; sanitized or synthetic cases can establish the initial test.

Label the examples using the written policy. Where reviewers disagree, resolve the policy or mark the ambiguity rather than forcing a convenient expected answer.

Then measure how many cases each system handles automatically and how many of those assignments are correct. That first number is automatic coverage. Reporting correctness without coverage hides the possibility that the system only handles a handful of easy messages.

For example, compare both approaches at the same acceptable error rate. One may require much more human review to stay within it. Alternatively, compare errors at the same coverage. Either view is more useful than comparing raw confidence values.

Keep a separate set of examples for the final check after choosing prompts and thresholds. Otherwise, repeated adjustments can make the system look good on the examples you used to design it.

Measure end-to-end response time too. Include validation and retries. A fast successful model call doesn't describe a workflow that frequently needs a fallback call.

## Migration can start in the background

If an existing structured-output implementation works, I would run a candidate decision model without letting it assign tickets. Record disagreements and inspect them.

Keep the current route visible to the team. Add the candidate answer and enough information to reconstruct the question, including the model identifier returned by the API. Avoid retaining message content longer than the workflow needs.

Review a disagreement against the policy and the message itself. Either implementation may have made the mistake.

After evaluating the results, enable automatic routing for a narrow category with a clear fallback. Expand only when the review workload and error rate justify it.

For this inbox, I would keep the general model responsible for writing replies and decide separately whether Jev improves routing. Each component needs to earn its place through the work it removes.