If the response is going to be read by a person, asking “respond in JSON” can work well. If it goes directly to a parser, database, queue, spreadsheet, API or automation, the question changes: which contract does your code need to receive? In 2026, OpenAI, Claude and Gemini offer native structured output mechanisms with JSON Schema, but this does not make prompt, free JSON and schema equivalent.
The rule of thumb is simple. The textual instruction attempts to guide the format. A JSON mode mainly takes care of the syntax. Structured Outputs tries to impose the shape of the object. None of these layers alone prove that the content is correct.
There are three different problems hidden in the phrase “I want JSON”
The first problem is the appearance of the answer. You want the template to not place an introduction, Markdown, comments, or explanations around the object. A prompt like “respond only in JSON” is often helpful, but it remains a behavioral instruction. The model can still change field names, omit properties, change types or choose a similar format, but incompatible with your code.
The second problem is syntactic validity. Here the question is: can the string be parsed as JSON without error? OpenAI still documents a separate, older JSON mode that guarantees valid JSON, but not the desired schema. This eliminates a class of parsing errors, but does not prevent the response from coming with the wrong fields.
The third problem is compliance with a contract. In this case, it is not enough to receive a valid object; it needs to contain the expected properties, the correct types, and, when applicable, enum-bound values. That's what Structured Outputs and equivalent mechanisms based on JSON Schema are for.
The most important difference is this: valid format is not the same as valid structure, and valid structure is not the same as true response.
The same extraction case shows where each approach breaks down
Imagine a system that receives a customer service message and needs to extract five fields: customer name, intention, priority, need for return and telephone number. The intent can only be “upgrade”, “cancellation” or “support”. Priority can only be “low”, “medium” or “high”. The phone may be null when not present.
With textual instruction, you can describe this format in the prompt and even provide an example. In many cases the model will comply. But in one execution, it may use “client” instead of “client_name”; in another, it can transform priority into number; in another, you can add an explanation outside of JSON.
With valid JSON, the parser is no longer the main problem. Still, nothing stops a perfectly parsable object like {"urgent": true, "category": "upgrade"} when your code expected another set of fields.
With JSON Schema, the contract becomes part of the generation configuration itself. The model is restricted to the set of keys, types and values allowed by the schema, within the capabilities of each provider. This drastically reduces integration errors, but does not prevent a semantic error. The model can still choose “support” when the text indicates “upgrade”, or fill in a plausible value in a field that should be null.
This separation between format error and content error is the point that most prevents false confidence.
OpenAI: JSON mode resolves parsing; Structured Outputs solves the contract
In the OpenAI API, Structured Outputs can be used to structure the final response or to restrict function calling arguments. Current documentation recommends Structured Outputs over JSON mode whenever the model and use case support it.
In JSON mode, the central guarantee is receiving valid JSON. The schema remains the responsibility of the application. If you expect specific fields, you need to validate the object later and decide what to do when there is a missing key, wrong type, or value outside the expected domain.
In Structured Outputs, OpenAI uses a subset of JSON Schema and imposes its own rules. Objects must block extra properties with additionalProperties set to false, and all fields must be set to required. When a field is conceptually optional, the output may model it as required but accepting null. The root object also has specific restrictions, such as it cannot be an anyOf at the top.
This means that a schema that works in another library or provider may need adaptation before being submitted to OpenAI.
There are still two operational exceptions that the code needs to handle. A security refusal may come outside the expected schema, on a specific refusal signal. And a response interrupted by a token limit may be incomplete. Structured Outputs eliminates the need for retries for normal schema violations, but does not turn every API call into a usable business object.
Claude: structured JSON output and strict tool use are different controls
Anthropic today offers Structured Outputs in Claude on two fronts. For the final response, the API uses JSON outputs through output_config.format. For tool calls, strict: true restricts parameters to the tool schema.
This distinction is useful because it answers two different problems. If the model needs to return an object to your application, you structure the final output. If it needs to trigger a function, you make the tool arguments strict. In agentic flows, both mechanisms can be used together.
Claude's documentation also shows why “JSON Schema” does not automatically mean “the entire JSON Schema specification”. Some constraints are not accepted directly by the generation mechanism. SDKs can simplify schemas generated by Pydantic or Zod, move certain constraints to descriptions, and then validate locally against the original schema. Thus, the application still needs to know which part was actually imposed during generation and which part was checked afterwards.
There are also explicit exceptions. A security refusal may end the response with refusal stop_reason and not comply with the schema. A max_tokens can clip the output. And the documentation itself notes an unusual caveat for string enums: capitalization may vary in some cases. If your values only differ by cash, that's weak contract design.
There is also an important portability trap: in the compatibility layer with the OpenAI SDK, Anthropic informs that strict in function calling is ignored. To ensure compliance in Claude, the recommended path is the native API with Structured Outputs.
Gemini: schema reduces ambiguity, but does not replace business validation
The current Gemini documentation allows you to configure structured outputs with JSON Schema, using a response in application/json and an associated schema. The result is generated as syntactically valid JSON that matches the format accepted by the provider implementation.
Gemini supports a subset of JSON Schema. Types such as string, number, integer, boolean, object, array, and null are supported, as well as useful features such as required, enum, numeric limits, and array properties on certain API surfaces. Not every feature of the specification is supported, and schemas that are too large or too nested may be rejected.
A particularly important detail appears in Google's own best practices: structured output guarantees the form, not the semantic correctness of values. The recommendation is to validate the final output in the code before using it in business rules.
This avoids a common architectural error. An enum can prevent the model from writing “very urgent” when the contract only accepts “low”, “medium” and “high”. But it does not prevent the model from choosing “high” when the message should be classified as “medium”. The object can be flawless for the parser and still be wrong for the product.
Portability is not about copying the same schema between three APIs
The three providers have converged around JSON Schema as the contract language, but have not converged on all implementation details.
In OpenAI, there are requirements such as required for all fields and additionalProperties false. In Claude, there is its own set of resources and complexity limits, as well as transformations that some SDKs do automatically. In Gemini, the documentation also explicitly talks about specification subsetting and possible rejections of complex schemas.
Therefore, the most robust strategy for a multi-provider application is to maintain a canonical data model within your code and create a thin adaptation layer for each API. The domain should say what “customer”, “priority” or “result” mean. The adapter must translate this contract into the subset accepted by the provider.
This also prevents details of an API from being leaked to the entire system. If tomorrow a provider changes the parameter name, expands JSON Schema support or changes a limitation, you modify the adapter and preserve the rest of the application.
The more portability matters, the better it is to prefer simple schemas: explicit objects, clear types, stable enums, few levels of nesting and a consistent strategy for missing values.
Missing field, refusal and “I don’t know” need to be drawn
Strict schemas create a consequence that is not always obvious: if the format requires a response, the model will attempt to fill the format.
OpenAI itself warns that, when user input does not allow producing a valid response for the schema, the model may end up hallucinating a value in order to satisfy the structure. The solution is not to abandon the schema; is to design the contract to represent absence of information.
If a phone may not exist, use null as a possibility. If a classification can be indeterminate, consider an explicit value to be “indeterminate” as long as it has real meaning in the domain. If the task could fail due to lack of evidence, represent that state in a field of its own rather than forcing the model to choose an answer that appears complete.
Security denials are another case. They should not be mixed with “missing data”. OpenAI and Claude expose their own opt-out signals that can replace normal structured output. The application must detect this state before attempting to consume the business object.
The same goes for garbled responses, network errors, and API limits. A parser is not a fault handling system.
Enums fix the vocabulary, not the decision
Enums are one of the biggest practical advantages of Structured Outputs because they remove variations that often break automation.
Without schema, a priority classification may come as “high”, “urgent”, “high”, “P1” or “critical”. With enum, you force the contract to a set of values that the code knows. This simplifies database, metrics, filters and conditional rules.
But enum is not a true checker. If the allowed values are “pass” and “fail”, the schema ensures that the response uses one of them; does not guarantee that the decision is correct. Therefore, classification tasks still need examples, class definitions, tests and, when risk requires, review or deterministic rules.
In Claude, there is also the documented caveat about capitalization of some enum values. In multi-provider integration, normalizing case on the application side and avoiding categories that are only distinguished by upper and lower case reduces surprises.
Subsequent validation remains mandatory
Structured Outputs greatly reduces the work of structural validation, but a production application still needs to validate in layers.
First: confirm the response status
Before analyzing the payload, check whether the call ended normally. Refusal, truncation, blocking, timeout and transport error must follow an error flow, not the same path as valid data.
Second: validate the local contract
Even when the provider promises compliance, validating the object with the same library that defines its types helps protect against SDK changes, integration differences, and transformed schemas. Pydantic, Zod and equivalents fulfill this role well.
Third: validate the semantics
This is where rules come in that JSON Schema cannot decide on its own. Does a date need to be in the allowed period? Does an identifier need to exist in the bank? Does the sum need to close? Is a category chosen by the model compatible with the source text? These checks remain the responsibility of the application.
This distinction also appears in research on structured generation. JSONSchemaBench, presented in 2025, brought together around 10 thousand real schemas and compared different constrained decoding mechanisms, including OpenAI and Gemini. The work found differences in coverage, efficiency and behavior between implementations. The practical lesson is not that schema “doesn't work”, but rather that structural compliance needs to be tested in the environment and set of schemas that you actually use.
When just writing “reply in JSON”
Textual instruction is sufficient when the JSON is merely a presentational convenience, the response will be reviewed by a human, or the consumer tolerates variations in structure.
It can also be acceptable in rapid prototypes, in tasks without further automation, or when you are still discovering which fields should actually exist. At this stage, a rigid contract can freeze a bad structure too soon.
The cost of this flexibility is clear: if the content goes into code, you have to assume that any part of the format may vary.
When to use free JSON or a JSON mode
A mode that guarantees valid JSON is useful when the first requirement is to eliminate parsing failures, but the framework is still open or legacy.
In OpenAI, JSON mode continues to exist precisely as this most basic layer. It can make sense when Structured Outputs are not available in the chosen model or integration, or when you just want to ensure a parsable object and want to validate the shape later.
For new applications that already know the contract, this intermediate layer usually brings less value: you continue writing format prompts and maintaining schema validation on the outside.
When to use JSON Schema
Use schema imposed by the API when the result is consumed automatically and the shape of the object is part of the application contract.
This is especially important in data extraction, sorting with enums, filling out forms, generating records, composing interfaces and any flow in which a wrong key generates an execution error or corrupted data.
Schema also helps reduce accidental prompt engineering. Instead of spending several lines repeating “don’t add fields”, “use these exact names”, and “don’t put text outside of JSON”, you transfer some of that control to a mechanism designed to restrict the output.
Still, keep instructions about the task. The schema tells how the response should be represented; the prompt continues to tell the model what to understand and decide.
When the tool is the right contract
If the objective is not to return data to the user, but to perform an action, function calling or tool use is usually the most appropriate interface.
OpenAI and Claude explicitly differentiate structured response output and tool calls with typed parameters. Gemini also treats structured outputs and function calling as distinct use cases. The architectural pattern is useful: structured response to represent information; tool calling to ask the system to do something.
Mixing the two roles often leads to confusing APIs. An object called “action” in the final response does not perform anything by itself. A tool call, on the other hand, is part of a loop in which the application decides whether to execute, validates arguments, returns results and continues the conversation.
A robust design for OpenAI, Claude and Gemini
Start with the data type of your application, not the syntax of a specific API. Define fields, enums, nullability and business rules in your domain. Then generate or adapt the schema for each provider.
Treat the schema as an interface, not as a replacement for the prompt. Use property descriptions to reduce ambiguity, write clear instructions about extraction or sorting, and include examples only when they help you decide the content.
Separate business states from execution states. “Missing phone” belongs to the payload. “Model refused”, “response truncated” or “API failed” belong to call control.
Validate locally before persisting or executing. If the object triggers a transaction, an account change, or any irreversible operation, add deterministic rules and authorizations outside the model.
And test the same set of inputs on the providers you intend to support. A schema accepted by one is not proof that another will accept the same keywords, depth or combination of constraints.
The right choice depends on the consumer, not the beautiful shape
“Reply in JSON” is an instruction. JSON mode is a syntax guarantee. Structured Outputs is a stronger structural contract. The difference only becomes clear when you look at who will consume the response.
If the consumer is a person, flexibility can be an advantage. If it is code, predictability becomes more valuable. If the output triggers business rules, even perfect JSON is not enough without semantic validation.
In production, the goal is not to receive an object that looks organized. It is receiving data that the parser understands, the contract accepts and the system knows when to reject.
