Choosing a model for working with tools is not the same as choosing the best model for chatting, programming, or answering questions. In a real flow, it needs to get three different things right: when to use a tool and which one to use, what arguments to send, and what to do when execution doesn't go as expected.
As of August 2026, the most useful public evidence points to a clear but not absolute answer: Claude Opus 5 is today the strongest starting point for tooled enterprise flows. In the official AutomationBench, run on private tasks and evaluated by the final state of the environment, it leads with a relevant margin. GPT-5.6 Sol and Gemini 3.6 Flash remain important alternatives, especially when architecture, cost, or platform-specific features outweigh gross completion rate.
The point that changes the decision is another: none of these numbers authorize putting an agent into production relying only on the model. Recent research into faulty tools shows that correctly diagnosing failure is still a separate bottleneck from the ability to make the initial call.
Who is completing the entire flow better?
AutomationBench is particularly interesting because it does not score the model's textual response. Each task bootstraps a small business environment with CRM, email, calendar, spreadsheets, and other simulated systems. After the agent finishes, deterministic assertions check whether the final state is correct. The official leaderboard uses over 600 private tasks and does not rely on another LLM as a judge.
In version 1.0.5 of the leaderboard consulted on August 7, 2026:
| Model and configuration | Official score | Measured cost per task |
|---|---|---|
| Claude Opus 5 — Max | 26.2% | US$ 1.27 |
| Claude Opus 5 — Medium | 23.6% | $0.89 |
| Gemini 3.6 Flash — High | 19.8% | $0.58 |
| GPT-5.6 Sol — Max | 18.1% | US$ 1.00 |
| Claude Fable 5 — Max | 17.4% | US$ 2.03 |
The scores seem low because the criterion is severe: meeting five out of six requirements is not enough to consider the task completed. The benchmark also reports that the typical variation between runs is within approximately one percentage point.
There are two practical findings here. The first is the advantage of Opus 5 in this type of work. The second is that configuring the same model can matter almost as much as switching vendors: Opus 5 on Medium remains above the best results shown for Gemini 3.6 Flash and GPT-5.6 Sol on this leaderboard, while reducing the measured cost of the task relative to Max itself.
This does not mean that AutomationBench measures in isolation “which model generates the most correct JSON”. In this test, the agent needs to search schemas and run endpoints, find records, distinguish outdated data, comply with policies, and change the right state. Therefore, the note combines tool selection, planning, reading results and persistence until completion. This is precisely why it is useful for those who are choosing a model for flows, and not just for a function calling demonstration.
And who chooses the function and arguments correctly?
For looking at the call itself, the Berkeley Function Calling Leaderboard remains a better reference. BFCL V4 includes single-turn and multi-turn calls, missing functions and parameters, parallel cases, hallucination measurement, and agentic tasks. Its evaluation uses mechanisms like AST and state transitions to verify the call more deterministically.
There is, however, a temporal problem: the leaderboard consulted was last updated on April 12, 2026 and still contains a previous generation of several models. Claude Opus 4.5 appeared in first place overall, with 77.47%, followed by Claude Sonnet 4.5 with 73.24%. The current Opus 5, GPT-5.6 Sol and Gemini 3.6 Flash are not all represented in this cut.
Therefore, the BFCL helps more to understand which families already demonstrated good function calling fundamentals than to decree the current winner. For a purchase or migration made now, the most recent end-to-end result should weigh more.
Valid argument does not mean right argument
Current APIs offer mechanisms that reduce an important class of failures: arguments that do not obey the schema.
In OpenAI, Structured Outputs in function calls with strict: true make arguments adhere to the JSON Schema defined for that function. GPT-5.6 models also support function calling and structured outputs; the platform adds tool search and Programmatic Tool Calling for certain flows with many operations.
Anthropic also offers strict tools with strict: true, as well as tool_choice controls to make the decision automatic, require a tool or force a specific tool. Tool results can be returned to Claude with is_error: true, allowing the failure to be explicitly part of the agent's next turn.
In Gemini, the API provides auto, any and none modes, in addition to the validated mode, still marked as preview, to adhere to the schema. Gemini 3.6 Flash supports function calling, structured outputs and tools such as code execution, file search and computer use in preview.
These protections are valuable, but they have a simple limit: a customer_id field can be perfectly valid as a string and still point to the wrong customer. Structural validation reduces malformed calls; does not guarantee semantically correct selection of the tool, entity or action.
The hardest test begins when the tool fails
ToolBench-X, published in June 2026, was built precisely to fill a gap in traditional benchmarks. Instead of assuming stable tools, it introduces five classes of recoverable problems: specification change, invocation error, execution failure, change in output format or meaning, and source conflict. Tasks remain solvable through paths such as retry, fallback, verification, or cross-evidence.
The models evaluated in the article are no longer all current flagships, so it would be incorrect to use their numbers to directly compare Opus 5 with GPT-5.6 or Gemini 3.6. The most important result is structural: all tested models were below 60% accuracy when the tools received these recoverable failures. GPT-5.4 scored 45.3%, Claude Sonnet 4.6 scored 41.0% and the best result of the set, Doubao-Seed-2.0-Lite, achieved 51.3%.
The study found something even more useful for those who are designing the agent. Just giving more inference time helped little compared to telling the model what kind of anomaly it was experiencing. In a subset of 200 tasks, targeted diagnostic hints raised the baseline result by 25.5 to 35.5 percentage points, recovering most of the missed cases. The authors identify failure diagnosis, not simply a lack of computing, as the central bottleneck.
This changes the engineering question. It is not enough to ask “which model tries again?”. A robust agent needs to recognize whether to repeat the same call, change tools, correct arguments, distrust the response or stop with an explicit failure.
Which model would I choose for each scenario?
If the priority is maximizing the chance of correctly completing a complex business flow today, Claude Opus 5 is the choice most supported by available public data. It leads AutomationBench's private suite, and Anthropic itself positions it for enterprise work and complex agentic tasks.
If the system will have a large catalog of tools or predictable steps that can be filtered, aggregated and chained programmatically, GPT-5.6 Sol deserves a side-by-side test even without leading AutomationBench. OpenAI offers tool search and Programmatic Tool Calling in GPT-5.6, allowing certain sequences of calls to be processed in a programmatic runtime rather than requiring re-trial of the model between all operations. The documentation itself recommends keeping direct calls when each result can semantically change the next decision.
For a more cost- and latency-sensitive flow, Gemini 3.6 Flash is the most interesting challenger among the three in this measurement: it scored 19.8% in AutomationBench's private set at $0.58 per task, versus 18.1% and $ 1.00 from GPT-5.6 Sol Max in that same test. Its standard API price is $1.50 per million input tokens and $7.50 per million output, including thought tokens in the output charge.
Claude Opus 5 also deserves to be tested below maximum effort. On AutomationBench itself, Medium came in at 23.6% for $0.89 per task. If this relationship is reproduced in your domain, reducing the effort may generate more interesting savings than immediately switching to a smaller model.
The test that should decide production
Before making a final choice, the public benchmark must be used to assemble the shortlist. The final decision should come from a small set of assessments that replicate your own tools.
- Set out 50 to 100 representative tasks, including easy, ambiguous, and multi-step cases.
- Record the correct tool, expected arguments, prohibited actions, and required end state.
- Inject controlled failures: rejected parameter, 401 or 403, non-existent entity, timeout or 429, error 500, empty result, outdated data and conflict between two sources.
- Separately measure tool selection accuracy, argument correctness, end-to-end completion, failure recovery, unnecessary calls, incorrect side effects, latency, and cost.
- Run the same cases in the two or three finalist configurations and fix versions or snapshots when the platform allows.
This procedure avoids a frequent mistake: choosing the model that works best when everything goes right. In production, the truly useful candidate is the one that fails in a detectable way, attempts adequate recovery, and doesn't turn a partial error into an irreversible bad action.
Today, the data puts Claude Opus 5 ahead for this shortlist. But the most important informational gain from the 2026 benchmarks is less comfortable: the model is still only one part of the reliability. Strict schema, unambiguous tool descriptions, structured error responses, retry limits, idempotence, end-state validation, and real-failure testing remain part of the product.
