Claude Code vs Codex: which changes a repository with fewer regressions?

Short answer: as of September 1, 2026, there is not enough broad public evidence to say that Claude Code or Codex universally produces fewer regressions in real repositories. The strongest current independent comparison of the two coding agents places Claude Code with Opus 5 (xhigh) ahead on the aggregate coding-agent index, at 68 versus 65 for Codex with GPT-5.6 Sol (max). At the same time, Codex leads the DeepSWE component, costs less per task, and finishes tasks in less than half the time in the same evaluation. That is an efficiency advantage, not proof of a lower regression rate.

For real engineering work, the more useful question is not “which is better?” but “which preserves my system’s behavior with the lowest total review cost?” The answer depends on the kind of change, the quality of the repository’s tests, and the agent configuration.

What current benchmarks actually say

The most useful comparison today is the Artificial Analysis Coding Agent Index v1.4, because it evaluates the model + agent harness combination instead of comparing isolated models. In the configurations highlighted on the page, Claude Code uses Opus 5 at xhigh effort and Codex uses GPT-5.6 Sol at max.

The important point is that none of these metrics directly equals “regressions per change.” The index combines different tasks, and a higher score may come from better terminal use, failure recovery, or repository understanding without necessarily meaning the agent breaks fewer existing behaviors.

The most regression-specific signal is too small to decide on its own

An independent benchmark called Gauntlet Bugfix tested six issues with hidden resolution tests and PASS_TO_PASS tests that check whether previously correct behavior still works. In that run, Claude Code with Opus 4.8 and Codex CLI with GPT-5.5 strictly resolved 6 of 6 cases and both scored 100% on the regression component. Claude Code scored 99 on the composite and Codex 98, with the difference mainly coming from patch minimality.

This is useful evidence because it measures something close to the central question, but six issues are not enough to turn the result into a general rule. The models used in that run also no longer represent the frontier configurations available in September 2026.

Another small benchmark points the other way

The open-source Coding Agent Benchmark runs 11 synthetic tasks with hidden tests, including bug fixes, refactors, and multi-file cases. In the published run using Claude Code with Opus 4.6 and Codex with GPT-5.4, Codex fully passed 11 of 11 tasks while Claude Code passed 10 of 11, with average correctness of 1.00 versus 0.99 and nearly identical runtime.

That does not necessarily contradict Gauntlet: the tasks, models, and scoring rules are different. In practice, the two small studies show exactly why “which agent causes fewer regressions?” cannot be answered by one percentage pulled from the web.

Bug fixes: Codex has a strong case, but not a definitive win

For bug fixes with clear acceptance criteria and reproducible tests, Codex has a favorable current signal: in the Artificial Analysis comparison, GPT-5.6 Sol in Codex scores 69% on DeepSWE versus 60% for Opus 5 in Claude Code. Codex also runs the suite at lower average cost and time.

That suggests a good fit for queues of independent issues, high-volume maintenance, and work that can be objectively validated. But “the failing test now passes” is not enough. Your internal benchmark should also include tests that were green before the change, because those are the tests that detect collateral regressions.

Refactoring: diff discipline matters more than raw success rate

In refactors, the main risk is changing behavior without noticing. An agent can finish the requested task and keep the visible test suite green while still expanding the diff, renaming unrelated code, or changing contracts that are not covered by tests.

A refactoring comparison should therefore score the number of files touched, out-of-scope line changes, public API changes, human rework, and coverage of changed code. Claude Code’s aggregate lead in the current index and its stronger Terminal-Bench and SWE-Atlas-QnA scores are useful signals for work that requires exploration and context, but they do not by themselves prove that it preserves behavior better in a specific refactor.

Tests: counting how many tests were written is a trap

An agent can increase the test count without increasing the suite’s ability to catch defects. To compare Claude Code and Codex on test generation, prefer mutation score, coverage of changed lines, added edge cases, and the number of deliberately injected bugs the new suite detects.

It is also worth preventing the agent from seeing the hidden evaluation suite. Otherwise, you are measuring adaptation to the evaluator rather than robustness.

Cost: compare cost per accepted patch, not subscription price

Subscription pricing is hard to compare directly. Claude Code can be used through Claude subscriptions and API billing, and Anthropic’s own Claude Code cost guide stresses that usage varies widely with model choice, repository size, and work pattern, recommending a pilot to establish a real baseline.

With Codex, ChatGPT-account usage draws from a shared agentic allowance and varies with model, duration, context, tools, and execution surface; the Codex usage documentation recommends checking the usage dashboard and available credits. As of September 1, 2026, current GPT-5.6 options in Codex include Sol, Terra, and Luna on eligible plans, after GPT-5.4 and GPT-5.4 mini were removed from ChatGPT-authenticated Codex on August 31.

For a serious benchmark, the better economic metric is total cost per accepted patch: add the first run, reruns, failed attempts, and review time. The agent that is cheaper per run can still cost more if it needs two manual correction passes.

Workflow differences can also affect regressions

Claude Code has built-in tools for reading and editing files, searching, executing commands and tests, plus CLAUDE.md, hooks, subagents, and permission rules. The documentation on how Claude Code works describes a loop of running tests, reading failures, locating code, editing, and testing again. Claude Code permissions can be version-controlled and combined with operating-system sandboxing.

Codex also works directly on repositories, runs commands and tests, and can be guided with AGENTS.md files. OpenAI documents sandboxing and approval policies in its guide to running Codex safely; in the CLI, sandbox modes include read-only and workspace-restricted write access.

These controls matter because some real-world “regressions” come not from weak coding ability but from poorly bounded scope: an agent writing outside the expected directory, installing unnecessary dependencies, or modifying files that should not be part of the task.

How to run a benchmark that actually answers the question

If the decision matters, test both agents on your own repository. A minimum protocol can use 12 to 20 real tasks split across bug fixes, refactors, and test maintenance, with three repetitions per task.

1. Freeze a baseline commit and restore exactly the same state before every run.
2. Use the same prompt, timeout, permissions, network access, and validation commands.
3. Pin models and effort levels; do not compare an old Claude model with a new Codex model and call it a tool comparison.
4. Keep a hidden test suite that the agent cannot edit or inspect.
5. Separate FAIL_TO_PASS tests, which prove the bug was fixed, from PASS_TO_PASS tests, which detect regressions.
6. Record the final diff, files touched, commands, duration, cost, retries, human interventions, and test results.
7. Review the patch without knowing which agent produced it when possible.

A simple regression-rate definition is: the number of completed runs that make at least one previously passing hidden test fail divided by total completed runs. For refactors, add a penalty for out-of-scope changes. For test-generation tasks, add mutation score or another measure of defect-detection power.

Which should you choose today?

Conclusion: if you must choose without running your own test, Claude Code currently has a small aggregate-quality advantage, while Codex offers a clear speed and cost advantage and shows strength in DeepSWE software-engineering tasks. For the specific question “which one modifies a repository with fewer regressions?”, however, direct evidence is insufficient, and even the small regression-focused tests show a tie. The most defensible approach is to make regression a CI metric and compare both agents on the same set of real tasks.