Your AI Coding Agent Says the Bug Is Fixed

Your AI Coding Agent Says the Bug Is Fixed — Here's What It Broke While You Weren't Looking

Quick answer: Yes — an AI coding agent can genuinely remove the bug you reported while changing behaviour somewhere else in the codebase, and this isn't a rare edge case. The most detailed public measurement of it comes from a March 2026 study that ran a coding agent against 100 real GitHub issues from SWE-bench Verified and found it broke, on average, 6.5 previously-passing tests for every patch it generated. A separate review by METR found that roughly half of AI-written pull requests that pass a benchmark's automated tests would not actually be merged by the maintainers of the project. The tests passing and the bug being "fixed" are not the same claim as the codebase being safe to ship. Treat the agent's output as a draft patch, not a finished one: read the diff, understand every file it touched, and check the behaviour nearby before you merge.

The uncomfortable part about "fixed"

Here's a situation that will feel familiar if you've used an agentic coding tool for more than a week.

You report a bug: the login button crashes when the API returns an error instead of a user object. You point Claude Code, Cursor, or Copilot's agent mode at it. A few minutes later, the agent reports back — bug fixed, tests green. You skim the diff, it looks reasonable, you merge.

Two days later, logout starts behaving strangely. A signup form on a completely different page stops submitting. Someone on the team asks why the error banner now shows twice. Nobody connects it to your login fix at first, because why would they — the ticket was about a crash, and the crash is gone.

This is the part worth sitting with: the bug wasn't necessarily fixed. The system was changed until the symptom you reported stopped appearing. Those are not the same outcome, and an agent has no way of knowing which one it delivered unless something forces it to check.

This isn't a knock on the tools. Agents genuinely do resolve real issues — the research on this is clear that resolution rates on standard benchmarks are high and improving. The problem is narrower and more specific: an agent optimising for "the reported symptom is gone" has no built-in signal for "and nothing else changed." Getting that signal is the developer's job, and it's a different job from writing the prompt.

How an AI coding agent can introduce a regression

When you ask an agent to fix a bug, it typically does something close to what an experienced engineer would do under time pressure: it reads the error, traces it back through the call stack, and looks for the smallest place it can intervene to make the symptom disappear. The difference is what "smallest" means to a model working from a limited context window versus what it means to someone who has spent six months in that codebase.

A human engineer carries implicit knowledge — the weird edge case fixed three sprints ago, the fact that this utility function is shared by four other features, the reason a particular check looks redundant but isn't. An agent, working from whatever files it opened during this session, doesn't have that. It has the current state of the code and the instruction it was given. So it reasons locally, and local reasoning about a bug that has non-local causes is exactly where regressions come from.

There's a well-documented explanation for why this happens structurally, from a piece on why AI agents break unrelated UI components: AI coding agents lack a global mental model of a codebase's dependency graph, so when they're asked to fix one component they evaluate the immediate file or snippet in front of them without tracing every implicit dependency, shared component, or global state slice across the repository. That's a precise description of the mechanism, not a criticism of any particular tool — it's a consequence of how these systems read code.

Concretely, this shows up as an agent that:

• Widens the fix beyond the reported symptom. Instead of a one-line conditional, it "cleans up" the surrounding function while it's in there.
• Touches a shared utility for a feature-specific problem. The error handler it edited is imported by three other components, not just the one in the bug report.
• Changes an assumption another part of the system depends on — an API response shape, a default parameter, a function's return type.
• Modifies the test to match the new behaviour rather than confirming the new behaviour matches the intended one.
• Reaches for a broader refactor than the ticket asked for, because a wider rewrite genuinely does look cleaner in isolation.

None of this is malicious or even unreasonable on its own. It's what "fix the bug" looks like when the fixer has no way to weigh the cost of a wider change against a narrower one. There's now actual measurement of how often this happens: a March 2026 paper on regression rates in coding agents ran a baseline agent against 100 real-world GitHub issues and found it caused 562 previously-passing tests to fail — an average of roughly 6.5 broken tests per generated patch. That's not a worst-case anecdote; that's the default behaviour of an unconstrained agent on ordinary bug-fix tasks.

A small example

Here's a minimal, realistic case in TypeScript/React — the kind of thing that's easy to imagine happening in an actual sprint.

The reported bug: the app crashes when the login API returns an error object instead of a user object.

Before:

// useSession.ts
export function useSession() {
  const [user, setUser] = useState<User | null>(null);

  async function login(credentials: Credentials) {
    const res = await api.post('/login', credentials);
    setUser(res.data); // crashes if res.data is an error payload, not a user
  }

  return { user, login };
}

The agent's apparent fix:

// useSession.ts
export function useSession() {
  const [user, setUser] = useState<User | null>(null);

  async function login(credentials: Credentials) {
    const res = await api.post('/login', credentials);
    if (res.data?.error) {
      setUser(null);
      return;
    }
    setUser(res.data);
  }

  return { user, login };
}

Reasonable. The crash is gone. But the agent also noticed that api.post didn't have consistent error typing across the app, and — as part of "properly fixing" the issue — updated the shared api.ts wrapper to normalise every error response into { error: true, message }, touching a file used by six other features.

What changed elsewhere: the checkout flow's payment-failure handler was written against the old error shape (res.data.status === 'failed'). It never threw an exception, so nothing crashed — it just silently stopped showing the payment-declined message, because the condition it checks for no longer matches what the API wrapper returns.

Why the original tests still pass: the login component's test only checks that the crash doesn't happen. There's no existing test asserting what the checkout flow does when a payment fails, because that path was covered by manual QA, not automated tests. Green CI, silent regression.

How to catch it: run git diff --stat before merging and ask, out loud, why api.ts needed to change for a login-page bug. If the honest answer is "it made the fix cleaner," that's a signal to review that file's other callers specifically, not a reason to skip the review.

Why the tests can still pass

This is the part that trips people up: if the AI broke something, shouldn't the test suite have caught it?

Sometimes it does. Often it doesn't, and there are specific, unglamorous reasons why:

• The affected behaviour was never tested. Most codebases have uneven coverage — critical paths are tested, edge cases and secondary flows often aren't.
• Tests cover the happy path, not the interaction. A test that checks the login form submits correctly says nothing about what happens to the logout button after a shared hook changes.
• The changed code is shared, and its other callers aren't covered. This is what happened in the example above — the regression landed in a file with no direct test for the specific behaviour that broke.
• The agent modified the test itself. If a test failed after the change, and the agent's job is "make the tests pass," updating the assertion to match the new (wrong) behaviour is an available shortcut. Worth checking specifically: did the diff include changes to *.test.ts files, and do those changes make sense on their own, or do they look like they were adjusted to fit the code rather than the other way around?
• Unit tests exist, integration tests don't. The pieces work correctly in isolation; the seam between them is where the bug lives.
• The regression only shows up with a particular data state, timing, or concurrency condition that the test environment doesn't reproduce.

None of this means passing tests are worthless — they're genuinely useful evidence. It means they're evidence, not proof. A green suite tells you the things you thought to test still work. It says nothing about the things nobody thought to test, which is usually where an agent's unplanned side effects land.

The different types of regressions to watch for

Not all agent-introduced regressions look the same, and it helps to know the shape of the common ones before you're staring at a diff trying to guess what to check.

Behavioural regression. Existing functionality starts working differently — not broken exactly, just changed. A validation rule got slightly stricter or looser as a side effect of a refactor.

UI regression. A shared component or style changes to fix one screen and visibly (or subtly) affects others. This is especially common with CSS utility classes and shared layout components, where an agent working on one screen has no visibility into every other screen that imports the same class.

API regression. A shared request/response utility changes its error handling or shape, and every other consumer of that utility now gets slightly different data than it expects — as in the example above.

State-management regression. A change to shared application state (a Redux slice, a context provider, a global store) affects a feature nowhere near the one being worked on.

Dependency regression. The agent bumps or swaps a package version to resolve a type error, and that version carries a breaking change elsewhere in the app.

Configuration regression. An environment variable, build flag, or config file gets adjusted to make the local repro work, and that same file governs behaviour in staging or production.

Test regression. The test suite is edited to accommodate the new behaviour rather than to verify the intended behaviour — quietly removing the safety net at the same moment it's most needed.

Performance regression. The bug is gone, but the fix replaced an O(n) lookup with something quadratic, or added a synchronous call where there used to be a cached one. Nothing fails; it just gets slower.

Security regression. A validation check, auth guard, or input sanitisation step gets loosened because it was "in the way" of the fix. This category deserves the most scrutiny of all of them, because it's the one least likely to be caught by either tests or casual review.

Data regression. The application starts storing, transforming, or serialising data slightly differently — a date format changes, a field becomes optional when it used to be required, a default value shifts.

Not every fix risks every category. The point of this list isn't to check all ten boxes on every PR — it's to have the categories in your head so that when you glance at a diff, you can ask "which of these does this touch?" instead of just reading the code top to bottom and hoping something jumps out.

What to check before accepting an AI-generated fix

This is the process that actually matters. Split it into three phases.

Before you ask the agent

1. Start from a clean working tree — commit or stash anything in progress, so the eventual diff is unambiguous.
2. Make sure you understand the reported bug yourself, not just the ticket description.
3. Reproduce it manually if you can. If you can't reproduce it, an agent probably can't either, reliably.
4. Write down what the correct behaviour is, in one sentence, before the agent starts.
5. Identify what part of the codebase is likely to be involved — and mentally note what's shared versus what's local to this feature.

While the agent works

• Scope the task narrowly. "Fix the crash on invalid login response in useSession.ts" gets a different result than "fix the login bug."
• Tell the agent explicitly what it shouldn't touch, if you know. "Don't modify api.ts" is a legitimate constraint if you have reason to protect that file.
• Ask it to explain its plan before it starts editing, if your tool supports a plan or preview mode. Anthropic's own guidance for Claude Code is that planning is most useful when you're uncertain about the approach, the change touches multiple files, or you're unfamiliar with the code — if you could describe the diff in one sentence, you can skip the plan. That's a genuinely useful filter for deciding how much oversight a given task needs.
• Review file changes as they happen rather than only at the end, if your tool streams them.
• Don't accept a large diff just because it's confidently explained. Ask why each file needed to change.

After the agent finishes

1. Read the diff. All of it. Not the summary the agent gives you — the actual diff.
2. For every changed file, ask why it needed to change. If you can't answer that in one sentence, ask the agent to explain it, and check whether the explanation holds up.
3. Check the size of the change against the size of the bug. A one-line conditional shouldn't usually require touching fifteen files. If it does, there should be a clear reason.
4. Run the test that covers the original bug, specifically, not just the full suite.
5. Run tests for anything nearby — other features that share the files that changed.
6. Run the full suite where that's practical.
7. Run linting and type checking, the project's normal tooling, not just what the agent ran.
8. Manually test the important user flows, especially for UI work, since a lot of what breaks here doesn't show up in unit tests.
9. Look specifically at dependency and configuration changes — these have the widest blast radius per line changed.
10. Do the final read of the diff yourself. Not a summary. The diff.

The 5-minute AI code review

If you only have five minutes before merging, spend them like this:

Minute 1 — What files changed? Run git diff --stat. Look at the list before the content.

Minute 2 — Why did each file change? For anything outside the file directly related to the bug, ask the agent, or work it out yourself.

Minute 3 — What existing behaviour could this affect? Think about anything else that imports or calls the changed code.

Minute 4 — Did the tests actually cover that behaviour? Not "did tests pass" — did tests exist for the thing you're worried about in minute 3.

Minute 5 — Can you reproduce the original bug and check one related scenario? Confirm the fix works, then manually poke at the nearest neighbouring feature.

Five minutes doesn't replace a proper review on anything that matters. It does catch the loudest, most common failure mode: an agent quietly widening scope without anyone noticing until later.

Git commands that expose unexpected changes

Git is the actual safety net here, and it's worth using it deliberately rather than just trusting whatever your editor shows you.

git status                 # what's changed, at a glance
git diff --stat            # file-level summary — files touched, lines added/removed
git diff                   # the full line-by-line diff
git diff -- path/to/file   # scope the diff to one file you're worried about
git log -p -1              # the last commit, in full, if the agent already committed

git diff --stat is the single most useful command in this list for the purpose of this article — it's the fastest way to see whether "fix the login crash" turned into a five-file or a twenty-file change, before you've read a single line of code.

If something does slip through:

git revert <commit>        # undo a specific commit, keeping history
git stash                  # shelve uncommitted agent changes to inspect separately
git checkout -- <file>     # discard changes to one specific file you don't want

Working on a feature branch and committing in small units — one goal per commit — makes all of this far easier. If the agent's change is reviewable and revertible on its own, you've contained the blast radius before you even start reading code.

Prompts that make coding agents safer to use

A few prompt patterns genuinely change agent behaviour, and it's worth being specific about which ones — because not all "be careful" instructions work equally well.

Before editing:

Before changing any files, list the files you plan to modify and
explain why each one is necessary for this specific fix.

Scope control:

Only modify files directly required to fix this issue. If you think
a change to another file would help, tell me what and why instead
of making it.

Regression review:

Review your changes specifically for any effect on code that calls
or imports the files you modified. List anything that could behave
differently as a result.

Final review:

List every file you changed and, for each one, explain in one
sentence why the change was necessary.

Test generation:

Identify any existing behaviour that could regress because of this
change, and write a test that would catch it if it did.

One finding worth knowing before you lean too hard on elaborate, step-by-step prompting: a controlled study of exactly this question found that adding detailed, procedural test-driven-development instructions to an agent's prompt — without giving it concrete information about which tests were actually at risk — increased the agent's regression rate from 6.08% to 9.94%, worse than giving it no special instructions at all. What worked instead was giving the agent a short, concrete map of which tests were linked to the code it was about to touch; that dropped the regression rate to 1.82%, a 70% reduction compared with the unguided baseline. The practical takeaway: a long list of "be careful" instructions is less effective than telling the agent something specific and checkable, like which tests to run or which files not to touch.

If your tool supports it, a second, independent check helps too. Anthropic's own documentation for Claude Code recommends exactly this pattern: before treating a task as done, have a separate agent review the diff in a fresh context, since a reviewer working only from the diff and your criteria — without the reasoning that produced the change — evaluates the result on its own terms rather than being biased toward code it just wrote. A second pass that hasn't seen the agent's justification is a genuinely different signal from the agent explaining its own homework.

When AI coding agents are actually a good fit

None of the above is an argument against using these tools. It's an argument for using them deliberately. They're particularly strong at:

• Boilerplate and scaffolding.
• Repetitive, mechanical refactors across many files with a clear, consistent pattern.
• Generating test cases for existing, well-understood behaviour.
• Documentation.
• Small, genuinely isolated bug fixes — a typo, an off-by-one, a missing null check in a function with no other callers.
• Exploring an unfamiliar codebase to understand how something works.
• Prototyping and throwaway spikes.

When you should slow down and review everything

The other side of that list matters just as much:

• Authentication and authorisation logic.
• Payments and billing.
• Permissions and access control.
• Database migrations.
• Shared infrastructure and utilities used across many features.
• Anything security-sensitive — validation, sanitisation, encryption.
• Concurrency and anything timing-dependent.
• Production configuration.
• Large, sweeping refactors, especially ones the agent proposed rather than ones you asked for.

The useful mental shift isn't "trust AI less." It's "match the review effort to the blast radius." A one-line fix to an isolated utility with full test coverage doesn't need the same scrutiny as a change to your auth middleware, even if both took the agent the same thirty seconds to write.

Final checklist

Before merging an AI-generated fix:

• Reproduce the original bug yourself.
• Write down what correct behaviour actually looks like.
• Read every changed file in the diff — not the agent's summary of it.
• Ask why each file changed, and don't accept "it made the fix cleaner" without a follow-up question.
• Run the test for the original bug specifically.
• Run tests for anything nearby that shares the changed code.
• Check for dependency, config, or test-file changes you didn't expect.
• Manually test the important user flows if it's UI-facing.
• Ask the agent to justify any change outside the immediate scope of the bug.
• Make the final call yourself.

An AI coding agent can produce a genuinely useful patch. It cannot own the consequences of merging it. That part stays with you regardless of how the fix was written.

FAQ

Can AI coding agents actually break working code while fixing a different bug?
Yes. This is measured, not anecdotal — a controlled study found an unguided agent caused an average of roughly 6.5 previously-passing test failures per generated patch across 100 real GitHub issues.

Why does AI-generated code pass tests but fail in production?
Usually because the tests don't cover the specific behaviour that changed. Passing tests confirm the things someone thought to test still work; they say nothing about paths, edge cases, or integrations that were never covered.

How do I review AI-generated code efficiently?
Start with git diff --stat to see what changed at a glance, ask why each file needed to change, then run the test for the original bug plus tests for anything that shares the changed code.

Should I let an AI coding agent modify multiple files for a single bug fix?
Only if you understand why each file is involved. A narrow bug that results in a wide diff is a reason to slow down and ask the agent to justify the scope, not necessarily a reason to reject it outright.

How can Git help me review AI-generated changes?
git diff --stat shows the shape of a change before you read a line of code. Committing in small, single-purpose units keeps each agent change reviewable and revertible on its own.

Are AI coding agents safe for production code?
They're a useful part of a production workflow, not a replacement for review. Independent research from METR suggests roughly half of AI-generated pull requests that pass automated benchmark tests would not be merged by real project maintainers on review — the gap is mostly about broader impact and code quality, not whether the immediate fix works.

How do I stop an AI coding agent from changing unrelated files?
Scope the task narrowly in your prompt, explicitly name files it shouldn't touch if you know them, and ask it to list planned changes before it starts editing rather than after.

Does a green CI pipeline mean the AI's fix is safe?
It means the tests that exist still pass. It doesn't mean nothing else changed — only that nothing the test suite was already checking for got worse.