Why Agentic Coding will Always Introduce Errors

When using an LLM to modify pre-existing code, it is not possible to get the error rate to zero.

This article shows why agentic coding will always introduce errors because LLMs generate statistically plausible patterns rather than system‑aware modifications, making semantic mistakes unavoidable even with strong agentic reasoning. Only human‑authored tests and architectural invariants can prevent these errors from entering production.

Why code modification is hard

Code modification is hard because the LLM is not solely operating inside its own learned patterns as it is when it produces greenfield code based just on a prompt.

To get code modification correct, the LLM must operate inside the patterns of the codebase.

Code modification requires a transformer to work with the codebase which is a structure it did not learn during training, and transformers are not designed to infer, maintain, or reason about such structure. They are built to continue patterns, not model systems.

Transformer design

Transformers are designed to model sequences using self‑attention, a mechanism that statistically captures relationships between positions regardless of what the sequence represents.

In natural language, this works extremely well because English contains stable statistical patterns. These patterns become apparent when processing vast amounts of text: "dogs run" is common, "dog runs" is common, but "dogs runs" is not.

A transformer does not know anything about plurality or grammatical agreement; its text processing simply shows that certain token pairings occur frequently and others almost never do. It is the statistically common case that makes modelling the English language with self-attention reliable.

Programming languages exhibit similarly common usage patterns.

For example:

if(condition) {
  do_something()
}
else {
  do_something_else()
}

appears extremely often in training data, whereas:

if condition) {
  do_something(
}
else
  do_something_else)
}

almost never appears.

Processing vast amounts of correct code gives the transformer the ability to generate correct code because correct usage is overwhelmingly more common than incorrect usage. When the model predicts the next token, it is statistically biased toward the patterns that appear most frequently. And in programming languages, those frequent patterns correspond to syntactically valid code.

The LLM does not understand the syntax of a programming language nor does it model programming language grammar or sementics.

The LLM does not understand the semantics of an if-else, i.e., that only one branch will be executed, and the LLM cannot determine what condition will be only by looking at the text.

An LLM simply learns that valid token sequences (like if(condition)) appear vastly more often than invalid ones (like if condition)).

And because correct usage dominates the training distribution, next‑token prediction is statistically biased toward the more common patterns — and those common patterns happen to correspond to syntactically valid programming code.

Correctness is a by-product of statistics, not understanding.

But code modification requires more, beyond bias towards common patterns.

An example

Imagine you have read through a codebase and you notice that all writes to user accounts are not made directly but via a wrapper. You can see that the wrapper performs audit logging.

This approach is not written down anywhere. It is enforced by the architecture and historical use of the codebase, as well as by team practice.

To safely modify this code, any change must continue to respect that all writes go via the wrapper for audit logging compliance.

Based on its training data, statistical patterns and next token prediction, the LLM generates:

account.balance += amount

because this is a high-probability token continuation.

But this code breaks the system as the update of 'account' does not go via the wrapper.

The generated text is correct but what it means is incorrect with respect to the system that text must operate within.

Fundamental difference

Pattern-continuation solves this:

From my training data, I have seen many examples of updating a balance; here is a plausible one

Safe code update requires:

In this system, balance updates must go through a wrapper because of a compliance rule that is not explictly represented anywhere in text.

When it comes to modifying pre-existing code, the model can only suggest a likely next token. The model is unaware it should only generate tokens that respect the use of the wrapper.

Safe code modification requires respecting constraints that are not present in LLM training patterns.

There may be constraint respected examples in its training data but the transformer has not been designed to statistically model them. To the transformner, everything is just text, there are no domain semantics.

The transformer models patterns in text with no regard to what that text means.

Error can never be zero

Because transformers generate text by continuing statistically likely patterns, and because system‑level constraints are not represented in those statistics, no amount of prompting or additional context can eliminate constraint‑violating continuations. Error cannot go to zero.

Prompting and additional context can reduce the error but will not eliminate it.

Agentic systems

Agentic systems add software engineering-specifc tooling to an LLM system.

Such systems operate a workflow of: LLM proposes → verifier checks → tool executes.

Before any LLM proposed action is executed, a number of verifiers are used. These are rule based and perform static programming language checks, schema validation, safety filtering and the enforcement of invariants.

The verifiers can only check explicit programming language semantics: types, schemas, pre- and post-conditions. Verifiers cannot check meaning that is not formally expressed. Any semantics that are implicit, domain-specific, or are derived from "that is how we do things" cannot be checked.

In any hybrid agentic system, the LLM can output a semantically wrong action, and no amount of tooling, verification, or scaffolding can eliminate that class of error.

Hybrid systems help reduce the mechanical error rate but semantic errors can never be eliminated.

But this is OK. To err is LLM.

Agentic failure modes

The work of Albayaydh–Zhao–Flechais (2026) in their "leaderboard paper" is a synthesis of 27 other works from between 2023 to 2026, covering 19 benchmarks. Their synthesis has produced these categories of agentic failure:

1. Tool‑invocation and parameter‑level errors
2. Planning and constraint‑satisfaction failures
3. Long‑horizon degradation from context accumulation
4. Multi‑agent coordination failures
5. Safety failures (unsafe agent behaviour)
6. Security failures (external adversarial conditions)
7. Measurement validity problems

Points 6 and 7 are not caused by the LLM. They are to do with conditions external to the agent.

Points 1 to 5 are caused by the probabilistic nature of the LLM.

Probabilistic token generation is an approximation. Given this, LLM-centred agentic software development will never be fault free.

Agentic software development lifecycle

A software engineer using an agentic system has to:

• shape prompts
• review agent output
• correct LLM mistakes (due to hallucinations or the semantic gap)
• run tests

and based on this the engineer must reprompt the LLM or retry some tooling.

Eventually, an LLM- and engineer-developed artefact will be generated that the engineer will be happy to integrate into a production system, while also ensuring architectural consistency and guarding again silent errors: both of which may require engineer-performed direct changes to the artefact.

In practice, agentic workflows reduce construction time but increase review and validation time.

Agentic systems introduce new overhead

Even if the agent produces good artefacts, teams must maintain:

• agent behaviours
• prompt libraries
• safety rails
• integration harnesses
• test suites designed for agents

This is additional overhead that was not required before agentic systems.

An agentic software development lifecycle requires an engineer at the centre scheduling tasks, orchestrating and conducting workflow, reviewing outcomes and validating them for subsequent deployment.

Human involvement is mandatory because LLM-centred tools make mistakes.

Is Agentic development about speed?

No. Using an agentic system to push for speed, creates pressure to ship faster by cutting corners: reducing review time, accepting lower-quality code, and reducing testing. This will have the effect of pushing features out more quickly but also prematurely. More is not better.

Software development is not solely about code generation. It is mostly about understanding the requirement to ensure that the right code is written, regardless of whether the code is LLM-generated or human authored.

Pushing for faster shipping is an approach that will affect the whole business:

• product (more time spent triaging issues instead of contributing to building)
• design (rework due to unstable requirements)
• support (increased customer issues)
• sales (prematurely released features break or regress)
• marketing (launches are undermined by defects)
• finance (the higher cost of incidents and rework due to premature code release)
• brand (trust erosion from unstable releases)
• board (confidence undermined)

Agentic development only makes economic sense if the final artefact is at least as good as the human‑only artefact.

If agentic development leads to a reduction in the quality of the final artefact, businesses will pay the cost later in increased rework, more production issues, and elevated maintenance costs.

Rework will correct the immediate defect. Maintenance cost reflects the long-term cost of poor-quality artefacts that adversely affect system structure and increase test fragility, both of which represent regression risk lt leaves the business with less room for manoeuver when needing to change code.

Quality reduction has a knock-on effect resulting in a long-tail of consequences.

Quality is king

When considering agentic development, the only thing that matters is:

Does the human plus AI workflow produce a higher‑quality artefact than a human alone?

At a minimum, does the agentic approach produce equal quality at lower cost?

There is no general answer to either of these. They can only be answered accurately with respect to the context of a company: their specific agentic software development lifecycle, and the company's approach to testing, confirmation, verification, and release.

Errors will always be with us

Before agentic development, a quality outcome was already the goal of every software delivery pipeline. Agentic development does not change this goal, it only changes how artefacts are produced.

Both humans and LLM‑centred agentic tooling produce errors.

LLMs introduce new classes of mistake: hallucinations, semantic gaps, mis‑inferred invariants, and silent failures. Humans introduce their own: misunderstandings, typos, incorrect assumptions, and incomplete reasoning.

Therefore, an agentic software development lifecycle must include testing. Human‑generated artefacts require tests; so do artefacts generated agentically.

And critically: those tests must be solely human‑authored.

If tests are generated by the same agentic system that produced the code, the tests may simply reproduce the same misunderstandings or hallucinations.

Errors can hide inside the volume of generated code and generated tests, reinforcing each other rather than revealing faults.

Only human‑authored tests provide an independent, trustworthy oracle of correctness.

Testing all calls are via a wrapper

Agentic development requires a new kind of test: tests that assert architectural invariants, not just functional behaviour.

If such a test is not written, eventually, the agent will by-pass the wrapper.

This test must assert a fact about the structure of the system, not test its logic. It is a test that confirms something about how the system is built.

This kind of code would need to build an abstract syntax tree of the codebase, and then test whether every call in that tree (the whole codebase), all go via the wrapper.

Conclusion

Agentic output will never be perfect. But that is OK. Results only need to be correct-enough that a human can efficiently get the code to 100%, where overall cost is cheaper than the human doing 100% of the work.

Outline Python to build an AST

The AI-generated code to build an abstract syntax tree from a .ZIP URL is short, only 51 lines.

# You are free to use this code. No warranty is expressly given or implied.
# You are using this code "as-is" and you should check its applicability for
# use within your systems and Python environment.

import ast
import pathlib
import tempfile
import urllib.request
import zipfile
import tarfile


def download_and_extract(url: str) -> pathlib.Path:
    """Download a codebase archive from a URL and extract it."""
    tmpdir = pathlib.Path(tempfile.mkdtemp())
    archive_path = tmpdir / "codebase"

    # Download the archive
    urllib.request.urlretrieve(url, archive_path)

    # Extract depending on format
    if zipfile.is_zipfile(archive_path):
        with zipfile.ZipFile(archive_path, "r") as z:
            z.extractall(tmpdir)
    elif tarfile.is_tarfile(archive_path):
        with tarfile.open(archive_path, "r:*") as t:
            t.extractall(tmpdir)
    else:
        raise ValueError("Unsupported archive format")

    return tmpdir


def build_ast_for_codebase(root: pathlib.Path):
    """Parse every .py file under root into an AST."""
    ast_map = {}

    for file in root.rglob("*.py"):
        try:
            source = file.read_text()
            tree = ast.parse(source, filename=str(file))
            ast_map[file] = tree
        except Exception as e:
            print(f"Failed to parse {file}: {e}")

    return ast_map


# Example usage:
if __name__ == "__main__":
    url = "https://example.com/my_project.zip"  # your codebase URL
    root = download_and_extract(url)
    ast_map = build_ast_for_codebase(root)

    print(f"Parsed {len(ast_map)} Python files into ASTs.")

This code gives you:

• a directory containing the full codebase
• a dictionary mapping the .ZIP file to an AST
• a typed AST you can traverse to assert architectural invariants

The code below provides a test that traverses the AST:

def find_calls(tree, module, func):
    calls = []
    for node in ast.walk(tree):
        # The node in the if is a function call and the
        # code is modelled as a Python attribute so
        # this code matches method invocations of the
        # form: object.method(...)
        if isinstance(node, ast.Call) and
           isinstance(node.func, ast.Attribute):
            if node.func.attr == func and
               getattr(node.func.value, "id", None) == module:
                calls.append(node)
    return calls

def test_no_direct_user_updates(ast_map):
    violations = []

    for file, tree in ast_map.items():
        # if account.update found, wrapper not used
        bad = find_calls(tree, "account", "update")
        if bad:
            violations.append((file, bad))

    assert violations == [], f"Direct calls found: {violations}"

Read next: Vibe Coding Is Not Engineering LLMs generate code, but they cannot see the engineering decisions that keep systems safe.

Related Articles

• Why Junior Engineers Matter More as AI Expands
• The Myth of Complete Specifications
• The Limits of Stateless LLMs
• What Software Engineers Need to Know About LLMs
• Evaluating AI Systems: Metrics that Matter