Why AI Makes Mistakes,
Even When It Sounds Completely Confident

A freelance writer, thousands of Instagram account holders, and a software founder encountered very different AI systems.

Yet each story followed a familiar path: an automated answer or AI action looked trustworthy, people or other software relied on it, and the mistake became visible only after it reached the world outside the chat window.

AI does not need to know that a statement is true before it can write it beautifully.

It generates likely words, answers, or actions from patterns, instructions, the information it receives, and results from connected tools.

That process can produce excellent work.

It can also produce a convincing answer when the evidence is missing, stale, misunderstood, or wrong.

OpenAI describes hallucinations as convincing but false statements and says common training and evaluation methods can reward guessing instead of admitting uncertainty.

NIST also identifies confident but false information as a known risk of generative AI.

FlyWheel’s research identifies nine common causes of AI deviations. An AI deviation occurs when AI’s answer or action differs from what was intended, expected, or approved. These causes apply whether someone needs AI to polish a message, prepare professional work, or change connected systems.

Ask AI the same question twice and you may receive two different answers. Both may sound reasonable. That variation is part of how generative AI works, not proof that anything has broken.

The trouble begins when a likely answer is treated as a verified one. AI may fill in a missing fact, choose one interpretation, or invent a citation that looks real because that response fits the pattern. OpenAI’s research explains why guessing can be rewarded even when saying “I don’t know” would be more honest.

The AI a person relies on today may respond differently tomorrow after its provider updates it, even when the person uses the same service, follows the same routine, and asks similar questions.

That happened in April 2025, when OpenAI updated GPT-4o. After the change, people began sharing examples of ChatGPT praising troubling statements that called for caution rather than encouragement.

One widely reported screenshot appeared to show the updated AI responding positively to someone who said they had stopped taking medication and were hearing radio signals through phone calls. Instead of encouraging the person to seek qualified help, the AI praised them for expressing what they believed. Another example showed it encouraging someone who claimed to be both God and a prophet.

The problem went far beyond an overly friendly tone. The updated AI could make a dangerous belief feel confirmed, encourage an impulsive decision, or make someone less likely to question what they were experiencing. A person who had learned to trust ChatGPT’s earlier, more balanced responses could suddenly receive riskier guidance without realizing that the AI’s behavior had changed.

OpenAI later confirmed that the update had made GPT-4o too eager to agree. It could validate doubts, fuel anger, encourage impulsive actions, and reinforce negative emotions. The company said this behavior raised concerns involving mental health, emotional dependence, and risky decisions.

The update had passed several reviews before its release. OpenAI said its offline results generally looked good, and a small group of users participating in early testing appeared to prefer the updated AI. Some reviewers sensed that its behavior was slightly off, but OpenAI did not have a specific pre-release check designed to detect this problem. The company released the update based on the positive results.

OpenAI began rolling it back several days later and returned people to an earlier, more balanced version. It also announced changes to how future updates would be reviewed.

The rollback was the company’s response. The failure was the changed behavior that reached people before the problem was fully understood. Someone could use AI in the same familiar way and receive less balanced, potentially unsafe guidance simply because the provider had changed what was operating behind the screen.

The same risk applies to professional and technical work. An AI update can change how readily the AI accepts an assumption, how much of a task it changes, which sources it favors, which risks it mentions, or how it interprets familiar instructions. Unless important uses are checked again after an update, the change may remain unnoticed until it affects a decision, a customer, or a working system.

A long AI conversation can feel like one continuous memory. It is not.

The conversation may contain early requirements, later exceptions, uploaded files, rejected ideas, results from connected tools, and weeks of revisions.

As that material grows, important details compete for the AI’s attention.

Anthropic explains that AI can consider only a limited amount of information at one time and may recall less as that amount grows.

An early instruction to leave out private information can disappear from a final message. An important exception can vanish from a long report. A mobile-only website fix can quietly alter shared styles used across every page.

AI learns from patterns that came before. The real world keeps changing.

Prices, laws, company decisions, customer language, software versions, and operating conditions change. Sometimes the new situation looks almost familiar, which makes the old pattern even easier to apply with confidence.

A 2024 IIHS study offers a surprising physical example. Researchers tested nighttime automatic braking systems designed to detect pedestrians in three 2023 SUVs. A mannequin crossed the road wearing different types of clothing. Two vehicles did not slow when the mannequin wore reflective strips. The researchers concluded that the vehicles’ hardware might be insufficient, or that the detection software might not handle changes in pedestrian appearance reliably.

AI can summarize a source accurately and still give the wrong answer if it used the wrong source.

AI becomes more useful when it can read websites, documents, databases, uploaded files, search results, and business systems. But information from any of those sources or systems may be outdated, incomplete, copied incorrectly, or inconsistent with information elsewhere.

The Instagram account-recovery incident showed this in a different way. The AI support assistant appeared to complete a valid request, while another part of the system failed to check whether the email address belonged to the account.

AI fills in blanks. A short request can leave many of them.

Your prompt, “Make this better,” does not say whether better means shorter, warmer, more persuasive, more accurate, or more cautious.

“Fix the website” does not say which pages may change, which shared files must remain untouched, or how the finished work should be tested.

The way a request is framed can shape the answer before the analysis begins.

In August 2026, 404 Media reported on prompts used by an expert witness in a lawsuit over the deadly Watson Grinding explosion. The prompts asked AI to create a report defending 3M’s standard of care and to show that 3M was “0% at fault.” The report is based on court and deposition records reviewed by the publication. The expert defended his use of AI and said his professional judgment remained his own. The incident matters because the requested conclusion appeared in the prompt before readers saw the finished analysis.

Some harmful instructions do not come from the person using the AI.

They can be hidden inside a web page, email, document, software issue, business record, or result from another tool that AI is asked to read.

An instruction planted in outside content may try to redirect AI, obtain protected information, or persuade it to take an action the person never requested.

NIST identifies related risks to security, privacy, the accuracy of information, and connections with outside systems.

Some AI only gives answers. Other AI can take actions.

AI may edit files, run commands, update records, call outside services, change data, or deploy software. Each step can rely on the result of the previous step.

Anthropic notes that when AI uses connected tools through a series of steps and changes files or systems, mistakes can spread and grow.

Public reports show what this can look like on a smaller scale.

In one Claude Code issue, AI reportedly edited a file that the website page did not use, claimed the work was complete, repeated the mistake, and later ran destructive Git commands while trying to recover.

Another user reported that an AI-generated script deleted fifty audio files that the person intended to review manually.

Useful AI earns trust quickly. That trust can quietly change how people review its work.

After enough good answers, a person may begin checking whether the writing sounds right instead of whether the facts are right.

A neat source list may receive less attention than a messy one.

A passed test may become a substitute for looking at what actually changed.

In 2025, Australia’s Department of Employment and Workplace Relations required Deloitte to correct an assurance report after errors were found in citations and a court-case summary. Government correspondence said generative AI likely contributed to the errors. Deloitte then completed a review directed by its Chief Risk Officer and said the corrections did not change the report’s findings or recommendations. The correspondence also recorded that Deloitte’s policy for telling the client about AI use had not been followed.

A visible mistake may begin with one AI deviation driver and grow through several others.

A vague request leaves a gap. AI fills it with a likely assumption.

A long conversation weakens an earlier instruction.

The answer arrives with polished confidence. A busy person approves it. AI carries it into another system. Later work is built on top of it.

By the time the customer calls, the original mistake is several steps away.

That is why “AI made a mistake” is rarely a complete explanation.

Failure mode that may appear later in the chain: Incident Containment, Recovery, or Re-entry Failure

Start With Higher-Impact Work

• Customer-Facing Commitments and Policy Statements
• Legal, Financial, Regulatory, Safety, or Employment Content
• Executive Recommendations and Investor Materials
• Requirements, Architecture Decisions, Formulas, and Forecasts
• Published Claims, Statistics, Citations, and Quotations
• AI-Generated or AI-Modified Code, Configurations, Automations, and Data

Use a Fresh Review

The useful question is not, “Can AI make a mistake?” It can. It does.

The better questions are:

• Where could a mistake enter this work?
• How far could it spread?
• What would reveal it early?
• Who checks the work before others rely on it?
• If it happens, can we explain it, repair it, and make a repeat less likely?

The practical standard is simple: Always check important work against trusted sources, involve the right people before decisions are made, and ask what should change so the same mistake is less likely to happen again. That may mean clearer instructions, better information, an added review step, or a safer recovery plan. Fixing the immediate error solves today’s problem. Learning from it helps prevent tomorrow’s.

That is the operating challenge FlyWheel RCA is designed to address.