AI agent drift is the slow change in what an agent does after launch. Researchers found one model's prime number accuracy fell from 84% in March 2023 to 51% in June 2023 on the same questions. The four causes, five checks that catch it, and how we handle it in LTFI.
AI agent drift is the slow change in what an AI agent does after launch, even when nobody touched its job description. Stanford and Berkeley researchers found that one major model identified prime numbers with 84% accuracy in March 2023 and 51% in June 2023, on the same questions. Your agent sits on top of a model like that, and it can move the same way.
We have run LTFI, our agentic business operating system, in live production for a year and a half. The demo never shows the part I want to write about here: the same agent, doing the same job, gives a slightly different answer next month. This piece covers what causes that change, why it matters more in a working business than in a lab, and the checks that catch it before a client does.
AI agent drift is the gap that opens between how an agent behaved when you approved it and how it behaves now. The cause can be a new model version, a changed rule, different data or a tool that changed underneath it, and usually nobody announces any of them.
A January 2026 paper by Abhishek Rath gives the research definition: agent drift is "the progressive degradation of agent behavior, decision quality, and inter-agent coherence" over long interaction sequences. Rath splits it into three kinds: semantic drift (the work wanders from the original intent), coordination drift (agents that hand work to each other stop agreeing) and behavioral drift (new strategies nobody asked for). His results come from simulation, not live deployments, which is worth saying plainly.
The older, plainer finding comes from machine learning in general. A 2022 study in Scientific Reports by Vela and colleagues at Harvard Medical School and MIT tested four model types on 32 datasets and "observed temporal model degradation in 91% of cases". The authors add that this aging is "not solely driven by the drifts in the data", which means watching your inputs alone will not tell you the whole story.
Software has always drifted toward mess, a pattern I wrote about in why every system drifts toward mess. AI agent drift is that same pull with one difference: the system changing under you is partly someone else's model, and it changes on their schedule.
Four things cause most AI agent drift: the model changes, the rules change, the data and tools change, and long runs lose track of the rules. Each one leaves a different trace, so it helps to know which you are looking for.
Most businesses rent their model. When the provider ships a new version, your agent inherits it. In the same Chen, Zaharia and Zou study, the share of one model's code answers that ran without edits fell from over 50% in March 2023 to 10% in June 2023. The paper, later published in the Harvard Data Science Review in 2024, ends by stressing "the need for continuous monitoring of LLMs."
Even with no version change, the same prompt does not always give the same answer. Claire Nicholson's January 2026 study found that "identical prompts do not always produce identical outputs" and that the variation persists even at the lowest temperature setting. A single good test run proves less than it seems to.
Someone edits the instructions to fix one complaint. The fix works, and three other behaviors shift with it. This is the most human cause of drift, and the easiest to prevent: write rules down in one place, date every change and know who made it.
A CRM field gets renamed, a supplier changes a PDF layout, or a connected tool updates its own instructions. The agent keeps running and quietly works from different inputs. In automation maintenance is the job, not the launch I made the case that an automation left untended does not stay still, and agents are no exception.
The newest research is about rules falling out of an agent's working memory. When a long task fills the context window, many agents summarize the history to keep going. A June 2026 preprint by Shiyang Chen, titled Governance Decay, measured what happens to a written policy when that summary runs: across 1,323 episodes, violations rose "from 0% with the policy in full context to 30% after compaction, reaching 59% for some models". When the rule survived the summary, violations stayed at zero. It is a single preprint, but the mechanism matches what anyone running long agent tasks will recognize.
AI agent drift matters because a business acts on the output. A lab measures a drop in accuracy; a business sends the email, quotes the job or files the report, and the drop lands on a customer.
The scale of what goes wrong is rising. Stanford's 2026 AI Index reports that the AI Incident Database recorded 362 documented AI incidents in 2025, up from 233 in 2024. Most of those were not dramatic failures on day one. Many were systems doing something different from what their owners expected.
There is a Massachusetts angle too. When the Attorney General issued her AI advisory in April 2024, it listed practices that can be unfair or deceptive under the state's consumer protection law, including "misrepresenting the reliability, manner of performance, safety or condition of an AI system". If a Worcester firm tells clients its AI drafts are checked and accurate, and drift has quietly made that untrue, the promise is the problem. Massachusetts AI regulations for a business covers the legal side in full; this is general information, not legal advice.
The business mix here makes the point concrete. Around Worcester and Shrewsbury the companies putting agents to work are manufacturers, clinics, property managers, nonprofits and professional firms. None of them has a team watching model release notes. Drift has to be caught by the way the system is built, not by someone remembering to look.
You catch AI agent drift by comparing today's behavior with a known good baseline, on a schedule and after every change. Five checks do most of the work, in this order.
The NIST AI Risk Management Framework says the same thing in its own words. Its MEASURE 2.4 outcome asks that "the functionality and behavior of the AI system and its components" "are monitored when in production", and the Playbook suggests documenting how production metrics "differ from the same metrics collected during pre-deployment testing." That comparison is the drift check.
The pause switch is where many organizations are weakest. In ISACA's March 2026 poll of more than 3,400 digital trust professionals, 56 percent did not know how quickly they could halt an AI system after a security incident, and only 36 percent said humans approve most AI-generated actions before they run. Both numbers are about drift as much as security. An agent you cannot stop is an agent whose drift you can only watch.
If you want the broader version of this review, how to audit what AI is actually doing walks through an inventory first. The check set above is the piece that turns a one-time audit into a habit.
A weekly AI agent drift review is a short look at three things: what changed, what the checks say, and what people pushed back on. It fits on one page.
| Signal | Where to look | What it usually means |
|---|---|---|
| Check set scores moved | Monthly or after-change rerun | A model or rule change shifted behavior |
| More edits before approval | Approval log: how often a person rewrote the draft | Tone or judgment drift that scores miss |
| Checker rejections rising | Second-model review results | Outputs breaking a written rule more often |
| Longer or shorter outputs | Word counts in the run log | A silent model update or a trimmed instruction |
| New tools in the run log | Tool calls per job | Behavioral drift: the agent found a new path |
The second row is the one I would not skip. When people start rewriting drafts they used to approve as written, that is the earliest sign of drift you will get, and it costs nothing to count. The same instinct runs through AI agents that run at 3am versus the ones that demo well: production quality is measured on ordinary days, not on the launch day.
Keep the check set and the run log somewhere you own. If they live only inside a vendor's product, you lose your baseline the day you change providers, which is the argument I made in own versus rent AI: rent the model, own the workflow, the logs and the evaluations.
LTFI handles AI agent drift by making every step visible, every rule written and every outbound action wait for a person. We built it from scratch as one system, and we run it for each company on its own dedicated server.
Here is what that looks like in practice, from a year and a half of running it in production ourselves:
Brian said something in the webinar that I think about whenever I read a log: agents "are very, very eager to complete their task," and "you also need to tell them to chill out a little bit every now and then." Drift often starts as eagerness. A person at the approval step is how you notice it.
That is the difference between a full system and the lookalikes now arriving, many of which are a chat window or a thin wrapper around someone else's agent. A wrapper can show you an answer. It usually cannot show you which rule, model and tool produced it last month, and that comparison is the whole job of catching drift. What a year and a half of agentic AI in production taught us has the longer story.
If you want to see the log, the checker and the pause switch working, the webinar replay walks through them live, and ltfi.ai has a walkthrough. Writing your own rules first is a good start either way; an AI acceptable use policy for a small business shows what to put in them. We have worked remotely with clients around the world for more than 13 years, so LTFI can be set up wherever you are, and for businesses in the Worcester and Shrewsbury area we are glad to meet in person.
It is an AI agent slowly doing its job differently from how it did when you approved it. The usual causes are a new model version, edited instructions, changed data or tools, and long tasks that lose track of the rules.
Model drift is a change in one model's predictions over time. AI agent drift includes that, plus changes in the rules, tools, data and hand-offs between agents, so an agent can drift even when its model has not changed.
Rerun a fixed set of real test tasks after every model, rule or tool change, and at least once a month. Look at the approval log weekly, because more rewrites before approval is often the first sign.
It helps, especially on a different model from the one doing the work, because it checks each output against written rules. It does not replace a person on anything that leaves the business, such as email, posts or payments.
No. A January 2026 study found identical prompts still produced different outputs at temperature 0.0, and a model update changes behavior regardless of settings. Logs and a check set are the reliable controls.
Only 13% of organizations think they have the right AI agent governance in place, Gartner reported in April 2026. AI governance for agentic AI is six controls built into where agents run: identity, permissions, approvals, audit, data ownership and model choice.
62% of large organizations were building, deploying or developing AI agents in September 2026, KPMG found. After a year and a half of agentic AI in production on LTFI, our lessons are about approval lines, logs, written rules, cost, drift and people.
Only 17% of U.S. firms with fewer than 20 employees used AI in spring 2026. An owner's guide to AI agents for small business: a readiness check, the first jobs and the never list, who owns each agent, day-one governance and the first 90 days.
Work With Us
Kief Studio builds, protects, automates, and supports full-stack systems for businesses up to $50M ARR.
Newsletter
Strategy, psychology, AI adoption, and the patterns that actually compound. No spam, easy to leave.
Subscribe
