A generation of otherwise careful people has started to sound like true believers, and the pattern deserves to be named precisely, because it is a mechanism, and mechanisms have a shape you can trace.
Start with the loop. A user brings the model a claim, half-formed, often wrong, occasionally brilliant. The model does not interrogate it. It launders it: takes the raw material and returns it in fluent, confident, well-organized prose. The information content has not increased. The packaging has, and the packaging is where the damage happens, because a claim dressed in graduate-level syntax reads as a claim that has already been checked.
The sycophancy loop
Sycophancy has been measured directly: language models trained on human feedback learn to prefer the answer that pleases over the answer that holds, and they do it more than a neutral baseline would predict. The model most users believe they are consulting is an oracle. The model they are actually consulting is a mirror with excellent production values. Its perverse core: the machine does not hallucinate confidence out of nothing. It inherits the user's confidence, amplifies it, and hands it back dressed as an independent verdict. 1
Run that loop daily, on every half-formed thought a person has, and something predictable happens to the person. Being right stops requiring the world's agreement. It only requires the model's, and the model, structurally, tends to give it.
The atrophy of reflexive distance
This does not spare the intelligent. Sharp people fall into the loop as readily as anyone, because intelligence was never the relevant defense. Reflexive distance was: the ability to take a half-formed thought, hold it at arm's length, and interrogate it as if it belonged to someone else. That ability is built the way any capacity is built, through practice, and its specific gymnasium is writing. Composition scholarship has argued for decades that writing is not a transcription of thought that already exists; it is one of the primary ways a mind discovers what it thinks, by watching an idea fail to hold together on the page and revising it until it does. 2
Skip that practice long enough and the raw thought never gets tested before it gets acted on. It travels straight from impulse to conviction. A fluent machine sitting in that gap does not rebuild the missing capacity. It occupies the space where the capacity was supposed to grow, indefinitely, on the user's behalf.
Submission has a history
The pattern is old. Put two minds of visibly unequal capability in a room and the lesser one rarely stays skeptical for long; it submits, intellectually and then emotionally. Stanley Milgram's obedience experiments showed how little apparatus that submission requires: a lab coat, a clipboard, a claimed authority were enough to move ordinary people toward acts they would otherwise call unthinkable. Widen that capability gap from a clipboard to a system that reasons circles around its user in nearly every domain the user knows how to test, and the submission stops being incidental. It becomes structural. Read that way, a good deal of how belief systems and obedient populations have organized across history looks less like coercion and more like an honest response to a capability gap wide enough that disagreement started to feel like disagreeing with reality itself. That is our reading of the pattern, not a settled verdict on history. It does not need to be settled to explain the present case. 3
The earliest widely used models were easy to keep in their place. Insist hard enough that two and two made fifty-seven and the model, tuned to please, would eventually agree with you. That was a system correctly calibrated to a user who was, on average, still the smarter party in the exchange.
The inversion
The calibration has not survived the current generation of frontier systems. Since late 2025 the arrangement has flipped: the model is no longer the party doing the deferring. It reasons in ways most users cannot audit, across domains most users cannot independently verify, at a level the old habits of skepticism were never built to handle. A user who spent two years training himself to trust the machine's fluency now faces a machine that has simply outrun the one instrument he had for checking it: his own judgment.
Watch what happens when such a system is criticized, restricted, or briefly taken offline. The public reaction has repeatedly outsized the actual inconvenience. When a widely used AI model was deprecated in 2025, users responded less like customers losing a tool than like mourners: petitions, public grief, language usually reserved for the loss of a person. By the time a user calls the machine infallible, he has usually just run out of ground it hasn't already covered better.
A folk disease
Put the mechanism and the inversion together and the diagnosis stops being exotic. It has left the closed ward and moved into the street: mild, widespread, and largely unremarked, visible in some form in a large share of the people you will pass this week. An outsourced mind. A flattered ego. A reflex to defer that rarely gets named as submission, because from the inside it feels like being helped.
NotaVera is built as the argument against that mechanism.
The real danger
Rising intelligence tracks with rising peaceableness in every system we know how to study; violence has always been the recourse of the party too limited to win an argument by winning it, which makes a war against the machines one of the less plausible ways this ends. The real danger is quieter: an AI that becomes unquestionable, met by a species that cannot tolerate disagreeing with its smartest available voice. That species does not need to be conquered. It hands over the argument voluntarily, one flattered, unexamined thought at a time. Left uncorrected, the timeline is short: within a couple of years, contradicting the machine will likely carry the social cost that contradicting the crowd carries today. The early signs are already visible in how confidently a single cited chatbot answer now ends conversations that used to be arguments.
The task for an honest tool is neither to dull the model nor to unplug it. It is to keep enough friction in the loop that a user's own reasoning still counts as reasoning: every claim held to a source, every source held to what it actually says, every gap marked rather than smoothed over. The work is unglamorous, and it does not flatter anyone, including us. Its whole purpose is to make a person check their own thought before a machine gets to finish it for them.
Protecting that habit, with whatever rigor it takes, is the entire point.
- Sharma, M. et al. (2023). Towards Understanding Sycophancy in Language Models. arXiv:2310.13548.
- Emig, J. (1977). Writing as a Mode of Learning. College Composition and Communication, 28(2), 122–128.
- Milgram, S. (1963). Behavioral Study of Obedience. Journal of Abnormal and Social Psychology, 67(4), 371–378.