Twenty years ago this month, my first post in this blog was "Security Myths and Passwords," addressing the folk wisdom that monthly password rotation improves security. That myth had survived for roughly thirty years before I wrote about it, and NIST did not formally retire the periodic-rotation guidance until 2017, in SP 800-63B. Two decades later, I co-authored a book on the broader phenomenon of cybersecurity myths, Cybersecurity Myths and Misconceptions, with Leigh Metcalf and Josiah Dykstra. The next round of mythmaking is now well underway, around AI in general and large language models (LLMs) in particular.

Once a claim is repeated enough times in policy memos, audit checklists, advertising, and vendor decks, it stops being scrutinized. The 2006 password post made that point. So did my 2019 CERIAS tech report on cloud computing, which warned against decisions driven by "fad" technologies such as cloud, blockchain, and AI. So did my 2023 post "AI and ML Sturm und Drang." Pascal Meunier's post in this blog, from the same week as my first one, "What is Secure Software Engineering?," got at the underlying failure mode: practices "based on experience" are inherently brittle against intelligent adversaries who invent new attacks. That description fits an LLM almost word-for-word.

The marketing claim has become explicit: LLMs will replace software developers, security analysts, compliance reviewers, and incident responders. Some vendors hedge the wording, but the direction is the same: the human becomes an optional component. This is the same type of myth as "change passwords every month" — repeated more often than examined. LLMs provide statistical interpolation based on a fixed training set. They recombine what has been written, and they usually recombine it well. However, they are poor at handling novel cases. A new attack pattern, an unfamiliar architecture, a recent regulation, a business context the training data did not cover — for those, the model produces plausible-sounding text without really addressing the full problem.

LLMs also hallucinate with confidence. They will hallucinate about whether they have followed security-by-design practices in the first place. The sycophancy of current LLMs is well known: an agent told to build a system with security by design will report that it has done so, regardless of whether that is true. Asked to verify its own compliance, an LLM will lie without acknowledging it. Domain expertise, the kind that prevents breaches, is difficult to formalize and slow to acquire. A senior engineer who knows that one system can tolerate a particular failure while another cannot is making a judgment that requires context that no LLM has access to. That expert judgment may take years to build, drawing on incident response, post-mortems, and watching the same problems recur in different forms.

News reports describe thousands of experienced personnel laid off across dozens of companies and replaced with AI. Some of those decisions undoubtedly reflect real reorganization in response to shifting demand. Others appear to use "AI replacement" as a cover for opportunistic cost-cutting. An Oxford Economics analysis found that explicitly AI-cited cuts amounted to roughly 4.5% of U.S. layoffs in the first eleven months of 2025. It concluded that firms may be dressing up layoffs as a good-news story rather than admitting to weak demand or past over-hiring. Either way, the assumption is that AI will fill the gap. That is the patch-instead-of-fix mindset extended to staffing.

It is fair to ask what these tools are good for. The answer is not "nothing." The most valuable thing AI tools currently do in security is an awkward fact for the industry: they are competent at finding flaws in software that should not have been shipped in the first place. The "penetrate-and-patch" culture I and others have been complaining about for decades has produced an enormous backlog of technical debt — code written under deadlines, with corners cut, security analysis skipped, and known limitations punted to the next release. Decades of that, accumulated across the industry, are now coming due, and AI tools are quite effective at locating the rot and kludges.

But there are limits. Much of what an LLM flags is coding flaws rather than security vulnerabilities, and some flaws that could be vulnerabilities turn out not to be exploitable in their deployed context. Telling real risk from noise takes domain expertise and operational experience. An improper buffer length behind validated input is not the same problem as a flagged buffer length on an externally reachable service, and an LLM does not know which is which. That distinction is what the engineers being laid off have spent careers learning to make. Not every smoke alarm is a fire, and not being able to tell the difference means dispatching the fire department hundreds or thousands of times for burnt toast.

There is an irony to all this. Many of the same vendors arguing that AI will replace their security teams are quietly using AI to find the bugs that their previous teams were not given resources or time to fix. That is a real use of the technology. It is not a substitute for the people who would have prevented the bugs to begin with, nor for the people who must understand and triage what the AI surfaces. "Shipped fast, found later, fixed maybe" is the pattern that produced the debt; using AI to keep skating ahead of the consequences without changing the practice is not progress.

Password rotation was mostly wasted effort. The damage was inconvenience, predictable user workarounds, and a generation of policies that pushed people toward weaker, memorable passwords reused across systems. The replacement-by-LLM myth is interacting with a much worse threat landscape, and the recent trends are not encouraging:

• The 2025 Verizon Data Breach Investigations Report found that breaches initiated through exploited vulnerabilities grew 34% year-over-year, that more than half of edge-device vulnerabilities remained unremediated after a full year, and that third-party involvement in breaches doubled, from 15% to 30%.
• IBM's 2025 Cost of a Data Breach Report put the global average breach cost at $4.44 million, with organizations carrying high levels of shadow AI paying an additional $670,000 on average; 97% of organizations that experienced an AI-related security incident lacked proper AI access controls.
• Upguard's State of Shadow AI report found that 81% of the general workforce and 88% of security professionals admit to using unapproved AI tools at work.

Then comes something new in kind. Agentic AI introduces autonomous actors with "write" access inside the perimeter: software that can create, delete, and modify files without a human in the loop. No security architecture I am aware of was designed for that threat model. It certainly isn't zero-trust (whatever that actually means). Treating agents as another category of third-party dependency, given that third-party involvement in breaches has already doubled, is negligent.

The pattern across decades is consistent. Skip the analysis, embed the assumption, repeat it until it counts as common knowledge, and defend the practice long after the evidence has turned against it. The corrective is also consistent: keep experienced humans in the verification loop, demand testable evidence of safety-by-design, resist the pressure to fire the people who carry context in their heads, and treat any system that tells you it is secure as the suspect that it is. The ACM Code of Ethics is explicit on the duty to anticipate and avoid harm. That duty does not pause for hype cycles.

There is also a pragmatic argument. Reckless deployment tends to produce backlash. Nuclear power and commercial aviation are useful precedents. In both cases, preventable incidents led to regulations strict enough to permanently shape how those industries operate, at substantial cost to the firms but with clear benefit to public safety. AI is on that trajectory. Companies that build prudently now will be in a better position when any regulatory wave arrives; companies that race to fire their experts and ship agentic systems into production will find themselves explaining their choices to regulators with enforcement authority.

Quantum computing is already queueing up to be the next myth. Vendors are selling "quantum-safe" and "quantum-ready" products well ahead of any clear definition of either term, and well ahead of any consensus on the threat timeline. We will have a version of this conversation again in five years, and probably every ten years thereafter.

Twenty years from now, I expect somebody — possibly me, possibly an LLM trained on my collected works and confidently misattributing them — will be writing the same post about whatever myth replaces this one.

(Portions of this blog were researched and assembled with the assistance of Anthropic Claude Opus 4.7, but the content is my own.)